AI & ML Quick Path

Maths map

The libraries do the calculations for you, but knowing the idea behind them makes results easier to trust and debug. This page lists the maths behind each lesson. Open the lesson's “Applied Maths - Statistics” section for the main formulas and worked examples.

The main areas

AreaTopics it includesWhere it appears in AI and ML
StatisticsAverages, spread, distributions, sampling, hypothesis tests, percentilesData exploration, evaluation, A/B tests, monitoring
ProbabilityChance of events, conditional probability, Bayes’ theoremClassifier scores, language-model sampling, spam filters
Linear algebraVectors, matrices, dot products, matrix multiplicationEmbeddings, neural networks, attention, recommenders, PCA
CalculusDerivatives, gradients, the chain ruleHow every model is trained (gradient descent, backpropagation)
OptimisationMinimising a loss, learning rates, regularisationTraining ML and deep-learning models
Information theoryEntropy, cross-entropy, KL divergenceClassification and language-model losses, perplexity, drift scores
Graph theoryNodes, edges, neighboursGraph neural networks, knowledge graphs, recommendations

By lesson

Taken from the "Maths behind it" section of every lesson.

LessonPartMaths names
What AI, machine learning, deep learning and generative AI are formulas1 Big picture
  • Statistics and probability (models output probabilities)
  • Optimisation (how a model adjusts itself during training)
  • Linear algebra (how data and model weights are stored)
How an AI or ML project runs from idea to production formulas1 Big picture
  • Sampling and train/test splitting (statistics)
  • Classification metrics: accuracy, recall (statistics)
  • Decision trees and ensembles (algorithms)
Set up your laptop for AI and ML in Python and .NET1 Big picture
  • None for setup. GPUs speed up linear algebra (matrix multiplication), which is the core operation of neural networks.
Data basics: types, features, labels and splits formulas2 Data
  • Sampling and stratified sampling (statistics)
  • Data types: continuous, discrete and categorical variables (statistics)
Collecting and labelling data, with SQL formulas2 Data
  • Sampling bias and representativeness (statistics)
  • Inter-rater agreement: Cohen's kappa (statistics)
  • Relational algebra (the theory behind SQL)
Cleaning and exploring data formulas2 Data
  • Descriptive statistics: mean, median, standard deviation, quantiles
  • Distributions and skewness (statistics)
  • Outlier detection rules such as the interquartile range (statistics)
Feature engineering: turning raw data into model inputs formulas2 Data
  • Standardisation (z-scores) and normalisation (statistics)
  • One-hot encoding as vectors (linear algebra)
  • TF-IDF term weighting (information retrieval)
How a model learns: loss, training and overfitting formulas3 Machine learning
  • Loss functions: mean squared error, cross-entropy
  • Derivatives and gradients (calculus)
  • Gradient descent (optimisation)
  • Bias–variance trade-off (statistics)
Regression: predicting a number formulas3 Machine learning
  • Linear regression and least squares (linear algebra, statistics)
  • Polynomial features
  • Error metrics: MAE, RMSE, R² (statistics)
  • Regularisation: ridge and lasso (optimisation)
Classification: picking a category formulas3 Machine learning
  • Logistic regression and the sigmoid function
  • Softmax (multi-class probabilities)
  • Cross-entropy loss
  • Confusion matrix, precision, recall, F1 (statistics)
The main ML algorithms and when to use each formulas3 Machine learning
  • Linear models and logistic function
  • Distance metrics (k-nearest neighbours)
  • Entropy and Gini impurity (decision trees)
  • Bagging and boosting (ensemble methods)
  • Margins and kernels (support vector machines)
  • Bayes' theorem (naive Bayes)
Unsupervised learning: groups, compression and anomalies formulas3 Machine learning
  • Distance measures and centroids (k-means)
  • Eigenvectors and singular value decomposition (PCA)
  • Isolation and random partitioning (Isolation Forest)
  • Probability distributions and outliers (statistics)
Measuring models honestly formulas3 Machine learning
  • Confusion matrix, precision, recall, F1 (statistics)
  • ROC curve and area under the curve
  • Precision-recall curve and average precision
  • Cross-validation and confidence intervals (statistics)
  • Base rates and Bayes' theorem
Improving and explaining models formulas3 Machine learning
  • Search strategies: grid, random, Bayesian optimisation
  • Regularisation: L1, L2, early stopping
  • Ensembles: bagging, boosting, stacking
  • Shapley values (cooperative game theory)
  • Permutation importance (statistics)
Time-series forecasting formulas3 Machine learning
  • Trend and seasonal decomposition
  • Autocorrelation and lag features
  • Exponential smoothing, ARIMA (time-series statistics)
  • Singular spectrum analysis
  • Prediction intervals (statistics)
Recommender systems formulas3 Machine learning
  • Matrix factorisation and low-rank approximation (linear algebra)
  • Dot products and embeddings
  • Stochastic gradient descent with regularisation (optimisation)
  • Ranking metrics: precision@k, recall@k, NDCG
Neural networks in plain words formulas4 Deep learning
  • Weighted sums and matrix multiplication (linear algebra)
  • Activation functions: ReLU, tanh, sigmoid
  • Chain rule and backpropagation (calculus)
  • Gradient descent and the Adam optimiser (optimisation)
  • Binary cross-entropy loss
Training deep networks well formulas4 Deep learning
  • Stochastic gradient descent, momentum, Adam (optimisation)
  • Learning-rate schedules (warm-up, cosine decay)
  • Regularisation: dropout, weight decay
  • Batch and layer normalisation (statistics)
  • Floating-point formats: FP32, FP16, BF16
Computer vision: classification, detection and segmentation formulas4 Deep learning
  • Convolution and pooling (signal processing)
  • Feature maps and receptive fields
  • Intersection over union, Dice score
  • Attention over image patches (vision transformers)
  • Softmax and cross-entropy
Text and sequences: tokens, embeddings and RNNs formulas4 Deep learning
  • Vectors and cosine similarity (linear algebra)
  • Co-occurrence statistics and pointwise mutual information
  • Singular value decomposition
  • Recurrence and backpropagation through time
Transformers and attention formulas4 Deep learning
  • Dot product and matrix multiplication (linear algebra)
  • Softmax (probability)
  • Scaled dot-product attention
  • Residual connections and layer normalisation
  • Positional encodings (sinusoidal, rotary)
Generative models: autoencoders, GANs and diffusion formulas4 Deep learning
  • Probability distributions and sampling
  • KL divergence (variational autoencoders)
  • Minimax games (GANs)
  • Gaussian noise and Markov chains (diffusion)
  • Score matching
Speech and audio: recognition and text-to-speech formulas4 Deep learning
  • Sampling rate and the Fourier transform (signal processing)
  • Spectrograms and mel scale
  • Edit distance (dynamic programming) for word error rate
  • Attention-based encoder-decoder models
Reinforcement learning formulas4 Deep learning
  • Markov decision processes
  • Bellman equation and value functions
  • Discount factor
  • Policy gradients and the REINFORCE estimator
  • Proximal policy optimisation (PPO)
Graph neural networks: learning from connections formulas4 Deep learning
  • Graph theory: adjacency matrix, degree
  • Normalised adjacency and spectral graph theory
  • Message passing and aggregation
  • Semi-supervised learning
How large language models work formulas5 Language models
  • Probability and conditional probability (next-token prediction)
  • Cross-entropy loss (pretraining objective)
  • Transformers and attention
  • Scaling laws (how quality grows with data and model size)
Tokens and sampling settings formulas5 Language models
  • Byte-pair encoding (compression algorithm)
  • Softmax and temperature scaling (probability)
  • Top-k and nucleus (top-p) sampling
  • Entropy (how spread out the choices are)
Designing a language model formulas5 Language models
  • Parameter counting (linear algebra)
  • Scaling laws and compute-optimal training
  • Attention complexity (grows with the square of context length)
  • Floating-point formats and memory arithmetic
Train a small language model on your laptop formulas5 Language models
  • Cross-entropy and log-likelihood
  • Backpropagation and gradient descent
  • Learning-rate schedules, gradient clipping
  • Perplexity (loss turned into "number of choices")
From laptop to big hardware: scaling training formulas5 Language models
  • Memory arithmetic: parameters × bytes per value
  • Floating-point formats: FP32, BF16, FP16, INT8, 4-bit
  • Gradient accumulation (averaging gradients)
  • Parallel computing: data, tensor and pipeline parallelism
  • Communication cost (all-reduce)
Fine-tuning: SFT, LoRA, QLoRA and preference tuning formulas5 Language models
  • Low-rank matrix factorisation (LoRA)
  • Quantisation to 4-bit (QLoRA)
  • Supervised learning with cross-entropy
  • Preference optimisation (DPO) and reinforcement learning from human feedback (RLHF)
Running language models on your own machine formulas5 Language models
  • Quantization (number formats and rounding)
  • Memory arithmetic: parameters × bits ÷ 8
  • KV cache size (grows with context length and number of users)
Evaluating language models formulas5 Language models
  • Cross-entropy and perplexity (information theory)
  • Accuracy and exact match (statistics)
  • Confidence intervals for scores (statistics)
  • Data contamination checks (n-gram overlap)
Calling an LLM from your application formulas6 AI applications
  • Probability and sampling (temperature)
  • Tokenisation (byte-pair encoding)
  • None needed for validation: it is ordinary software engineering
Prompt engineering that works in production formulas6 AI applications
  • Conditional probability (the prompt is the condition for the next tokens)
  • Sampling and temperature
  • Evaluation statistics: accuracy and confidence intervals
Embeddings and vector search formulas6 AI applications
  • Vectors and vector spaces (linear algebra)
  • Dot product and cosine similarity
  • Approximate nearest-neighbour search (HNSW graphs)
  • Contrastive learning (how embedding models are trained)
RAG: answering from your own documents formulas6 AI applications
  • Embeddings and cosine similarity (linear algebra)
  • Nearest-neighbour search
  • Information retrieval metrics: recall@k, precision@k
  • Thresholds and calibration (statistics)
Advanced RAG: chunking, hybrid search, re-ranking and evaluation formulas6 AI applications
  • BM25 (probabilistic information retrieval)
  • Cosine similarity of embeddings
  • Reciprocal rank fusion
  • Cross-encoder re-ranking
  • Retrieval metrics: recall@k, mean reciprocal rank
Tools and agents with human approval formulas6 AI applications
  • State machines and workflow graphs (computer science)
  • Exponential backoff for retries
  • Idempotency (distributed systems)
  • No special maths: agents are mostly careful software engineering
AI frameworks and MCP: what to use in Python and .NET6 AI applications
  • None: frameworks are software architecture. The maths lives in the models they call.
Multimodal AI apps: images, documents and voice formulas6 AI applications
  • Shared embedding spaces and contrastive learning
  • Convolutional and transformer image encoders
  • Optical character recognition
  • Spectrograms (for audio)
Evaluating AI applications formulas6 AI applications
  • Accuracy and agreement rates (statistics)
  • Hypothesis tests and p-values (A/B testing)
  • Confidence intervals and sample size
  • Inter-rater agreement (judge versus humans)
Security for AI applications formulas6 AI applications
  • None required. Security here is threat modelling and software engineering.
  • Regular expressions (formal languages) for pattern-based redaction
Responsible AI: fairness, privacy, transparency and oversight formulas6 AI applications
  • Group rates and differences (statistics)
  • Fairness metrics: demographic parity, equal opportunity, equalised odds
  • Differential privacy (privacy-preserving statistics)
  • Shapley values (explanations)
"Generative AI and Agentic AI: the whole picture" formulas6 AI applications
  • Probability (next-word choice, chance of a chain of steps working)
  • Basic arithmetic for token and cost estimates
  • Statistics for testing: counting how many example tasks succeed
Serving a model as an API formulas7 Production
  • Percentiles (p50, p95) for latency
  • Queueing theory (why latency rises sharply as a server gets busy)
"MLOps: versioning, pipelines and safe releases" formulas7 Production
  • Cryptographic hash functions (fingerprinting data)
  • Hypothesis testing and statistical significance (A/B tests)
  • ROC AUC (comparing candidates)
Monitoring models and AI apps in production formulas7 Production
  • Population stability index and Kullback-Leibler divergence (drift)
  • Kolmogorov-Smirnov test (comparing distributions)
  • Percentiles (latency)
Speed, cost and scaling formulas7 Production
  • Quantisation (number representation, rounding error)
  • Knowledge distillation
  • Amdahl's law and queueing theory (limits of scaling)
Production architecture for AI systems formulas7 Production
  • Queueing theory and capacity planning
  • Probability of failure (availability of chained services)
  • Exponential backoff