- Learn
- Glossary
Glossary
1362 terms defined in the lessons, each linked to where it is taught.
1
- 1x1 convolution
- A convolution with a 1 by 1 kernel: a learned weighted mix of channels at each pixel, used to reduce or expand the number of channels cheaply. In: Landmark CNNs: LeNet to ResNet
A
- A*
- Best-first search that expands the frontier state with the smallest f(n) = g(n) + h(n): cost so far plus estimated cost to go. In: Informed search and A*
- A/A tests
- An experiment where both groups get the identical version; any 'significant' difference is a false positive. Used to validate the testing system. In: Offline vs online evaluation and A/B tests
- A/B test
- A controlled experiment that randomly splits live traffic between two versions and compares a pre-chosen metric, so the difference can be attributed to the change. In: Online evaluation and monitoring , Offline vs online evaluation and A/B tests
- acceptance rate
- In speculative decoding, the probability that the target model accepts a token proposed by the draft model. It equals the sum over tokens of min(p, q), so it is high when the two models agree. In: Speculative decoding and other speedups
- access token
- A secret string that proves who you are to the Hugging Face Hub. Scripts send it instead of a password. It has a role: read, write or fine-grained. In: Accounts, tokens, downloads and datasets
- Accuracy
- The share of all predictions that are correct: (TP + TN) / all cases. In: Classification metrics
- ACID
- Atomicity, Consistency, Isolation, Durability: the four guarantees of database transactions. In: Transactions and ACID
- action
- A choice the agent makes at one step, from a discrete set (move left) or a continuous range (apply 0.3 newtons). In: RL fundamentals and MDPs
- action value
- The expected return from taking action a in state s and following the policy afterwards, written Q(s, a). Choosing the action with the highest Q gives a greedy policy. In: RL fundamentals and MDPs
- Activation checkpointing
- Saving only some layers' activations in the forward pass and recomputing the rest during the backward pass, trading extra compute for much less memory. In: GPUs and distributed training
- activation function
- The function a neuron applies to its weighted sum, such as a step, sigmoid or ReLU. It decides the neuron's output. In: The perceptron and the artificial neuron , Activation functions
- activations
- Intermediate outputs of each layer saved during the forward pass because the backward pass needs them to compute gradients. They grow with batch size and sequence length. In: GPUs and distributed training
- Active learning
- A loop where the current model picks which unlabeled examples a human should label next, usually the ones it is least sure about. In: Data collection and labeling , Design a spam / content moderation classifier
- activity
- In a workflow engine, a single unit of real work with side effects, such as calling an API or writing to a database; it can fail, time out and be retried. In: Workflow engines
- Actor-critic
- A reinforcement learning architecture with two learned parts: an actor (the policy) that chooses actions and a critic (a value function) whose estimates supply the advantage used to update the actor. In: Policy gradients and actor-critic, PPO
- AdaBoost
- Adaptive Boosting: trains weak classifiers in sequence, increasing the weight of misclassified examples after each round, and combines them with a weighted vote. In: Boosting: AdaBoost to gradient boosting
- Adam
- Adaptive moment estimation: an optimizer that combines momentum (a running mean of gradients) with RMSProp-style scaling (a running mean of squared gradients), plus a bias correction for the first steps. In: Optimizers: SGD, momentum, RMSProp, Adam, AdamW
- AdamW
- Adam with decoupled weight decay: the weights are shrunk directly (w -= lr * wd * w) instead of adding an L2 term to the gradient, so every weight decays at the intended rate. In: Optimizers: SGD, momentum, RMSProp, Adam, AdamW
- additive attention
- Attention whose score is a small feed-forward network applied to the decoder state and an encoder state, as in Bahdanau et al. 2014. In: Seq2seq and attention
- additive model
- Writing a time series as trend plus seasonal plus remainder, appropriate when the seasonal swings stay about the same size as the level changes. In: Time series fundamentals
- ADF test
- Augmented Dickey-Fuller test: a hypothesis test whose null hypothesis is that a series has a unit root (is non-stationary in the random-walk sense). In: Stationarity and classical models
- admissible
- A heuristic that never overestimates the true cheapest cost from a state to the goal: h(n) is at most h*(n) for every n. In: Informed search and A*
- advantage
- In reinforcement learning, how much better an action turned out than expected. Group-relative methods use the reward minus the average reward of other attempts at the same problem. In: Reasoning models and test-time compute , Policy gradients and actor-critic, PPO
- adversarial example
- An input changed by a small, deliberately chosen perturbation so that a model gives a wrong output, usually with high confidence, while a person sees no meaningful change. In: Robustness and adversarial examples
- Adversarial search
- Search in a setting with an opponent whose choices work against yours, as in two-player games. In: Adversarial search: minimax and alpha-beta
- Adversarial training
- Training on adversarial examples generated against the current model at each step, alongside or instead of clean examples, so the model learns to resist them. In: Robustness and adversarial examples
- agent
- Anything that perceives its environment through inputs and acts on it through outputs, ideally choosing actions that best achieve its goals. In: What is artificial intelligence? , Agents and environments , The agent loop (ReAct)
- agent-computer interface
- The set of commands, their inputs and the feedback they return, designed for a model as the user rather than a person. In: Coding agents and computer-use agents
- agentic system
- Any system in which language models and tools work together over several steps, whether the steps are fixed in code or chosen by the model. In: Workflows vs agents: the patterns
- Agglomerative clustering
- Bottom-up hierarchical clustering: every point starts as its own cluster and the two closest clusters are merged at each step. In: Hierarchical clustering and DBSCAN
- aggregate
- A cluster of domain objects treated as one unit for changes, accessed only through its root, which enforces the cluster's invariants; one aggregate is saved in one transaction. In: Domain-driven design for architects
- AI Act
- The European Union's law on artificial intelligence, in force since 1 August 2024, which sorts AI systems into risk tiers with obligations that apply in stages through 2028. In: AI regulation and governance
- AI winters
- A period when interest and funding for AI research collapsed after expectations were not met. The main ones were in the mid 1970s and the late 1980s to early 1990s. In: A short history of AI
- Alignment
- The problem of making an AI system pursue the goals its designers and users actually intend, not just the objective that was written down or measured. In: Alignment and interpretability, the big picture , Seq2seq and attention
- alignment tax
- Any loss of capability on some tasks caused by alignment training such as RLHF. Methods try to keep it small. In: The LLM training pipeline
- all-reduce
- A collective operation where every worker contributes a buffer and every worker ends up with the sum (or average) of all of them. In: GPUs and distributed training
- alpha
- A LoRA hyperparameter that scales the adapter's output by alpha / r, controlling how strongly the adapter changes the layer. In: Parameter-efficient fine-tuning: LoRA and QLoRA
- Alpha-beta pruning
- A version of minimax that skips branches which cannot affect the final decision, using two bounds: alpha (the best MAX can guarantee so far) and beta (the best MIN can guarantee so far). In: Adversarial search: minimax and alpha-beta
- Alternating least squares
- Alternating least squares: fix all item vectors and solve each user vector exactly as a small ridge regression, then fix users and solve items, and repeat. In: Collaborative filtering and matrix factorization
- alternative hypothesis
- The claim that there is a real effect, accepted only if the data is surprising enough under the null. In: Hypothesis testing and p-values
- anchor boxes
- Preset reference boxes of several sizes and aspect ratios placed at every feature-map location; the detector predicts offsets from them instead of raw coordinates. In: Object detection
- anomaly
- A data point that differs markedly from the bulk of the data, often signalling an error, fraud, a fault or a rare event. Also called an outlier. In: Anomaly detection
- anomaly detection
- Finding data points that do not fit the pattern of the rest, usually without labelled examples of anomalies. In: Anomaly detection
- anti-corruption layer
- A translation layer between a new system and a legacy or external one, converting the old model into the new model so legacy concepts do not leak into new code. In: Migrating architectures: the strangler fig
- Anti-entropy
- A background process that compares replicas and repairs any differences, instead of waiting for reads to find them. In: Design a distributed key-value store
- anycast
- Announcing one IP address from many locations, so the network routes each user to the nearest one. In: DNS in depth , Global traffic: GeoDNS, anycast and GSLB
- API gateway
- A server that is the single entry point for client requests to many backend services, handling routing and cross-cutting concerns such as authentication and rate limiting. In: API gateway and backend-for-frontend
- approval gate
- A point in an agent's run where a proposed action waits for a person to approve or reject it before it executes. In: Coding agents and computer-use agents
- Approximate nearest neighbour
- Approximate nearest neighbour search: finding vectors that are very likely, but not guaranteed, to be the closest to a query, by searching only a small, well-chosen part of the data. In: Vector databases and ANN indexes
- approximate nearest-neighbour (ANN) index
- An index that finds most of the nearest vectors to a query while scanning only a small part of the collection, trading a little recall for large speedups. In: Two-tower retrieval and embedding-based retrieval
- Apriori
- An algorithm for frequent itemsets that grows them one item at a time and discards any candidate with an infrequent subset, because a superset can never be more frequent than its subsets. In: Association rules and market basket analysis
- Arc consistency
- A binary constraint from X to Y is arc consistent if every value left in X's domain has at least one compatible value in Y's domain. AC-3 enforces this for every constraint. In: Constraint satisfaction problems
- ARIMA
- AutoRegressive Integrated Moving Average: an ARMA model fitted to a series differenced d times, written ARIMA(p, d, q). In: Stationarity and classical models
- arithmetic intensity
- FLOPs performed per byte moved from memory. Below the hardware's ratio of compute to bandwidth, work is memory-bound; above it, compute-bound. In: How LLM inference works
- Artificial general intelligence
- A hypothetical AI with broad, flexible competence across almost any intellectual task a human can do. No agreed definition or test exists. In: What is artificial intelligence?
- Artificial intelligence
- The field of building computer systems that perceive, reason, learn, decide or act in ways we would call intelligent if a person did them. In: What is artificial intelligence? , What is machine learning?
- artificial neuron
- A simple computing unit that multiplies each input by a weight, adds the results and a bias, and passes the total through an activation function. In: The perceptron and the artificial neuron
- as-of join
- A join that, for each row with a timestamp, takes the most recent feature value at or before that timestamp, so no row sees values from its future. In: Feature stores and training-serving skew
- ASR
- Automatic speech recognition: turning recorded or live speech into text. Also called speech-to-text (STT) or transcription. In: Speech: ASR and TTS
- Association rule mining
- Finding rules of the form 'if a basket contains X, it tends to contain Y' from many transactions, measured by support, confidence and lift. In: Association rules and market basket analysis
- asynchronously
- Not waiting for the result: the caller hands off the work and continues; the result arrives or is checked later. In: Why async? Queues explained
- At-least-once
- Each message is processed one or more times: it is never lost, but failures and retries can cause duplicates. In: Delivery semantics
- at-least-once delivery
- A delivery guarantee where every message arrives one or more times; duplicates are possible but losses are not. In: Idempotency and safe retries
- At-most-once
- Each message is processed zero or one times: it may be lost on failure but is never duplicated. In: Delivery semantics
- attention
- A mechanism that lets a model compute, for each output step, a weighted average of all input positions, with weights from a softmax over learned relevance scores. In: Seq2seq and attention
- Attribute-based access control (ABAC)
- Attribute-based access control: access decisions evaluate rules over attributes of the user, the resource, the action and the context, such as department, region or time. In: Authentication vs authorization
- Authentication
- Verifying who a user or service is, for example with a password, passkey, certificate or single sign-on. In: Authentication vs authorization
- Authorization
- Deciding what an authenticated user or service is allowed to do, such as reading a document or deleting an account. In: Authentication vs authorization
- Autocorrelation
- The correlation between a time series and a copy of itself shifted by k steps (lag k); plotted for many lags it is the ACF. In: Time series fundamentals
- autoencoder
- A neural network trained to reproduce its input through a narrow middle layer, forcing it to learn a compressed representation of the data. In: Anomaly detection , Autoencoders
- autograd
- Automatic differentiation: software that records the operations performed on tensors and applies the chain rule in reverse to compute exact gradients. In: Autograd and your first PyTorch training loop
- Automatic differentiation
- Software that records the operations of a computation and applies the chain rule automatically to produce exact gradients. Reverse mode, used for neural networks, is backpropagation. In: Backpropagation
- autoregressive
- A model that predicts a value as a linear combination of the series' own previous p values plus noise. In: Stationarity and classical models
- Autoscaling
- Automatically changing the number or size of instances in response to load or another metric. In: Autoscaling
- Availability
- The fraction of time, or of requests, for which a system works correctly. Often written as a number of nines, such as 99.9 percent. In: Availability, reliability and the "nines"
- availability zones (AZs)
- One or more data centers in a cloud region with independent power, cooling and networking, close enough to other zones in the region for fast, synchronous communication. In: Redundancy, failover and multi-AZ
- Average pooling
- Pooling that keeps the mean of each window, smoothing the feature map. In: Pooling and CNN architecture
- Average precision
- Average precision: the area under a class's precision-recall curve, computed from detections ranked by confidence. In: Object detection
- Axial coding
- Grouping open-coding notes into a small set of named categories, so they can be counted. In: Error analysis
B
- B-tree
- A balanced tree whose nodes are disk pages holding many sorted keys; every leaf is at the same depth, so any key is found in a few page reads. In: Indexes and B-trees
- Back-of-the-envelope estimation
- Rough calculations, done quickly with rounded numbers, that give the right order of magnitude for traffic, storage, bandwidth or machines. In: Back-of-the-envelope estimation
- backbone
- The feature-extracting part of a network, such as the convolution and pooling layers of a CNN, before the task-specific head. In: Pooling and CNN architecture
- backend-for-frontend
- An API layer built for one specific client type, such as the iOS app or the web app, shaping and aggregating backend data for that client's screens. In: API gateway and backend-for-frontend
- Backpressure
- A mechanism by which a slow downstream stage signals upstream stages to slow down, so work is not produced faster than it can be processed. In: Backpressure and flow control
- backpropagation
- The algorithm that computes the gradient of the loss for every weight by applying the chain rule backward through the network. In: Derivatives and the chain rule , Backpropagation
- backpropagation through time
- Backpropagation applied to an RNN unrolled over the sequence; because the weights are shared, the gradient for each weight is the sum of its contributions from every time step. In: Recurrent neural networks
- backtest
- Evaluating a forecasting method by replaying history: train on data up to a past date, forecast the following period, compare with what happened, then move the date forward and repeat. In: Backtesting and forecast evaluation
- Backtracking search
- Depth-first search for CSPs that assigns one variable at a time, checks the constraints immediately, and undoes the most recent choice when a variable has no consistent value left. In: Constraint satisfaction problems
- Backward chaining
- Inference that starts from a goal and works backwards, finding rules whose conclusion matches the goal and trying to prove their conditions as subgoals, depth first. In: First-order logic
- bag of words
- Representing text as counts of each word in a vocabulary, ignoring word order. In: Feature engineering , Bag of words and TF-IDF
- Bagging
- Bootstrap aggregating: train many copies of a model on different bootstrap samples of the data and average (or vote on) their predictions. In: Bagging and random forests
- bandwidth
- How many bytes per second a link or device can move once data is flowing. In: Latency numbers every engineer should know
- base model
- A model straight out of pretraining, before any fine-tuning. It continues text rather than following instructions. In: The LLM training pipeline , Stacking and blending
- base rate
- The share of a group that truly belongs to the positive class; also called prevalence. In: Fairness and bias
- base rate neglect
- Judging a probability from evidence while ignoring how common the thing was to begin with. In: Probability fundamentals
- baseline
- A simple reference result, such as a rule or a very simple model, that a new model must beat to be worth shipping. In: The ML project lifecycle , Learning curves and baselines , Policy gradients and actor-critic, PPO
- basis function
- A fixed function applied to the raw inputs to create new features, such as a power, a sine wave or a bump around a point. A linear model on basis functions can fit non-linear shapes. In: Polynomial regression and basis functions
- batch
- A group of examples processed together in one forward pass. Stacking inputs as rows of a matrix turns many matrix-vector products into one matrix-matrix product. In: Forward pass by hand
- Batch normalization
- A layer that standardizes each feature to zero mean and unit variance using statistics of the current mini-batch, then applies a learned scale and shift. Uses running averages at inference time. In: Weight initialization and normalization
- Bayes' theorem
- P(A given B) = P(B given A) times P(A) divided by P(B). It flips a conditional probability and updates a prior belief with evidence. In: Probability fundamentals
- Bayesian network
- A directed acyclic graph whose nodes are random variables and whose edges show direct influence, with a conditional probability table at each node giving its distribution for each combination of parent values. In: Bayesian networks
- Bayesian optimization
- A tuning strategy that fits a cheap model of score as a function of the hyperparameters and uses it to choose the most promising setting to try next. In: Hyperparameter tuning
- Beam search
- Search that keeps only the k best partial solutions at each step, expanding all of them and again keeping the best k. With k = 1 it is greedy. In: Local search and optimization , Decoding and sampling
- beginner
- No background needed beyond basic programming and using the web. In: How to use this track
- Bellman equation
- The recursive equation linking a state's value to the values of the states its actions lead to: V(s) = max over actions of the expected reward plus discounted value of the next state. In: Sequential decisions: MDPs (bridge to RL)
- benchmarks
- A public, fixed set of test questions with automatic scoring, used to compare models on the same task. In: Benchmarks and their limits
- best effort
- Delivery with no guarantee: the network tries, but packets may be lost, duplicated, delayed or reordered. In: Networking basics: IP, TCP, UDP and ports
- bi-encoder
- A model that embeds each text independently into a vector; texts are compared by vector similarity, so documents can be embedded once in advance. In: Sentence and document embeddings , Hybrid search and reranking
- bias
- A systematic difference in how a model treats people or describes groups that is not justified by the task, for example different answers to the same request depending on a name, gender or age. In: Bias and fairness in LLMs , Linear regression intuition , Bias, variance, underfitting and overfitting , The perceptron and the artificial neuron , Fairness and bias
- bias correction
- In Adam, dividing the running averages by (1 - beta^t) so they are not biased towards zero in the first steps, when they start from zero. In: Optimizers: SGD, momentum, RMSProp, Adam, AdamW
- biases
- Per-user and per-item offsets in a factorization model that capture generous raters and well-liked items, independent of taste. In: Collaborative filtering and matrix factorization
- BIC
- Bayesian information criterion: minus twice the log-likelihood plus the number of parameters times the log of the number of points. Lower is better. In: Gaussian mixture models and EM
- bidirectional
- Two RNNs read the sequence in opposite directions and their states are concatenated, so each position sees both left and right context. In: LSTM and GRU
- bin
- Replacing each continuous feature by the index of the bucket (bin) it falls in, typically 255 buckets, so split search scans bins instead of every distinct value. In: XGBoost, LightGBM and CatBoost in practice
- binary cross-entropy
- The loss -[y log p + (1 - y) log(1 - p)] averaged over examples. Also called log loss. It punishes confident wrong predictions very heavily. In: Logistic regression
- Binning
- Turning a numeric feature into ranges, such as age 18 to 25, 26 to 40 and so on, then treating each range as a category. In: Feature engineering
- birthday paradox
- The surprising fact that collisions among random values become likely once you have about the square root of the number of possible values. In: Design a URL shortener
- blameless postmortem
- A written review after an incident covering what happened, its impact, how it was mitigated, contributing causes and follow-up actions, written without assigning blame to individuals. In: Incident response and postmortems
- blast radius
- The share of users, customers or functionality affected when something fails. In: Cell-based architecture and shuffle sharding
- Blending
- A simpler form of stacking: base models train on one part of the data and the meta-model trains on their predictions for a separate holdout part. In: Stacking and blending
- blocks
- A call that does not return until the operation finishes, so the thread making it can do nothing else in the meantime. In: Processes, threads and concurrency
- bloom filter
- A compact probabilistic structure that answers 'definitely not present' or 'possibly present' for a key, with no false negatives. In: Storage engines: B-tree vs LSM-tree , Probabilistic data structures , Design a web crawler
- Blue-green
- Running two complete environments, switching all traffic from the current one (blue) to the new one (green) at once, and keeping blue available for instant rollback. In: Deployment strategies
- BM25
- A keyword ranking function that scores a document by query-term frequency, saturating repeated terms and normalizing for document length, weighted by how rare each term is (IDF). In: Design a search ranking system
- Bonferroni correction
- Divide the significance level by the number of tests, so the chance of any false positive across all of them stays near alpha. In: Hypothesis testing and p-values
- Boosting
- An ensemble method that trains models sequentially, each one concentrating on the errors left by the ensemble so far, and adds them together. In: Boosting: AdaBoost to gradient boosting
- bootstrap
- Estimating uncertainty by resampling your own data with replacement many times and recomputing the statistic each time. In: Statistics: sampling, CLT and confidence intervals
- bootstrap sample
- A sample of n rows drawn with replacement from a dataset of n rows; some rows appear several times and about 37 percent do not appear at all. In: Bagging and random forests
- bottleneck
- The narrowest layer of an autoencoder; its size limits how much information the code can carry. In: Autoencoders
- bounded contexts
- A part of the business with its own consistent model and vocabulary, within which each term has one precise meaning. In: Domain-driven design for architects
- bounding box
- The smallest axis-aligned rectangle that encloses an object, stored as four numbers. In: Object detection
- box plot
- A compact plot of a distribution: a box from the first to the third quartile with a line at the median, whiskers for the typical range, and dots for outliers. In: Exploratory data analysis
- Bradley-Terry
- A model of pairwise comparisons in which the probability that A beats B is the sigmoid of the difference in their scores. The standard loss for reward models. In: RLHF
- branching factor
- The number of successors each state has on average. A grid with four moves has a branching factor of about 4. In: Uninformed search: BFS, DFS, uniform cost
- Brier score
- The mean squared difference between predicted probabilities and the 0/1 outcomes. Lower is better; it rewards both calibration and discrimination. In: Probability calibration
- broadcasting
- NumPy's rule for combining arrays of different shapes by stretching the smaller one across the larger, without copying data. In: Your toolkit: Python, NumPy, pandas, scikit-learn, PyTorch
- broker
- A Kafka server that stores partitions and serves producers and consumers. In: Kafka internals: the distributed log
- budget
- A target limit for one resource, such as seconds of latency or dollars per request, split across the parts of the system so each part has its own share. In: Latency and cost budgets
- bulkhead
- Isolating resources such as thread pools or connection pools per dependency, so that one failing dependency cannot consume the resources needed by others. In: Circuit breakers and bulkheads
- burn rate
- How fast the error budget is being consumed relative to a steady pace that would exactly use it up by the end of the window. A burn rate of 1 is on track; 14 means it would be gone in about 2 days of a 30-day window. In: SLIs, SLOs, SLAs and error budgets , Monitoring and alerting
- business metric
- The outcome the product exists to move, such as revenue, retained customers or bookings, usually measured with an online experiment. In: Framing ML problems for products
- byte-pair encoding
- A tokenization method that starts from single characters or bytes and repeatedly merges the most frequent adjacent pair into a new token, until the vocabulary reaches a target size. In: Tokenization
C
- C2PA
- Coalition for Content Provenance and Authenticity: an open standard for attaching signed provenance records (content credentials) to media files. In: Video and audio generation overview
- cache
- The folder where Hugging Face libraries keep downloaded files, by default ~/.cache/huggingface/hub, shared by transformers, datasets, diffusers and the hf CLI. In: Accounts, tokens, downloads and datasets , What is system design? , Why caching works
- cache avalanche
- A burst of cache misses that overloads the backing store, often after a mass invalidation or a cluster change. In: Consistent hashing
- Cache invalidation
- Removing or updating cached copies when the underlying data changes, so readers stop seeing old values. In: Cache invalidation and consistency
- cache key
- The parts of a request a cache uses to decide whether two requests ask for the same object, typically the URL plus selected headers. In: Content Delivery Networks
- cache stampede
- Many requests missing the cache for the same key at once and all hitting the backing store to recompute it, overloading it. In: Thundering herd, cache stampede and hot keys
- calibrated
- A model is calibrated when, among all cases where it predicts probability p, the outcome happens about p of the time. In: Probability calibration
- canary
- A release where a small share of users gets the new version while monitoring compares it with the old one, before the rest of users get it. In: LLMOps: shipping and iterating
- canary deployment
- Releasing a new version to a small share of real traffic first, comparing its metrics with the current version, and widening the share only if it holds up. In: Model serving patterns
- canary release
- Releasing a new version to a small percentage of traffic first, comparing its metrics with the current version, and widening the rollout only if it is healthy. In: Deployment strategies
- candidate generation
- Another name for retrieval: producing the candidate set that later stages rank. In: Two-tower retrieval and embedding-based retrieval
- capacity
- How flexible a model is: how many different patterns it can represent. More parameters, deeper trees or higher polynomial degree mean more capacity. In: Bias, variance, underfitting and overfitting
- Capacity planning
- Forecasting future demand and making sure enough resources are in place ahead of it, with headroom for spikes and failures. In: Capacity planning and load testing
- cardinality
- The number of distinct time series a metric produces, determined by the number of unique combinations of its label values. In: Logs, metrics and traces , Data types and datasets
- CART
- Classification and Regression Trees: the standard greedy algorithm that grows binary trees by choosing the impurity-minimising split at each node. In: Decision trees
- cascade
- A setup in which a cheap model answers first and the request is escalated to a stronger, more expensive model only when a check says the cheap answer is not good enough. In: Routing, cascades and cost optimization
- catastrophic forgetting
- When training a model on new data makes it lose abilities or knowledge it had before. In: Supervised fine-tuning and instruction tuning
- CatBoost
- A gradient boosting library from Yandex with built-in handling of categorical features through ordered target statistics and symmetric trees. In: XGBoost, LightGBM and CatBoost in practice
- Categorical
- Data that names a group with no natural order, such as city, colour or product type. In: Data types and datasets
- categorical cross-entropy
- The loss -log p of the probability the model gave to the true class, averaged over examples. The multi-class version of binary cross-entropy. In: Multi-class and multi-label classification
- causal mask
- A mask that sets the attention scores from each token to all later tokens to minus infinity, so their softmax weights become 0 and a token can only attend to itself and earlier tokens. In: Self-attention, step by step , Build a tiny GPT from scratch
- CDF
- Cumulative distribution function: the probability that the variable is less than or equal to a value. In: Probability fundamentals
- cell
- A complete, independent copy of a service stack, with its own compute and data, serving a fixed subset of customers. In: Cell-based architecture and shuffle sharding
- cell state
- The LSTM's separate memory vector, updated only by elementwise scaling (forget) and addition (input), so information and gradients can pass many steps unchanged. In: LSTM and GRU
- Cell-based architecture
- An architecture that splits a system into many independent, complete copies called cells, each serving a subset of customers, with a thin routing layer that sends each customer to its cell. In: Cell-based architecture and shuffle sharding
- central limit theorem
- For large enough samples, the mean of independent draws is approximately normally distributed, whatever the shape of the original distribution. In: Statistics: sampling, CLT and confidence intervals
- centred moving average
- The average of the values in a window around each point; a window of exactly one season length cancels the seasonal pattern. In: Time series fundamentals
- centroid
- The centre of a cluster: the mean of the points assigned to it. In: k-means clustering
- certificate
- A signed document binding a domain name to a public key, issued by a certificate authority and valid for a limited time. In: TLS and HTTPS
- certificate authority
- An organization trusted by browsers and operating systems to verify domain ownership and sign certificates. In: TLS and HTTPS
- chain of thought
- Prompting a model to write out intermediate reasoning steps before its final answer, which often improves accuracy on multi-step problems. In: Chain of thought and step-by-step reasoning
- chain rule
- The derivative of a composition equals the product of the derivatives of its parts: dy/dx = dy/du times du/dx. In: Derivatives and the chain rule , Backpropagation
- change data capture
- Change data capture: reading a database's log of committed changes and publishing them as a stream of events. In: Cache invalidation and consistency
- change data capture (CDC)
- Change data capture: reading a database's own change log (such as the PostgreSQL WAL or MySQL binlog) to stream every committed row change as an event. In: Transactional outbox and change data capture
- changepoints
- A date at which a piecewise trend is allowed to change its slope; Prophet places many candidates and shrinks most of their changes to zero. In: Prophet and modern forecasting libraries
- channel
- One layer of values in an image tensor, such as the red, green or blue intensities, or one feature map inside a network. In: Images as tensors
- Chaos engineering
- The practice of deliberately injecting failures into a system, under controlled conditions, to verify that it keeps behaving acceptably and to find weaknesses before they cause real outages. In: Chaos engineering
- character-level
- A language model whose tokens are single characters, so the vocabulary is tiny and no tokenizer training is needed. Common for teaching. In: Build a tiny GPT from scratch
- chat template
- The exact text format, with special tokens marking roles and turn boundaries, that a chat model was trained on. Prompts must use the same template at inference. In: Supervised fine-tuning and instruction tuning
- choreography
- Coordinating a saga by having each service react to other services' events, with no central controller. In: Sagas: choreography and orchestration
- chunks
- A passage cut from a document, typically a few hundred tokens long, that is embedded, indexed and retrieved as one unit in a RAG system. In: Document ingestion and chunking
- CI
- Continuous integration: an automated pipeline that builds and tests every proposed change before it can be merged. In: LLMOps: shipping and iterating
- circuit breaker
- A wrapper around calls to a dependency that tracks failures and, once they pass a threshold, stops sending calls for a while and fails immediately instead. In: Circuit breakers and bulkheads
- circuits
- A connected set of features and the computations between them that together carry out a specific behaviour, such as a step of reasoning. In: Alignment and interpretability, the big picture
- Classical planning
- Finding a sequence of actions that turns an initial state into one satisfying a goal, where states are sets of facts and actions are described by preconditions and effects. In: Classical planning
- Classification
- Supervised learning where the label is a category, such as spam or not spam, or which digit an image shows. In: Types of learning
- classifier
- A model that reads a piece of text and returns a label or a score, for example the probability that the text breaks a given policy. In: Guardrails
- Classifier-free guidance
- A diffusion sampling trick that runs the denoiser with and without the prompt and moves further in the direction of the difference, trading variety for prompt adherence. In: Image generation , Diffusion models
- click model
- A probabilistic model of how users scan and click a results list, used to separate the effect of position from the item's real relevance. In: Design a search ranking system
- Clustering
- Grouping data points so that points in the same group are more similar to each other than to points in other groups, without using labels. In: k-means clustering
- CNN
- Convolutional neural network: a network built mainly from convolution layers, usually with pooling or strided convolutions, followed by layers that make the final prediction. In: Pooling and CNN architecture
- coding agent
- An agent whose tools let it search and read a codebase, edit files and run commands such as tests, iterating until a task is done. In: Coding agents and computer-use agents
- Cohen's kappa
- An agreement score for two labelers that subtracts the agreement expected by chance: 1 is perfect, 0 is no better than chance. In: Data collection and labeling
- coherent
- Forecasts across a hierarchy are coherent when every aggregate equals the sum of the forecasts beneath it. In: Design a demand forecasting system
- cold start
- The extra latency when a serverless platform has to create a new execution environment for a function, rather than reusing a warm one. In: Serverless and event-driven compute
- cold-start problem
- The problem of recommending for new users or new items that have little or no interaction history. In: Recommender fundamentals , Cold start, diversity and feedback loops
- Collaborative filtering
- Recommending items based on the behaviour of many users: people who agreed in the past tend to agree again, so similar users' choices predict yours. In: Recommender fundamentals
- Colour jitter
- Randomly changing an image's brightness, contrast, saturation or hue during training so the model becomes robust to lighting and camera differences. In: Data augmentation
- command
- A request asking a specific service to do something, which it may accept or reject, such as 'PlaceOrder' or 'ChargeCard'. In: Event-driven architecture and event sourcing
- Compaction
- Summarising a conversation that is nearing the context limit and continuing in a fresh context that starts from the summary plus the most recent turns. In: Context engineering
- compensating action
- An action that semantically undoes the effect of an earlier committed step, such as a refund for a charge or a cancellation for a booking. In: Sagas: choreography and orchestration
- computational graph
- A graph whose nodes are simple operations (add, multiply, activation) and whose edges carry values. Frameworks record it during the forward pass so they can run it backwards. In: Backpropagation
- computer-use agent
- An agent that operates graphical software the way a person does: it looks at screenshots and replies with mouse and keyboard actions. In: Coding agents and computer-use agents
- Concept drift
- A change in the relationship between inputs and outcomes, P(y given x): the same input now leads to a different outcome, so the model's learned mapping is wrong. In: Monitoring, drift and retraining
- Concurrency
- Making progress on many tasks during overlapping periods of time, for example by switching to another request while one waits for I/O. In: Processes, threads and concurrency
- conditional probability table
- A table at each node of a Bayesian network giving the probability of each of its values for every combination of its parents' values. In: Bayesian networks
- conditionally independent
- Two features are conditionally independent given the class if, once you know the class, learning one feature tells you nothing more about the other. In: Naive Bayes
- confidence
- For a rule X -> Y, the share of baskets containing X that also contain Y: support(X and Y) / support(X). In: Association rules and market basket analysis
- confidence interval
- A range computed from a sample by a procedure that captures the true value in a stated share of repeated samples, such as 95 percent. In: Statistics: sampling, CLT and confidence intervals
- conflict-driven clause learning
- Conflict-driven clause learning: when a SAT solver hits a conflict, it derives a new clause explaining it, so the same mistake is never repeated, and can jump back several levels at once. In: Propositional logic
- conflict-free replicated data types
- Conflict-free replicated data types: data structures such as counters, sets and text whose concurrent updates always merge to the same result, regardless of order. In: Multi-leader and leaderless replication
- confusion matrix
- A table counting true positives, false positives, false negatives and true negatives for a classifier at one threshold. In: Classification metrics
- conjunctive normal form
- Conjunctive normal form: an AND of clauses, where each clause is an OR of literals (propositions or their negations). In: Propositional logic
- Consensus
- Getting a group of nodes to agree on a single value, or a sequence of values, even when some nodes fail or messages are lost. In: Consensus: Raft
- consistency models
- A contract describing which values reads may return given the order of writes, from strict (linearizable) to loose (eventual). In: PACELC and consistency models
- consistent
- A heuristic for which h(n) is at most c(n, n') + h(n') for every action from n to n'. Consistent heuristics are also admissible. In: Informed search and A*
- consistent prefix reads
- A guarantee that readers see writes in the order they happened, never an effect before its cause. In: Replication lag and read consistency guarantees
- constitution
- In Constitutional AI, the written list of rules or principles that guides the model's self-critiques and the AI's preference judgments. In: Constitutional AI and RLAIF
- Constitutional AI
- An alignment method that trains a model using a written list of principles: the model critiques and revises its own outputs against them, and AI judgments based on them replace human harm labels. In: Constitutional AI and RLAIF
- constrained decoding
- Restricting which tokens a model may generate at each step, by masking tokens that cannot lead to output matching a grammar or schema, so the output is guaranteed to have the required structure. In: Structured outputs
- constraint
- A restriction on which parameter values are allowed, such as all weights being non-negative. In: Optimization basics
- constraint graph
- A graph with one node per variable and an edge between every two variables that share a constraint. In: Constraint satisfaction problems
- constraint satisfaction problems
- A problem defined by variables, a domain of possible values for each, and constraints that restrict which combinations of values are allowed. A solution assigns every variable a value that satisfies all constraints. In: Constraint satisfaction problems
- consumer
- A program or service that reads messages from a queue and processes them. In: Why async? Queues explained
- consumer group
- A set of consumers sharing one group id; Kafka assigns each partition of a topic to exactly one consumer in the group so they share the work. In: Kafka internals: the distributed log
- Consumer lag
- How far a consumer group is behind: the newest offset in a partition minus the group's committed offset. In: Kafka internals: the distributed log
- container
- A packaged, isolated process that bundles an application with its libraries and settings, so it runs the same way on any machine with a container runtime. In: Containers and Kubernetes for system designers
- contamination
- When test questions or close copies of them appear in a model's training data, inflating its score. In: Benchmarks and their limits
- content delivery network
- Content Delivery Network: a global network of edge servers that cache content close to users and serve it on behalf of your origin servers. In: Content Delivery Networks
- Content-based filtering
- Recommending items whose features resemble those of items the user liked, using item attributes rather than other users' behaviour. In: Recommender fundamentals
- Context engineering
- Deciding, at every model call, which tokens go into the limited context window: instructions, tool definitions, conversation history, retrieved documents and intermediate results, and what to compress or leave out. In: Context engineering
- context propagation
- Passing trace identifiers along with a request, usually in headers, so that downstream services can attach their spans to the same trace. In: Distributed tracing
- context switch
- The operating system pausing one thread and resuming another on a CPU core, saving and restoring their state. It costs a few microseconds of pure overhead. In: Processes, threads and concurrency
- context vector
- The weighted average of encoder states, using the attention weights, that the decoder receives at one output step. In: Seq2seq and attention
- context window
- The maximum number of tokens a model can take into account at once: the prompt plus everything it has generated so far. In: Context windows and the KV cache
- contextual bandit
- A bandit whose best arm depends on observed features (the context), such as the user's history or device, so the strategy learns a model from context to reward per arm. In: Multi-armed bandits
- Contextual retrieval
- Prepending a short, model-written description of where a chunk sits in its document (title, section, what it is about) before embedding and keyword-indexing it. In: Contextual retrieval and query transformation
- Continuous batching
- Scheduling at the granularity of single decode steps: after every step, finished requests leave the batch and waiting requests join it. Also called iteration-level scheduling or in-flight batching. In: Batching and PagedAttention , Design an LLM chat service (ChatGPT-like)
- contrastive learning
- Training that pulls the embeddings of matching pairs (a question and its answer, two paraphrases) together and pushes non-matching pairs apart. In: Sentence and document embeddings
- ControlNet
- A method that adds spatial conditions such as edge maps, depth maps or human poses to a pretrained text-to-image diffusion model by training a copy of its encoder while the original stays frozen. In: Image generation
- convex
- Shaped like a single bowl: the straight line between any two points on the curve never goes below it, so there is one global minimum and no other dips. In: Optimization basics , Gradient descent, the engine of ML
- convolution
- An operation that slides a small grid of weights (the kernel) across an input and computes a weighted sum at each position, producing a feature map. In: Convolutions
- cooldown
- A minimum time between scaling actions in one direction, so the fleet does not shrink and grow repeatedly in response to brief changes. In: Autoscaling
- coordination service
- A small, strongly consistent, replicated key-value store, such as ZooKeeper or etcd, used by other systems for configuration, leader election, locks and membership. In: Coordination services (ZooKeeper, etcd)
- coordinator
- The process that runs a two-phase commit: it collects votes from participants and decides and announces commit or abort. In: Two-phase commit
- copy-on-write
- Sharing one copy of data between several users until one of them needs to modify it, at which point that user gets a private copy. In: Batching and PagedAttention
- core point
- In DBSCAN, a point with at least minPts points (itself included) within distance eps. In: Hierarchical clustering and DBSCAN
- Correlation
- Pearson correlation r: a number from -1 to 1 measuring how well two columns fit a straight line. 0 means no linear relationship, not no relationship. In: Exploratory data analysis
- cosine similarity
- The cosine of the angle between two vectors: their dot product divided by the product of their lengths. 1 means same direction, 0 unrelated, -1 opposite. In: Sentence and document embeddings , Vectors and vector spaces
- cost per unit
- Total infrastructure cost divided by the number of requests, users or transactions served, so efficiency can be compared over time and across services. In: Cloud cost engineering
- Cost-complexity pruning
- Pruning that removes subtrees whose impurity reduction is smaller than a penalty alpha times their number of leaves; alpha is chosen by cross-validation. In: Decision trees
- Count-Min Sketch
- A small grid of counters with one hash function per row that estimates how often each item appeared in a stream; it can over-count but never under-count. In: Probabilistic data structures
- counterfactual evaluation
- An evaluation that sends pairs of inputs identical except for one attribute, such as a name or a stated age, and measures how much the output changes. In: Bias and fairness in LLMs
- counterfactual explanation
- An explanation of the form: if your inputs had been these instead, the decision would have been different; usually the smallest such change. In: Explainability
- covariance matrix
- A d by d matrix whose entry (j, k) is the average product of centred features j and k. The diagonal holds each feature's variance; off-diagonal entries show how features move together. In: Principal Component Analysis
- coverage
- The fraction of actual outcomes that fall at or below a predicted quantile; a well calibrated P90 has coverage near 90 percent in every slice. In: Design an ETA prediction system
- covering index
- An index that includes every column a query needs, so the query is answered from the index without reading the table rows. In: Indexes and B-trees
- CQRS
- Command Query Responsibility Segregation: using one model for writes (commands) and one or more separate models, optimized per query, for reads. In: Denormalization, materialized views and CQRS
- critical path
- The chain of operations in a trace that determined the total latency; speeding up anything off this path does not make the request faster. In: Distributed tracing
- cross features
- Features that depend on a specific user and item together, such as how many times this user clicked this seller before; available to rankers but not to two-tower retrieval. In: Two-tower retrieval and embedding-based retrieval
- cross-attention
- Attention in which queries come from one sequence (the decoder) and keys and values from another (the encoder output), letting the output consult the input. In: Encoder, decoder and decoder-only models
- cross-encoder
- A model that reads a query and a document together and outputs a relevance score. More accurate than a bi-encoder but must run once per pair. In: Sentence and document embeddings , Hybrid search and reranking
- cross-entropy
- The negative log of the probability the model assigned to the correct answer, averaged over examples. 0 means certain and right; large means confident and wrong. In: Pretraining: next-token prediction , Information theory for ML , Loss functions
- cross-validation
- Rotating which part of the data is held out, training several times, and averaging the scores, so every row is used for both training and validation. In: Train, validation and test splits , Cross-validation
- Crowdsourcing
- Splitting labeling work across many paid workers on a platform, usually with several workers per item and quality checks. In: Data collection and labeling
- curse of dimensionality
- The collection of problems that appear as the number of features grows: data becomes sparse, distances between points become similar, and far more data is needed to learn. In: Feature selection
- Cursor pagination
- Paging by remembering the last item seen, such as its id or timestamp, and asking for items after it, instead of skipping a number of rows. In: Pagination patterns
- Cutout
- An augmentation that blanks out a random square region of the input image during training. In: Data augmentation
D
- d-separation
- A graphical test for whether two variables in a Bayesian network are independent given some evidence: they are if every path between them is blocked by the chain, fork and collider rules. In: Bayesian networks
- DAG
- Directed acyclic graph: tasks joined by one-way dependencies with no cycles, so there is always a valid order to run them in. In: Training pipelines and orchestration
- Data augmentation
- Creating modified copies of training examples (flips, crops, noise, paraphrases) that keep the label, so the model sees more variety and learns invariances. In: Regularization for deep nets , Data augmentation
- Data drift
- A change in the distribution of a model's inputs, P(x), such as more mobile users or a new price range, while the relationship between inputs and outcomes may stay the same. In: Monitoring, drift and retraining
- data leakage
- When information that would not be available at prediction time ends up in the training features, making offline results look far better than reality. In: Exploratory data analysis , Data leakage
- data minimisation
- The principle that personal data should be adequate, relevant and limited to what is necessary for the purpose it is processed for (GDPR Article 5). In: Privacy and PII handling
- Data parallelism
- Training with a full copy of the model on every GPU, each processing a different slice of the batch, and averaging gradients across GPUs every step. In: GPUs and distributed training
- Data pipelines
- Scheduled or streaming jobs that collect raw data, clean and validate it, and compute features and labels from it. In: ML system architecture overview
- DataFrame
- pandas' table type: named columns, each one a typed array, with an index for the rows. In: Your toolkit: Python, NumPy, pandas, scikit-learn, PyTorch
- dataloader
- A per-request helper that collects the keys resolvers ask for during one tick and loads them all in a single batched query, also caching repeats. In: GraphQL
- DBSCAN
- Density-Based Spatial Clustering of Applications with Noise: groups points that have many neighbours within a radius eps, and labels isolated points as noise. In: Hierarchical clustering and DBSCAN
- DCG
- Discounted cumulative gain: the sum over ranked positions of each item's gain divided by a discount that grows with the position, so relevant items count more near the top. In: Learning to rank
- DCN
- Deep and Cross Network: a model that adds a cross network, whose layers explicitly multiply the input with the current layer to build feature interactions of increasing degree, alongside a regular deep network. In: Design an ad click-through-rate prediction system
- dead letter queue (DLQ)
- A separate queue or topic where messages go after they fail processing too many times, so they can be inspected, fixed and replayed without blocking the main flow. In: Retries, dead letter queues and poison messages
- dead ReLU
- A ReLU neuron whose input is negative for every training example, so its output and gradient are always zero and it stops learning. In: Activation functions
- deadline
- An absolute time by which a call must finish. gRPC passes it along to downstream calls, which can give up early once it has passed. In: gRPC and Protocol Buffers
- decision boundary
- The surface in feature space where a classifier switches from predicting one class to the other. In: Logistic regression
- decision network
- A Bayesian network extended with decision nodes (choices) and utility nodes (scores), used to compute the action with the highest expected utility and the value of information. In: Decision theory and utility
- Decision theory
- The theory of choosing actions under uncertainty by combining probabilities of outcomes with utilities that measure how much each outcome is wanted. In: Decision theory and utility
- decision tree
- A model that predicts by asking a sequence of yes-or-no questions about the features, following one branch per answer until it reaches a leaf that holds the prediction. In: Decision trees
- Decode
- The second phase of generation: one token at a time, each step reading all weights and the whole KV cache to produce a single token. Usually limited by memory bandwidth. In: Context windows and the KV cache , How LLM inference works
- decoder
- The part of an autoencoder that maps a code back to the input space, producing a reconstruction. In: Autoencoders
- decoder-only
- A transformer that generates text left to right with causal self-attention, each token seeing only earlier tokens. GPT-style models, including most modern chat assistants, are decoder-only. In: Encoder, decoder and decoder-only models
- decoding
- The procedure that turns a model's next-token probabilities into an actual output token, step by step: greedy, beam search, or sampling with settings such as temperature, top-k and top-p. In: Decoding and sampling
- decoupling
- Arranging components so that a change, slowdown or failure in one does not directly affect the other. In: Why async? Queues explained
- Deep learning
- Machine learning with neural networks of many layers, which learn their own features from raw data such as pixels, audio or text. In: What is artificial intelligence? , What is machine learning?
- deep Q-network
- A Q-learning agent whose action values come from a deep neural network that takes the state (for example, stacked game frames) as input and outputs one Q-value per action. In: Deep Q-Networks
- DeepAR
- An autoregressive recurrent network trained on many related series that outputs a probability distribution for each future step (Salinas et al., 2017). In: ML and deep learning for forecasting
- delivery semantics
- The guarantee a messaging system gives about how many times a message is delivered or processed: at most once, at least once, or exactly once. In: Delivery semantics
- Demographic parity
- Fairness criterion requiring the positive decision rate to be the same in every group, regardless of the true outcomes. In: Fairness and bias
- dendrogram
- A tree diagram of hierarchical clustering. Leaves are data points; the height of each join is the distance at which two clusters were merged. In: Hierarchical clustering and DBSCAN
- denoising autoencoder
- An autoencoder trained to reconstruct the clean input from a deliberately corrupted copy of it, which forces it to learn the structure of the data. In: Autoencoders
- Denormalization
- Deliberately storing duplicated or precomputed data so reads need fewer joins or calculations, at the cost of more work on writes. In: Denormalization, materialized views and CQRS
- Dense retrieval
- Retrieval by nearest neighbours in a learned vector space, where the query and items are embedded by models trained so that relevant pairs are close. In: Design a search ranking system
- dense vector
- A vector whose entries are mostly non-zero real numbers, so information is spread across all dimensions; the opposite of a sparse one-hot vector. In: From symbols to vectors
- deployer
- Under the EU AI Act, an organisation that uses an AI system under its own authority in a professional activity. In: AI regulation and governance
- Deployment strategies
- The method used to replace a running version of software with a new one, such as rolling, blue-green or canary, which determines how much traffic a bad release can affect and how fast it can be undone. In: Deployment strategies
- depth
- The number of layers in a network. In: Multi-layer networks
- derivative
- How fast a function's output changes as its input changes: the slope of the function at a point. In: Derivatives and the chain rule
- determinant
- A number that says how a matrix scales area (or volume). Zero means space is flattened and the matrix has no inverse; negative means it is flipped. In: Matrices and linear transformations
- Deterministic simulation
- Running a whole distributed system inside one process where the network, disks, clocks and scheduling are simulated and driven by a single seeded random generator, so any run can be replayed exactly. In: Testing distributed systems
- device
- Where a tensor's memory and computation live, such as the CPU or a particular GPU. Operations require all their tensors on the same device. In: Autograd and your first PyTorch training loop
- Dictionary learning
- A technique that learns a large set of directions so that each activation vector can be written as a sparse sum of a few of them; in interpretability it is often trained as a sparse autoencoder. In: Alignment and interpretability, the big picture
- Differencing
- Replacing each value of a time series by its change from the previous value (or from the same season one period earlier), which removes trends or seasonal patterns. In: Stationarity and classical models
- differential privacy
- A mathematical guarantee that the output of an analysis or training run is almost equally likely whether or not any single person's data was included; the strength is set by epsilon (and delta). In: Privacy-preserving ML
- Diffie-Hellman key exchange
- A technique letting two parties derive the same secret by exchanging public values; someone watching the exchange cannot compute the secret. In: TLS and HTTPS
- diffusion models
- A generative model that learns to reverse a gradual noising process: starting from pure noise, it removes a little noise at each step until an image (or audio, or video) appears. In: Image generation , Diffusion models
- diffusion transformer
- A diffusion model whose denoising network is a transformer over patches instead of a convolutional U-Net. In: Video and audio generation overview
- dimension
- The number of entries in a vector, which is the number of axes of the space it lives in. In: Vectors and vector spaces
- direct
- A forecasting strategy that trains a model to predict h steps ahead directly from information available now, rather than feeding its own one-step forecasts back in. In: Design a demand forecasting system
- Direct preference optimisation
- Direct preference optimisation: a method that trains a language model directly on chosen and rejected response pairs with a classification-style loss, skipping the separate reward model and RL loop. In: DPO and other preference methods
- discount factor
- A number gamma between 0 and 1 that multiplies each later reward once more, so a reward k steps away counts gamma^k as much as one now. In: RL fundamentals and MDPs
- discriminator
- The network in a GAN that outputs the probability that its input came from the real data rather than the generator. In: Generative adversarial networks
- Distillation
- Training a smaller student model to imitate a larger teacher model's outputs, so that it keeps much of the teacher's skill at a fraction of the cost. In: Distillation and synthetic data
- distributed cache
- A cache that runs as its own service, shared by all application servers over the network, usually spread across several nodes. In: Distributed caches: Redis and Memcached
- distributed denial of service
- Distributed denial of service: many machines sending traffic at once to exhaust a target's network, servers or application so real users cannot get through. In: DDoS protection and abuse prevention
- distributed lock
- A lock held through a shared service so that only one process across many machines does a piece of work, or touches a resource, at a time. In: Distributed locks and fencing tokens
- Distributed tracing
- Recording the path of a single request through all the services it touches, as a tree of timed operations, so you can see where its time was spent. In: Distributed tracing
- distribution
- A description of how likely each possible value of a random variable is, as a PMF for discrete values or a PDF for continuous ones. In: Distributions you will meet
- Distribution shift
- A difference between the data a model was trained on and the data it sees in use, in the inputs, the labels, or the relationship between them. In: Robustness and adversarial examples
- distributional hypothesis
- The idea that words appearing in similar contexts tend to have similar meanings, the principle behind learned word embeddings. In: From symbols to vectors , Word embeddings
- distributional shift
- The mismatch between the actions in the logged data and the actions a new policy would take; value estimates for rarely logged actions are unreliable and often too optimistic. In: RL in the real world
- document frequency
- The number of documents in the collection that contain a term at least once. In: Bag of words and TF-IDF
- document-term matrix
- A matrix with one row per document and one column per vocabulary term; each entry counts, or weights, that term in that document. In: Bag of words and TF-IDF
- Domain Name System
- Domain Name System: the internet's distributed directory that turns names such as example.com into IP addresses and other records. In: DNS in depth
- Domain randomization
- Training in simulation with randomly varied physics, sensing or appearance in every episode, so the policy must work across all of them and treats the real world as one more variation. In: RL in the real world
- Domain-driven design (DDD)
- An approach to software design that models the business domain closely, dividing it into bounded contexts with their own models and language, developed with domain experts. In: Domain-driven design for architects
- dominates
- h2 dominates h1 when h2(n) is at least h1(n) for every state n. An A* search with the dominating admissible heuristic never expands more states. In: Informed search and A*
- dot product
- Multiply two vectors entry by entry and add the results. Large and positive when they point the same way, zero when perpendicular, negative when opposite. In: Vectors and vector spaces
- double-entry bookkeeping
- A bookkeeping method where every movement of value is recorded as two entries, a debit to one account and an equal credit to another, so the books always balance. In: Reconciliation and correctness jobs
- Downsampling
- Replacing high-resolution data points with aggregates over larger time buckets, such as 5-minute minimum, maximum, sum and count, to save storage for older data. In: Time-series data
- DP-SGD
- Differentially private stochastic gradient descent: clip each example's gradient to a maximum norm, add Gaussian noise to the sum, then take the step; a privacy accountant tracks the total epsilon. In: Privacy-preserving ML
- Dropout
- A regularizer that sets each hidden unit's output to zero with probability p on every training step, so the network cannot rely on any single unit. All units are used at test time. In: Regularization for deep nets
- dual-write problem
- Writing the same change to two systems (such as a database and a message broker) as separate steps, which can leave them inconsistent if a failure happens between the two writes. In: Transactional outbox and change data capture
- durability
- Stored data is not lost once it has been acknowledged as written. In: Availability, reliability and the "nines"
- durable execution
- Execution of ordinary-looking code whose progress is persisted, so it survives process crashes, restarts and long waits. In: Workflow engines
- Dynamic batching
- A model-server feature that holds incoming requests for a short time and runs them together as one batch, trading a little latency for much higher throughput. In: Model serving patterns
E
- E4M3
- An 8-bit floating-point encoding with 4 exponent bits and 3 mantissa bits: more precision, less range. Usually used for weights and activations at inference. In: Quantization for inference
- early stopping
- Monitoring validation error during training and stopping when it stops improving, keeping the best number of rounds. In: Boosting: AdaBoost to gradient boosting , Regularization for deep nets
- Early stopping in tuning
- Stopping an unpromising tuning trial before it finishes, based on its intermediate validation score, to spend the budget on better candidates. In: Hyperparameter tuning
- edge cases
- Unusual inputs at the boundaries of normal use, where systems are most likely to break. In: Building an eval set
- eigenvalue
- The factor by which a matrix stretches its eigenvector. Negative means the direction flips. In: Eigenvectors, eigenvalues and SVD
- eigenvectors
- A non-zero vector that a matrix only stretches or shrinks, without changing its direction: A v = lambda v. In: Eigenvectors, eigenvalues and SVD
- Elastic net
- Regularization that mixes L1 and L2 penalties, getting lasso's sparsity with ridge's stability on correlated features. In: Regularization: ridge, lasso, elastic net
- embarrassingly parallel
- Work that can be split into independent pieces with no communication between them, so it scales almost perfectly across cores or machines. In: Bagging and random forests
- embedding model
- A model that maps a piece of text to a fixed-length vector so that texts with similar meaning get vectors that are close together. In: Embedding and indexing
- embedding retrieval
- Retrieving items whose learned embedding is near the user's embedding, using a two-tower model and an approximate nearest neighbour index. In: Design a feed ranking system
- embedding table
- A learned matrix with one row per vocabulary token; looking up a token id returns its row, the token's embedding vector. In: Embeddings and positional encoding
- embeddings
- Lists of numbers that represent words, images or other items so that similar things end up close together. In: The map of modern AI , From symbols to vectors , Vector search in system design , Feature engineering
- encoder
- The part of an autoencoder that maps an input to its compact code. In: Autoencoders
- encoder-decoder
- A transformer with an encoder that reads the full input bidirectionally and a decoder that generates the output causally while attending to the encoder's outputs through cross-attention. T5 and the original transformer are examples. In: Encoder, decoder and decoder-only models , Image segmentation
- encoder-only
- A transformer whose tokens attend in both directions over the whole input. It produces a contextual vector per token and is used for understanding tasks such as classification and embeddings. BERT is the classic example. In: Encoder, decoder and decoder-only models
- entails
- KB entails alpha (written KB |= alpha) when alpha is true in every model in which KB is true. In: Propositional logic
- Entity linking
- Deciding which entity in a knowledge graph a mention in text refers to, such as mapping 'Paris' in a sentence to the French capital rather than the city in Texas. In: Knowledge graphs and ontologies
- Entropy
- The average surprise of a distribution: minus the sum of p times log p over all outcomes. High when outcomes are evenly spread, zero when one outcome is certain. In: Information theory for ML , Decision trees
- Envelope encryption
- Encrypting data with a data key and then encrypting that data key with a master key held in a key management service, storing only the encrypted data and the wrapped data key. In: Encryption, keys and secrets
- epoch
- One complete pass of the training algorithm over the whole training dataset. In: Gradient descent, the engine of ML , Autograd and your first PyTorch training loop
- eps
- In DBSCAN, the radius of the neighbourhood around each point. In: Hierarchical clustering and DBSCAN
- equal opportunity
- Fairness criterion requiring only the true positive rate to be equal across groups: qualified people are approved at the same rate whatever their group. In: Fairness and bias
- Equalized odds
- Fairness criterion requiring the true positive rate and the false positive rate to be equal across groups; equal opportunity requires only equal true positive rates. In: Fairness and bias
- erasure coding
- Splitting data into fragments plus extra parity fragments, so the original can be rebuilt from any sufficient subset; durable like replication with far less extra storage. In: Object storage, block storage and file systems
- Error analysis
- Reading real outputs and traces, noting what went wrong in each, grouping the notes into failure categories and counting them to decide what to fix. In: Error analysis
- error budget
- The amount of unreliability an SLO permits in a window, such as 0.1 percent of requests over 30 days. Teams spend it on releases and experiments. In: SLIs, SLOs, SLAs and error budgets
- ETA
- Estimated time of arrival: a prediction of how long a trip, pickup or delivery will take, shown to users and used internally for matching, pricing and planning. In: Design an ETA prediction system
- Euclidean distance
- The straight-line distance between two points: the square root of the sum of squared differences across features. In: k-Nearest Neighbours
- eval
- A set of test inputs plus a way to judge each output, run automatically so you get a score you can compare across versions. In: Why evals are the core skill
- eval gate
- An automated check in CI that runs the eval suite on a proposed change and blocks the merge if quality drops below a set threshold. In: LLMOps: shipping and iterating
- eval harness
- The code that runs evaluations end to end: it loads test cases, runs the system under test on each, grades the outputs, stores the results and reports them. In: Capstone: build your own eval harness
- evaluation function
- A function that estimates the value of a non-final game position, such as a weighted sum of material and positional features. In: Adversarial search: minimax and alpha-beta
- event
- A record of something that has already happened, named in the past tense, such as 'OrderPlaced'; it cannot be rejected, only reacted to. In: Event-driven architecture and event sourcing
- event history
- The durable log a workflow engine keeps for each workflow run, recording every activity scheduled and completed, timer and signal, used to rebuild the workflow's state by replay. In: Workflow engines
- event sourcing
- Storing every change to an entity as an immutable event in an append-only log, and deriving current state by replaying those events. In: Event-driven architecture and event sourcing
- event stream
- A durable, append-only, ordered log of events that consumers read by position (offset); messages are kept for a retention period whether or not they have been read. In: Message queues vs pub/sub vs event streams
- event time
- The time at which an event actually occurred, as recorded in the event itself, as opposed to when it was processed. In: Stream processing
- eviction policy
- The rule a full cache uses to choose which entry to remove to make room for a new one. In: Eviction policies
- evidence lower bound
- Evidence lower bound: a quantity that is always at most log p(x) and can be estimated and optimized; maximizing it trains both the encoder and the decoder of a VAE. In: Variational autoencoders
- Exactly-once
- Each message affects the final result exactly one time, even if it was physically delivered more than once. In: Delivery semantics
- expand and contract
- Making a breaking change in safe steps: first add the new structure alongside the old (expand), migrate traffic and data to it, then remove the old structure (contract). In: Zero-downtime schema and data migrations
- expectation-maximization
- Expectation-maximization: an iterative algorithm for models with hidden variables that alternates estimating the hidden variables given the parameters (E-step) and the parameters given those estimates (M-step). In: Gaussian mixture models and EM
- expected utility
- The probability-weighted average of an action's utilities over all its possible outcomes. A rational agent chooses the action that maximizes it. In: Decision theory and utility
- Experience replay
- Storing past transitions (state, action, reward, next state) in a buffer and training on random minibatches drawn from it, instead of on each transition as it happens. In: Deep Q-Networks
- Experiment tracking
- Recording, for every training run, the code version, data version, parameters, metrics and output files, so runs can be compared, searched and reproduced. In: Experiment tracking and reproducibility
- expert capacity
- The maximum number of tokens an expert accepts per batch, usually the even share times a capacity factor. Tokens over the limit skip the expert. In: Mixture of experts
- expert systems
- Programs that encode a human expert's knowledge as many if-then rules to give advice in a narrow domain, such as diagnosing infections or configuring computers. In: A short history of AI , Expert systems and rule engines
- experts
- One of the parallel feed-forward networks inside a mixture-of-experts layer. Each token is processed by only the few experts the router selects. In: Mixture of experts
- Explainability
- The degree to which people can understand why a model produced its outputs, either overall or for a single case. In: Explainability
- explained variance ratio
- The fraction of the data's total variance that lies along one principal component. Summed over the kept components, it tells you how much spread the reduced data retains. In: Principal Component Analysis
- Explicit feedback
- Feedback a user gives on purpose, such as a thumbs up or down, a star rating or a written comment. In: Online evaluation and monitoring , Recommender fundamentals
- exploding gradients
- When gradients grow exponentially as they are propagated back through many layers, causing huge, unstable updates and often NaN losses. In: Vanishing and exploding gradients, residual connections
- exploration-exploitation trade-off
- The tension between using the option that currently looks best (exploit) and trying other options to learn whether they are better (explore). In: Multi-armed bandits
- Exploratory data analysis
- Looking at a dataset's summary numbers and plots before modelling, to understand it and to find bugs, oddities and leaks. In: Exploratory data analysis
- explored set
- The set of states a graph search has already expanded, so it never expands them again. In: Problem formulation and state spaces
- exponential backoff
- Waiting longer after each failed attempt, typically doubling the delay, often with random jitter so many clients do not retry in sync. In: Retries, dead letter queues and poison messages , Timeouts, retries, backoff and jitter
F
- F1 score
- The harmonic mean of precision and recall, 2PR / (P + R). It is high only when both are high. In: Classification metrics
- fact
- A single statement taken to be true, such as 'Paris is the capital of France', usually stored as a relation between named things. In: Knowledge representation
- failure detector
- A component that decides, from missing or late messages, which nodes are probably down. In: Membership, gossip and failure detection
- fairness through unawareness
- Trying to make a model fair by leaving the protected attribute, such as race or sex, out of its inputs. It fails when other features act as proxies. In: Fairness and bias
- faithfulness
- Whether every claim in an answer is supported by the retrieved context, rather than invented or taken from the model's memory. In: Evaluating RAG
- false positive
- An answer of 'yes, present' for an item that was never actually inserted. In: Probabilistic data structures
- false positive rate
- The share of real negatives that the model wrongly flags: FP / (FP + TN). In: Classification metrics
- fan-out
- How many peers each node contacts per gossip round. In: Membership, gossip and failure detection
- fast gradient sign method
- FGSM: an attack that adds epsilon times the sign of the loss gradient with respect to the input, the best single step under a max-norm limit if the loss were linear. In: Robustness and adversarial examples
- Fault injection
- Deliberately introducing failures (killed processes, network partitions, delays, clock skew, disk errors) to see how a system behaves under them. In: Testing distributed systems
- feature cross
- A feature that is the combination of two or more features, such as AND(user_interest=running, ad_category=shoes), letting a linear model learn interaction effects. In: Design an ad click-through-rate prediction system
- Feature engineering
- Turning raw data into model inputs (features) that make the patterns you care about easy for a model to find. In: Feature engineering
- feature extraction
- Using a frozen pretrained network purely to compute features, and training only a new head on top of them. In: Transfer learning and fine-tuning
- feature flag
- A runtime switch, read from configuration, that decides which users get which version of a feature, without a new deploy. In: LLMOps: shipping and iterating , Deployment strategies
- Feature hashing
- Mapping category values to a fixed number of columns with a hash function. Handles huge or unbounded vocabularies; unrelated values sometimes collide. In: Feature engineering
- Feature importance
- A score for how much a model relies on a feature, either overall (global) or for one prediction (local). In: Feature importance and SHAP
- feature map
- The output grid of a convolution: one value per kernel position, high where the kernel's pattern appears in the input. In: Convolutions
- Feature selection
- Choosing a subset of the available features to train on, dropping those that add noise, cost or risk without improving predictions. In: Feature selection
- feature store
- A system that stores feature values computed from shared definitions, serving full history to training (offline store) and the latest values to live predictions (online store). In: ML system architecture overview , Feature stores and training-serving skew
- features
- A direction in a model's activation space that tends to be active for one human-interpretable concept and can be read out or amplified. In: Alignment and interpretability, the big picture , What is machine learning?
- federated averaging
- The standard federated learning algorithm: sampled clients run several local SGD epochs and the server takes a data-size-weighted average of their models. In: Privacy-preserving ML
- Federated learning
- Training a shared model across many devices or organisations that keep their data locally and send only model updates to a coordinating server. In: Privacy-preserving ML
- feed-forward network
- A small two-layer neural network applied to each token's vector separately: widen (often 4x), apply a nonlinearity, narrow back. Also called the MLP. In: The transformer block
- feedback loop
- A loop where a model's outputs shape the data it is later trained on, so early choices and biases reinforce themselves. In: Cold start, diversity and feedback loops , Design a feed ranking system
- fencing token
- A number that increases with every lock grant; the protected resource rejects requests carrying a token lower than the highest it has already accepted. In: Distributed locks and fencing tokens
- few-shot prompting
- Including a few worked examples of inputs and their desired outputs in the prompt, so the model infers the task, format and standard from them. With no examples it is called zero-shot. In: Few-shot prompting and examples
- filter bubble
- A popular term for a user seeing a narrowing range of content because personalization keeps showing more of what they engaged with before. In: Cold start, diversity and feedback loops
- Fine-tuning
- Continuing to train some or all of a pretrained network's weights on the target task, usually with a much smaller learning rate. In: Transfer learning and fine-tuning
- finite difference
- Checking a hand-derived gradient by nudging a weight up and down by a tiny amount and measuring how the loss changes. In: Build a tiny GPT from scratch
- First-order logic
- A logic whose sentences talk about objects, the relations between them and functions on them, with variables and the quantifiers 'for all' and 'there exists'. In: First-order logic
- FlashAttention
- An exact attention algorithm that processes queries, keys and values in tiles that fit in fast on-chip memory, using a running softmax so the full score matrix is never written to main GPU memory. In: Speculative decoding and other speedups
- Focal loss
- A variant of cross-entropy that down-weights easy, well-classified examples so training focuses on hard ones; common in object detection. In: Loss functions
- folds
- One of the k equal parts the data is split into for cross-validation. In: Cross-validation
- forecast horizon
- How far ahead a forecast is made, for example 1 day or 14 days after the last observed data point. In: Backtesting and forecast evaluation
- forecast origin
- The last time point whose data a forecast may use; the forecast starts right after it. In: Backtesting and forecast evaluation
- foreign key
- A column that stores the primary key of a row in another table, linking the two, and that the database can check actually exists. In: Relational databases and SQL
- forget gate
- The LSTM gate that scales the previous cell state entry by entry, deciding what to keep and what to erase. In: LSTM and GRU
- forward algorithm
- Dynamic programming over the trellis that sums the probabilities of all hidden paths, giving the total probability of an observation sequence in time proportional to T times N squared. In: Hidden Markov models
- Forward chaining
- Inference that starts from the known facts and repeatedly applies rules whose conditions are satisfied, adding their conclusions, until nothing new can be derived. In: First-order logic
- Forward checking
- After each assignment, remove the assigned value from the domains of all unassigned neighbours, and backtrack at once if any domain becomes empty. In: Constraint satisfaction problems
- forward pass
- Running inputs through a network layer by layer, from input to output, to compute its prediction. In: Forward pass by hand
- forward proxy
- A proxy that acts for clients: clients send their requests through it to reach other servers, for example a corporate web filter. In: Reverse proxies (Nginx, Envoy, HAProxy)
- FP-Growth
- Frequent Pattern Growth: an algorithm that compresses transactions into a prefix tree and mines frequent itemsets without generating candidates level by level. In: Association rules and market basket analysis
- frame
- A structured record for a concept or object, with named slots holding values, defaults, or links to other frames, and an is-a link to inherit slots from a parent. In: Knowledge representation
- Freezing
- Marking parameters as not trainable, so the optimizer leaves their pretrained values unchanged. In: Transfer learning and fine-tuning
- frequent itemsets
- An itemset whose support is at least the chosen minimum support. In: Association rules and market basket analysis
- fresh context
- A sub-agent starts with an empty context holding only its own system prompt, tools and brief, not the lead's history. In: Multi-agent systems
- frontier
- The set of discovered states waiting to be expanded. Which one is taken next defines the search algorithm. In: Uninformed search: BFS, DFS, uniform cost
- FSDP
- Fully sharded data parallel: PyTorch's implementation of sharding parameters, gradients and optimizer states across data-parallel workers, the same idea as ZeRO stage 3. In: GPUs and distributed training
- full table scan
- Reading every row of a table to find the ones that match a query. Cost grows with the size of the table. In: Indexes and B-trees
- fully convolutional network
- A convolutional network with no fully connected layers, so it accepts inputs of any size and outputs a spatial map of predictions. In: Image segmentation
- function calling
- A model feature where the model, instead of answering in prose, returns a structured request to call one of the functions you described, with arguments as JSON. Also called tool use. In: Function calling and tool use
- functional requirements
- What the system must do: its features, such as post a photo or send a message. In: What is system design? , Functional and non-functional requirements
G
- gain
- The drop in impurity from a parent node to the size-weighted impurity of its children. With entropy it is called information gain. In: Decision trees
- game tree
- A tree whose nodes are game positions and whose edges are moves, alternating between the players, with final scores at the leaves. In: Adversarial search: minimax and alpha-beta
- gate weights
- The weights a mixture-of-experts router gives to the outputs of the experts it chose for a token, usually its softmax probabilities renormalised over the chosen experts. In: Mixture of experts
- gated
- A model or dataset whose files you can download only after you log in, agree to share your username and email with the authors, and are approved. In: Accounts, tokens, downloads and datasets
- gated model
- A model whose files can be downloaded only after you log in and agree to share your username and email with the authors, sometimes with extra form fields and a manual review. In: How to read a model page
- gated recurrent unit
- Gated recurrent unit: a simpler gated cell with an update gate and a reset gate and no separate cell state. In: LSTM and GRU
- gates
- A programmatic check between steps of a chain, such as a length, schema or keyword test, that stops or reroutes the workflow when an intermediate result looks wrong. In: Workflows vs agents: the patterns , LSTM and GRU
- Gaussian mixture model
- A probability model in which data are generated by first choosing one of k Gaussian components with given weights, then drawing a point from that component's normal distribution. In: Gaussian mixture models and EM
- generative adversarial network
- A pair of networks trained against each other: a generator that turns random noise into samples, and a discriminator that tries to tell those samples from real data. GAN for short. In: Generative adversarial networks
- Generative AI
- AI that produces new content such as text, images, audio, video or code, rather than only labelling or scoring inputs. In: What is artificial intelligence?
- generator
- The network in a GAN that maps random noise to a sample, such as an image. In: Generative adversarial networks
- genetic algorithm
- Population-based search that evolves candidate solutions using selection of the fittest, crossover between pairs, and random mutation. In: Local search and optimization
- geo cells
- A region of a fixed grid laid over the map (square, geohash or hexagon) used as a key for aggregating location data. In: Design an ETA prediction system
- geohash
- An encoding of a latitude and longitude into a short string, where each extra character narrows the location to a smaller cell and points sharing a prefix are usually close together. In: Geospatial indexing
- GGUF
- A single-file model format for llama.cpp and the apps built on it, holding the weights (usually quantized to fewer bits) plus metadata such as the chat template. In: Hugging Face in ten minutes: the cool stuff first , Can I run it? What a model needs
- Gini impurity
- The probability that two examples drawn at random (with replacement) from a node have different labels: 1 minus the sum of squared class proportions. In: Decision trees
- Global average pooling
- Averaging each whole feature map down to a single number, so a C by H by W tensor becomes a vector of C values regardless of image size. In: Pooling and CNN architecture
- global explanation
- An explanation of a model's overall behaviour across all inputs, such as which features it relies on most or the average shape of a feature's effect. In: Explainability
- global minimum
- The lowest point of the whole function: no other parameter setting gives a smaller loss. In: Optimization basics
- global model
- A single forecasting model trained on rows from many related time series at once, as opposed to one local model per series. In: ML and deep learning for forecasting , Design a demand forecasting system
- global traffic management
- Steering each user to an appropriate region or point of presence, based on location, latency, health and capacity. In: Global traffic: GeoDNS, anycast and GSLB
- GloVe
- Global Vectors: word embeddings trained by fitting dot products of word vectors to the logarithms of global word-word co-occurrence counts. In: Word embeddings
- golden set
- A curated set of real questions with known correct answers or known source passages, used as a fixed benchmark for a system. In: Evaluating RAG
- Goodhart's law
- The observation that when a measure becomes a target, it stops being a good measure. In RLHF, optimising a reward model too hard breaks its correlation with real quality. In: RLHF
- gossip protocol
- A protocol where each node periodically exchanges information with a few randomly chosen peers, so updates spread through the cluster like an epidemic. In: Membership, gossip and failure detection
- Graceful degradation
- Keeping core functionality working under stress by turning off or simplifying expensive, non-essential features. In: Load shedding and graceful degradation
- grader
- The code, model or person that decides whether one output passes, or how good it is. In: Why evals are the core skill
- gradient
- The vector of all partial derivatives of a function. It points in the direction of steepest increase. In: Derivatives and the chain rule , Gradient descent, the engine of ML
- Gradient clipping
- Rescaling the gradient before the update when its norm exceeds a threshold, keeping its direction but capping its size, to prevent exploding updates. In: Vanishing and exploding gradients, residual connections
- gradient descent
- An optimisation method that repeatedly moves the parameters a small step in the direction that reduces the loss fastest, the negative gradient. In: Solving linear regression: normal equation and gradient descent
- Gradio
- An open-source Python library that turns a function into a web interface with inputs, outputs and an API, in a few lines. In: Build and share your first Space
- graph
- Data made of items (nodes) and the connections between them (edges), such as users and who follows whom. In: Data types and datasets
- Graph neural networks
- A neural network whose layers update each node's vector by combining it with an aggregate of its neighbours' vectors, so stacked layers mix in information from several hops away. GNN for short. In: Graph-based recommendations
- GraphQL
- A query language and runtime for APIs where clients send a query describing exactly the fields they want, against a typed schema, usually to a single endpoint. In: GraphQL
- GraphRAG
- Retrieval over a knowledge graph built from your documents: entities and their relations are extracted, so questions can be answered by following relations or from summaries of groups of related entities. In: GraphRAG and structured retrieval
- Greedy best-first search
- Informed search that always expands the frontier state with the smallest heuristic value h(n), ignoring the cost already paid. In: Informed search and A*
- Greedy decoding
- Decoding that always picks the single most likely next token. Deterministic but prone to bland, repetitive text. In: Decoding and sampling
- Grid search
- Trying every combination of a fixed list of values for each hyperparameter. In: Hyperparameter tuning
- Group k-fold
- Cross-validation that keeps all rows sharing a group id (such as a patient, user or store) in the same fold. In: Cross-validation
- group split
- A split that keeps all rows belonging to one group, such as a user or patient, on the same side, so the model is tested on unseen groups. In: Train, validation and test splits
- grouped-query attention
- An attention variant where several query heads share one key/value head, shrinking the KV cache by the group size with little quality loss. Multi-query attention is the extreme case of a single shared key/value head. In: Context windows and the KV cache
- gRPC
- A remote procedure call framework from Google that defines services in Protocol Buffers, generates client and server code, and sends binary messages over HTTP/2. In: gRPC and Protocol Buffers
- Guardrail metrics
- Metrics that must not get significantly worse in an experiment, such as latency, errors, crashes or unsubscribes, even if the primary metric improves. In: Offline vs online evaluation and A/B tests
- Guardrails
- Automated checks around a model that block or repair unsafe or invalid inputs, outputs and actions according to explicit rules. In: Making agents reliable , Guardrails
H
- hallucinations
- When a generative model produces content that sounds confident and plausible but is false or unsupported. In: What is artificial intelligence? , Hallucinations
- handoff
- Transferring control of a conversation from one agent to another, which then talks to the user directly with its own instructions and tools. In: Multi-agent systems
- happens before
- Event a happens before event b if they are on the same process and a comes first, or a is a message send and b its receipt, or there is a chain of such steps from a to b. In: Time and clocks
- hash slots
- One of 16,384 buckets in Redis Cluster; a key's slot is CRC16 of the key modulo 16,384. In: Distributed caches: Redis and Memcached
- hash tag
- A part of a Redis key in curly braces; only that part is hashed, so keys sharing a tag land in the same slot. In: Distributed caches: Redis and Memcached
- hashing trick
- Mapping a feature string to an index with a hash function modulo the table size, so new values need no vocabulary update and memory is fixed, at the cost of occasional collisions. In: Design an ad click-through-rate prediction system
- HDBSCAN
- Hierarchical DBSCAN: runs DBSCAN over all eps values and keeps the most stable clusters, handling clusters of different densities. In: Hierarchical clustering and DBSCAN
- He (Kaiming) initialization
- Initialization with weight variance 2/n_in, which compensates for ReLU discarding half its inputs and keeps signal scale steady in ReLU networks. Also called Kaiming initialization. In: Weight initialization and normalization
- head
- One of the parallel attention operations in multi-head attention, with its own learned query, key and value projections into a smaller subspace. In: Multi-head attention
- head replacement
- Swapping a pretrained model's final classification layer for a new one sized for the target task's classes. In: Transfer learning and fine-tuning
- head-of-line blocking
- When one delayed or lost item at the front of a queue holds up everything behind it, even items that are ready. In: Networking basics: IP, TCP, UDP and ports , Ordering, partition keys and parallelism
- health checks
- A periodic request, such as GET /health, that a load balancer or orchestrator uses to decide whether an instance should receive traffic. In: Redundancy, failover and multi-AZ
- heartbeat
- A periodic 'I am alive' message sent by a node so others can notice when it stops. In: Membership, gossip and failure detection
- Hedged requests
- Sending a second copy of a request to another replica if the first has not answered within, say, the p95 latency, and using whichever answers first. In: Latency vs throughput, and why percentiles matter
- herding
- Many independent clients making the same choice at the same moment because they all saw the same stale signal, overloading the target. In: Load balancing algorithms
- heuristic
- A function h(n) that estimates the cost of the cheapest path from state n to a goal. It is cheap to compute and need not be exact. In: Informed search and A*
- Hierarchical clustering
- Clustering that builds a nested hierarchy of clusters, usually bottom-up by repeatedly merging the two closest clusters, displayed as a dendrogram. In: Hierarchical clustering and DBSCAN
- hierarchical task network
- A planning approach in which compound tasks are decomposed by methods into subtasks, recursively, until only primitive actions remain. In: Classical planning
- High level design
- Deciding the components of a system, their responsibilities and how data flows between them. Also called architecture. In: What is system design?
- Hill climbing
- Local search that always moves to a better neighbour and stops when no neighbour is better. Also called greedy local search. In: Local search and optimization
- hinge loss
- The loss max(0, 1 - y * s) for labels -1 or +1 and raw score s. It is zero once an example is on the correct side with a margin of at least 1. Used by support vector machines. In: Loss functions
- histogram
- A chart that splits a numeric range into bins and shows how many values fall in each, revealing the shape of a distribution. In: Exploratory data analysis
- history
- The log of every operation clients issued in a test: when it started, when it finished, and what it returned (ok, failed, or unknown). In: Testing distributed systems
- hit ratio
- The fraction of requests a cache answers itself, without going to the origin. In: Content Delivery Networks , Why caching works
- HLS or DASH
- HTTP Live Streaming and Dynamic Adaptive Streaming over HTTP: formats where a manifest lists the renditions, and each rendition is a list of short segment files fetched over plain HTTP. In: Design YouTube / Netflix (video streaming)
- HNSW
- Hierarchical Navigable Small World: an ANN index made of layered proximity graphs, searched by greedy hops from a sparse top layer down to a dense bottom layer. In: Vector databases and ANN indexes , Design a visual search system
- Holt-Winters
- Exponential smoothing with three smoothed components, level, trend and season, each with its own smoothing parameter. In: Stationarity and classical models
- Horizontal scaling
- Adding more machines and spreading the work across them. Also called scaling out. In: Vertical vs horizontal scaling
- host
- In MCP, the AI application the user works in, such as a chat app or an IDE, which runs MCP clients, decides which servers to connect to and controls what the model may do. In: Model Context Protocol (MCP)
- hostname
- The human-readable name of the machine or service to contact, such as shop.example.com. In: What happens when you type a URL and press Enter
- hot key
- A single key so popular that the one cache node holding it becomes overloaded, even though the cluster as a whole has spare capacity. In: Thundering herd, cache stampede and hot keys
- hot partition
- A partition that receives a disproportionate share of reads or writes, overloading its node while others sit idle. In: Hot partitions and the celebrity problem
- HTTP
- HyperText Transfer Protocol: the request and response protocol browsers, apps and APIs use to fetch and send data on the web. In: HTTP/1.1, HTTP/2 and HTTP/3
- HTTP cache
- Copies of earlier responses the browser keeps, reused while the server's caching headers say they are still fresh. In: What happens when you type a URL and press Enter
- Hub
- huggingface.co: a hosting site of Git repositories for models, datasets and demo apps (Spaces), each with files, versions and discussions. In: Hugging Face in ten minutes: the cool stuff first
- Huber loss
- A loss that is quadratic for small errors (below a threshold delta) and linear for large ones, combining MSE's smoothness with MAE's robustness to outliers. In: Loss functions
- Hybrid logical clocks
- Timestamps that combine physical time with a logical counter: close to wall-clock time, but still guaranteeing that causes get smaller timestamps than effects. In: Time and clocks
- hybrid search
- Retrieval that runs keyword search (such as BM25) and vector search for the same query and merges the two result lists. In: Hybrid search and reranking
- HyDE
- Hypothetical Document Embeddings: asking a model to write a plausible answer to the query and searching with the embedding of that hypothetical answer instead of the question. In: Contextual retrieval and query transformation
- hydrate
- Turning ids into full objects by fetching their data, usually with a batched multi-get from a cache. In: Design a news feed (Twitter/Instagram)
- HyperLogLog
- An algorithm that estimates the number of distinct items in a set using a small array of registers that record the longest run of leading zeros in hashed values. In: Probabilistic data structures
- hyperparameter
- A setting chosen before training rather than learned from the data, such as polynomial degree or learning rate. Chosen using validation data. In: Polynomial regression and basis functions , Hyperparameter tuning
I
- idempotency key
- A client-chosen unique key sent with a request so the server can recognize retries and return the original result instead of acting twice. In: Design a URL shortener
- idempotent
- An operation that has the same effect whether it runs once or several times, such as setting a value or a payment request with a unique key. In: Making agents reliable , REST API design , Idempotency and safe retries
- identity matrix
- The matrix with ones on the diagonal and zeros elsewhere. Multiplying by it leaves any vector unchanged. In: Matrices and linear transformations
- IDF
- Inverse document frequency: a weight that is high for terms that occur in few documents and low for terms that occur in most of them, typically log(N / df). In: Bag of words and TF-IDF
- ILSVRC
- The ImageNet Large Scale Visual Recognition Challenge, a yearly competition (2010 to 2017) on a 1,000-class subset of the ImageNet photo dataset with about 1.2 million training images. In: Landmark CNNs: LeNet to ResNet
- image
- An immutable, versioned package from which containers are started. Built once, run anywhere. In: Containers and Kubernetes for system designers
- image embedding
- A vector produced by a neural network such that images that look alike, or show the same thing, have nearby vectors. In: Design a visual search system
- Image segmentation
- Assigning a label to every pixel of an image, so the output is a mask with the same height and width as the input. In: Image segmentation
- image tokens
- Vectors standing for parts of an image, produced by a vision encoder and projector, placed in a language model's input sequence alongside text tokens. In: Multimodal models
- imbalanced data
- A classification dataset where one class is much rarer than the other, such as fraud, disease or equipment failure. In: Imbalanced data
- Immutable infrastructure
- Never modifying running servers; every change ships as a new image or instance that replaces the old one. In: Infrastructure as code and environments
- Implicit feedback
- Feedback inferred from behaviour, such as clicks, plays, watch time and purchases, rather than stated on purpose. In: Recommender fundamentals
- implicit reward
- In DPO, the quantity beta times the log ratio of the policy's and the reference model's probability of a response. DPO training makes it behave like a reward model. In: DPO and other preference methods
- Implicit signals
- Feedback inferred from what users do, such as accepting a suggestion, retrying, editing the output or asking for a human. In: Online evaluation and monitoring
- impression
- One showing of an ad to a user, logged with the features the model saw, the model version and the auction outcome. In: Design an ad click-through-rate prediction system
- impurity importance
- Feature importance computed from the total impurity reduction of all splits on a feature during training; also called mean decrease in impurity. In: Feature importance and SHAP
- Impute
- Filling in missing values with an estimate, such as the column's mean or median, or a prediction from other columns. In: Data cleaning
- in doubt
- The state of a prepared participant in two-phase commit that has voted yes but has not learned the decision, so it can neither commit nor abort on its own. In: Two-phase commit
- in-context learning
- A model's ability to perform a task from instructions and examples in its prompt alone, without any change to its weights. In: Few-shot prompting and examples
- in-sync replicas (ISR)
- In-sync replicas: the set of replicas (including the leader) that are fully caught up with the leader of a partition within a configured time. In: Kafka internals: the distributed log
- Incidents
- An unplanned event that disrupts or degrades a service and needs an urgent, coordinated response. In: Incident response and postmortems
- Independence
- A and B are independent if learning one tells you nothing about the other: P(A | B) = P(A). In: Probability for AI (recap)
- independent
- Knowing one event happened does not change the probability of the other: P(A given B) = P(A). In: Probability fundamentals
- index
- A separate data structure that keeps a table's values for some columns in sorted order with pointers to the rows, so matching rows can be found without scanning. In: Indexes and B-trees
- inductive
- Able to produce outputs for nodes that were not in the training graph, because the model is a function of features and neighbourhoods rather than a lookup table of per-node vectors. In: Graph-based recommendations
- inductive biases
- Assumptions built into a model's architecture that steer what it learns, such as a CNN's locality and weight sharing. In: Vision transformers
- inertia
- The sum of squared distances from each point to its assigned centroid; the objective k-means minimises. Also called within-cluster sum of squares. In: k-means clustering
- Inference
- Using a trained model to make a prediction on new data it has not seen. In: What is machine learning?
- inference by enumeration
- Answering a probability question by summing the joint probabilities of all worlds consistent with the query and the evidence, then normalising. In: Probability for AI (recap)
- inference engine
- The program that applies the rules in the knowledge base to the facts of a case, by forward or backward chaining, to derive conclusions. In: Expert systems and rule engines
- Inference Providers
- Hugging Face's pay-per-use hosted inference: one API in front of many partner providers, used by model page widgets, the playground and code. In: Hugging Face in ten minutes: the cool stuff first , Four ways to run a model
- information content
- The surprise of an outcome with probability p, measured as minus log base 2 of p, in bits. In: Information theory for ML
- Informed search
- Search that uses a heuristic estimate of the remaining cost to decide which state to expand next. In: Informed search and A*
- Infrastructure as code
- Describing infrastructure such as networks, servers, databases and permissions in versioned text files that a tool applies, instead of configuring it by hand. In: Infrastructure as code and environments
- input drift
- A change over time in the kind of inputs a system receives, so the data it was evaluated on no longer represents real use. In: Online evaluation and monitoring
- input gate
- The LSTM gate that scales the candidate content before it is added to the cell state, deciding what new information to write. In: LSTM and GRU
- Instance segmentation
- Segmentation that detects each object and gives it its own mask and id, so objects of the same class are kept apart. In: Image segmentation
- instruction dataset
- A collection of prompts paired with high-quality responses, covering many kinds of tasks, used for supervised fine-tuning. In: Supervised fine-tuning and instruction tuning
- instruction tuning
- Supervised fine-tuning on a collection of instructions paired with good responses, so a model learns to follow instructions in general rather than one task. In: Supervised fine-tuning and instruction tuning
- inter-annotator agreement
- How often independent labelers give the same label to the same item, often measured with Cohen's kappa so that chance agreement is discounted. In: Data collection and labeling
- interaction
- A feature made by combining two or more features, such as their product or ratio, to capture an effect that depends on both together. In: Feature engineering
- Interleaving
- An online evaluation that merges the result lists of two rankers into one list per query and credits each ranker with the clicks its items receive, detecting differences with far less traffic than an A/B test. In: Design a search ranking system
- intermittent
- Demand with many periods of zero sales and occasional small positive values, typical of slow-moving products at a single store. In: Design a demand forecasting system
- interpretability
- Research into what happens inside a neural network: which internal representations it uses and how computations combine them to produce an output. In: Alignment and interpretability, the big picture
- interpretable model
- A model simple enough to read directly, such as a short decision tree, a sparse linear model or a scorecard. In: Explainability
- interquartile range
- Interquartile range: Q3 minus Q1, the spread of the middle half of the data. Robust to extreme values. In: Exploratory data analysis
- inverse
- The matrix that undoes another: A times its inverse is the identity. It exists only when the determinant is not zero. In: Matrices and linear transformations
- inverted dropout
- Dropout that divides the kept activations by (1 - p) during training, so nothing needs rescaling at test time. In: Regularization for deep nets
- inverted index
- A mapping from each term to the list of documents (and often positions) that contain it, the core data structure of full-text search. In: Inverted indexes and full-text search
- IoU
- Intersection over union: the area where two boxes overlap divided by the area they cover together. 1 means identical boxes, 0 means no overlap. In: Object detection
- IP address
- A numeric address that identifies a machine on a network. IPv4 addresses look like 203.0.113.7; IPv6 addresses are much longer. In: Networking basics: IP, TCP, UDP and ports
- isolation forest
- An ensemble of random trees that isolates points with random splits; points that are isolated after few splits on average are scored as anomalies. In: Anomaly detection
- isolation levels
- A setting that controls which effects of concurrently running transactions one transaction may observe, trading strictness for performance. In: Isolation levels and anomalies
- Isotonic regression
- Fitting a non-decreasing step function from raw scores to observed outcomes. Flexible, but needs more data than Platt scaling. In: Probability calibration
- Iterative deepening
- Repeated depth-first searches with a depth limit of 0, 1, 2 and so on, combining DFS's small memory with BFS's completeness and shortest-step solutions. In: Uninformed search: BFS, DFS, uniform cost
- IVF
- Inverted file index: an ANN index that clusters vectors with k-means and, at query time, scans only the vectors in the few clusters whose centroids are nearest to the query. In: Vector databases and ANN indexes
J
- Jailbreaking
- A form of direct prompt injection aimed at getting a model to ignore its safety training and produce content it is designed to refuse. In: Prompt injection and data exfiltration
- jitter
- A small random variation added to a timer, such as plus or minus 10 percent on a TTL, so that many timers do not fire at the same moment. In: Thundering herd, cache stampede and hot keys , Timeouts, retries, backoff and jitter
- join
- A query operation that combines rows from two or more tables based on matching key values. In: Relational databases and SQL
- joint distribution
- The probability of every complete combination of values of all the variables. Any question about those variables can be answered from it. In: Probability for AI (recap)
- JSON mode
- A provider setting that guarantees the model's reply is syntactically valid JSON, without necessarily matching any particular schema. In: Structured outputs
- JSON-RPC 2.0
- A lightweight remote procedure call format in which each request is a JSON object with a method name, parameters and an id, and each response carries the same id with a result or an error. In: Model Context Protocol (MCP)
- JSONL
- JSON Lines: a text file with one JSON object per line. Good for records with nested fields or long text. In: Data types and datasets
- JWTs
- JSON Web Token: a compact token made of a base64url-encoded header, payload of claims and signature, which services can verify without contacting the issuer. In: OAuth 2.0, OpenID Connect and JWT
K
- k-means
- A clustering algorithm that partitions data into k groups, each represented by its mean (centroid), by alternating assignment of points to the nearest centroid and recomputation of centroids. In: k-means clustering
- k-means++
- An initialization for k-means that picks the first centroid at random from the data and each next one with probability proportional to its squared distance from the nearest centroid already chosen. In: k-means clustering
- k-nearest neighbours
- A method that predicts a new point's label from the labels of the k training points closest to it, by majority vote for classification or by averaging for regression. In: k-Nearest Neighbours
- k-nearest-neighbour search
- Finding the k stored items whose vectors are closest to a query vector, by cosine similarity or distance. In: Semantic search basics
- Keep-alive
- Reusing one TCP connection for several requests one after another, instead of opening a new connection each time. In: HTTP/1.1, HTTP/2 and HTTP/3
- kernel
- A function K(a, b) that equals the dot product of a and b after some feature mapping, computed without building the mapped features explicitly. In: Support Vector Machines and kernels , Convolutions
- kernel trick
- Replacing every dot product in an algorithm with a kernel function, so the algorithm works in a high-dimensional feature space at the cost of the original one. In: Support Vector Machines and kernels
- key
- In attention, the vector a token uses to advertise what it contains; queries are compared against it. In: Self-attention, step by step
- key management service (KMS)
- Key management service: a managed system, usually backed by hardware security modules, that stores master keys and performs encryption operations with them so the keys never leave it. In: Encryption, keys and secrets
- key/value head
- In grouped-query attention, several query heads share one key and value head. The number of key/value heads, not query heads, sets the KV cache size. In: GPU memory math
- keyframe
- A frame that can be decoded on its own, without earlier frames. Segments start on keyframes so each can be played independently. In: Design YouTube / Netflix (video streaming)
- Keyword search
- Search that finds documents by matching the words in the query, typically ranking them with a scoring function such as TF-IDF or BM25. In: Semantic search basics
- kill switch
- A configuration switch that turns a feature off immediately, without a deploy. In: Design a rate limiter service
- kill switches
- A feature flag that lets operators instantly disable a feature in production without deploying code. In: Load shedding and graceful degradation
- KL divergence
- Kullback-Leibler divergence: a measure of how different one probability distribution is from another. In RLHF it measures how far the policy has drifted from the reference model. In: RLHF , Information theory for ML , t-SNE and UMAP , Variational autoencoders
- knowledge base
- The part of an expert system that stores the domain knowledge, usually as if-then rules, separately from the program that applies them. In: Expert systems and rule engines
- Knowledge distillation
- Training a small student model to match the output distribution (soft targets) of a larger teacher model or ensemble, usually at a raised softmax temperature, often combined with the true labels. In: Model compression
- knowledge graph
- A graph of real-world entities (people, places, products, concepts) connected by typed relationships, stored as subject-predicate-object facts. In: Knowledge graphs and ontologies
- Knowledge representation
- The study of how to store what a system knows (facts, rules, objects and their relationships) in a form a program can reason with. In: Knowledge representation
- KS test
- Kolmogorov-Smirnov test: compares two distributions by the largest vertical gap between their cumulative distribution functions, and gives a p-value for the hypothesis that they are the same. In: Monitoring, drift and retraining
- Kubernetes
- An open source system that runs containers across a cluster of machines, continuously reconciling the actual state toward a declared desired state. In: Containers and Kubernetes for system designers
- KV cache
- The stored keys and values of every earlier token, per layer, kept during generation so each new token only computes its own key and value and attends to the stored ones. In: Context windows and the KV cache , GPU memory math , Can I run it? What a model needs
L
- Label
- The correct answer for a training example, such as spam or not spam, or the price a house sold for. In: What is machine learning?
- label delay
- The time between a prediction and the moment its true outcome is known, such as weeks for a chargeback or months for a loan default. In: Monitoring, drift and retraining , Design a fraud detection system
- Lag features
- A feature built from an earlier value of the series, such as sales 7 days before the target date. In: ML and deep learning for forecasting
- LambdaMART
- The boosted-tree version of LambdaRank: gradient boosting (MART) where each new regression tree fits the lambda gradients of every item. In: Learning to rank
- LambdaRank
- A learning-to-rank method that multiplies each pair's RankNet gradient by the change in NDCG that swapping the two items would cause. In: Learning to rank
- landscape
- The objective value plotted over all states, with neighbouring states next to each other. Peaks are good solutions for maximization. In: Local search and optimization
- Laplace noise
- Noise from the Laplace (double exponential) distribution, whose density falls off as exp(-|x|/b); with scale b = sensitivity / epsilon it gives epsilon-differential privacy. In: Privacy-preserving ML
- Laplace smoothing
- Adding a small count (usually 1) to every word in every class before computing probabilities, so no word ever gets probability zero. In: Naive Bayes
- Lasso
- Linear regression with an L1 penalty, the sum of absolute weights. It drives some weights exactly to zero, performing feature selection. In: Regularization: ridge, lasso, elastic net
- Latency
- How long one request takes from start to finish, usually measured in milliseconds. In: Latency vs throughput, and why percentiles matter
- Latent diffusion
- Diffusion run in the compressed latent space of a pretrained autoencoder instead of on pixels, so each denoising step works on far fewer numbers. In: Image generation
- latent factors
- Hidden dimensions learned by a factorization model; each user and item gets a coordinate on each, and their dot product predicts the interaction. In: Collaborative filtering and matrix factorization
- latent space
- The space of codes produced by an encoder; nearby points should correspond to similar inputs. In: Autoencoders
- latent variable
- A variable that is part of the model but never observed, such as which mixture component generated each point. In: Gaussian mixture models and EM
- layer 4 load balancer
- A load balancer that routes TCP or UDP connections using only IP addresses and ports, without reading the application data inside. In: L4 vs L7 load balancing
- layer 7 load balancer
- A load balancer that understands an application protocol such as HTTP, so it can route, modify and retry individual requests based on their content. In: L4 vs L7 load balancing
- Layer normalisation
- An operation that rescales each token's vector to mean 0 and standard deviation 1 across its features, then applies a learned scale and shift. It keeps activations in a stable range. In: The transformer block
- Layer normalization
- A normalization that standardizes each example across its own features, then applies a learned scale and shift. It does not depend on the batch, so it behaves the same in training and inference. In: Weight initialization and normalization
- lazy learner
- A model that stores the training data and defers computation until a prediction is requested, instead of learning parameters up front. In: k-Nearest Neighbours
- leader-follower
- A replication setup where one node, the leader, accepts all writes and streams changes to follower nodes, which apply them in the same order. In: Replication: leader-follower
- leaf
- An end node of a decision tree. It holds a prediction: the majority class, or the average target, of the training examples that reached it. In: Decision trees
- leaky bucket
- A rate limiter that queues requests and releases them at a fixed rate; requests that do not fit in the queue are rejected, so output is perfectly smooth. In: Rate limiting algorithms
- learning curve
- A plot of training and validation performance against the number of training examples. In: Learning curves and baselines
- learning rate
- A small positive number that scales the size of each update step. In: Gradient descent, the engine of ML , Learning rate schedules and warmup
- learning rate range test
- A short run in which the learning rate is increased exponentially every step while the loss is recorded, to find the range of rates where training makes fast progress without diverging. In: Learning rate schedules and warmup
- learning rate schedule
- A rule that changes the learning rate over the course of training, for example decaying it step by step or along a cosine curve. In: Learning rate schedules and warmup
- Learning to rank
- Training a model whose scores are used only to sort a list of items for a query, with a loss built around the quality of that order. In: Learning to rank
- learning-to-rank
- A model trained to order a list of items for a query, using pointwise, pairwise or listwise losses on graded relevance labels. In: Design a search ranking system
- lease
- A time-limited grant of leadership or ownership that must be renewed before it expires. In: Quorums and leader election , Distributed locks and fencing tokens , Coordination services (ZooKeeper, etcd) , Design a unique ID generator
- lease token
- A token the cache hands out on a miss; a later SET for that key is accepted only if the token is still valid, and a delete invalidates it. In: Cache invalidation and consistency
- least squares
- The line, or plane, that makes the sum of squared residuals as small as possible. In: Linear regression intuition
- Lemmatization
- Reducing a word to its dictionary form (lemma) using a vocabulary and the word's part of speech, for example mice to mouse and better to good. In: Text preprocessing and tokenization basics
- lethal trifecta
- The combination of access to private data, exposure to untrusted content and a way to communicate externally. An agent with all three can be made to leak data by a prompt injection. In: Prompt injection and data exfiltration
- Lift
- For a rule X -> Y, confidence divided by the support of Y. Above 1 means X and Y co-occur more than if they were independent; 1 means no association; below 1 means they co-occur less. In: Association rules and market basket analysis
- LightGBM
- A gradient boosting library from Microsoft that grows trees leaf-wise and uses histogram-based splits, built for speed on large datasets. In: XGBoost, LightGBM and CatBoost in practice
- LIME
- Local Interpretable Model-agnostic Explanations: explains one prediction by sampling points near it, weighting them by closeness, and fitting a simple model such as a sparse linear model to the black box's outputs there. In: Explainability
- linear regression
- A model that predicts a number as a weighted sum of the features plus a constant: a straight line for one feature, a flat plane for two, and so on. In: Linear regression intuition
- linearizability
- Every operation appears to take effect at a single instant; once a write completes, every later read, from any client, sees it. Behaves like a single copy. In: CAP theorem, properly explained
- linearizability checker
- A program that decides whether a recorded history could have been produced by a single copy of the data executing operations one at a time, each taking effect between its start and end. In: Testing distributed systems
- linearly separable
- Two classes are linearly separable when a single straight line (or flat hyperplane) can put all of one class on one side and all of the other on the other side. In: The perceptron and the artificial neuron
- linkage
- The rule for measuring distance between clusters in hierarchical clustering: single (closest pair), complete (farthest pair), average (mean pairwise) or Ward (increase in within-cluster variance). In: Hierarchical clustering and DBSCAN
- listwise
- Learning-to-rank methods whose loss is defined on the whole ranked list for a query, such as ListNet or methods tied to NDCG. In: Learning to rank
- Little's Law
- Average number of requests in a system equals arrival rate times average time each spends there: L = lambda x W. In: Processes, threads and concurrency , Latency vs throughput, and why percentiles matter
- LLM classifier
- A large language model prompted or fine-tuned to read content together with a written policy and output a label, often used as an expensive second-stage classifier. In: Design a spam / content moderation classifier
- LLM judge
- Using a language model, prompted with a rubric, to grade outputs such as answers, for example for faithfulness or relevance. In: Evaluating RAG , LLM-as-judge
- load balancer
- A component that receives incoming requests and spreads them across a pool of servers, skipping unhealthy ones. In: Vertical vs horizontal scaling , What a load balancer does
- Load shedding
- Deliberately rejecting or dropping some work when a system is overloaded, so the work it does accept is completed in time. In: Backpressure and flow control , Load shedding and graceful degradation
- Load testing
- Generating synthetic traffic against a system to measure how its latency, errors and resource use change as load increases. In: Capacity planning and load testing
- load-balancing loss
- An extra training loss term in mixture-of-experts models that is smallest when tokens and router probability are spread evenly over experts, discouraging the router from overloading a few. In: Mixture of experts
- local explanation
- An explanation of a single prediction: why this input received this output. In: Explainability
- local gradients
- The derivative of one node's output with respect to its own input, computed from values available at that node alone. In: Backpropagation
- local maximum
- A state better than all of its neighbours but worse than the best state overall. In: Local search and optimization
- local minimum
- A point lower than all its immediate neighbours, but not necessarily the lowest overall. In: Optimization basics , k-means clustering
- Local search
- Optimization that keeps one current solution (or a few) and repeatedly moves to a neighbouring solution, without remembering the paths explored. In: Local search and optimization
- locality
- The tendency of programs and users to access the same data, or nearby data, repeatedly over short periods. In: Why caching works
- log
- A timestamped record of a discrete event emitted by a service, ideally structured as key-value fields rather than free text. In: Logs, metrics and traces
- log scale
- A scale where each equal step multiplies the value by ten, so values from 1 to 100 million fit on one chart. In: Latency numbers every engineer should know
- log-odds
- The natural logarithm of the odds, log(p / (1 - p)). Logistic regression is linear in the log-odds. In: Logistic regression
- Logical clocks
- A logical clock: counters attached to events so that causes always get smaller values than their effects, without using physical time. In: Time and clocks
- Logistic regression
- A linear classifier that passes a weighted sum of the features through the sigmoid function to output the probability of the positive class. In: Logistic regression
- logits
- The raw, unnormalised scores a model outputs for each token before softmax turns them into probabilities. In: Decoding and sampling , Multi-class and multi-label classification
- long short-term memory
- Long short-term memory: a recurrent cell with a separate cell state and three learned gates (forget, input, output) that control what is erased, written and exposed at each step. In: LSTM and GRU
- Long-term memory
- Information kept outside the model, in a database or file store, that persists across sessions and is retrieved into the context when relevant. In: Memory for agents
- long-term value
- An outcome measured over weeks or months, such as retention or survey satisfaction, that short-term engagement metrics only partly predict. In: Design a feed ranking system
- LoRA
- Low-Rank Adaptation: fine-tuning that freezes the pretrained weights and learns a small low-rank update (two thin matrices B and A) added to selected weight matrices. In: Parameter-efficient fine-tuning: LoRA and QLoRA
- loss
- A function that measures how wrong a model is on the training data. Training searches for the parameters that make it smallest. In: Linear regression intuition , Loss functions
- loss function
- A function that measures how wrong the model's predictions are on the training data. Lower is better. In: Gradient descent, the engine of ML
- loss scaling
- Multiplying the loss by a large constant before the backward pass so small gradients do not underflow to zero in 16-bit floats, then dividing the gradients by the same constant before the update. In: GPUs and distributed training
- lost-in-the-middle
- The tendency of language models to use information at the beginning or end of a long context more reliably than information in the middle. In: Context windows and the KV cache
- Low level design
- Designing the code inside one component: classes, functions, data structures and interfaces. In: What is system design?
- LSM-tree
- Log-structured merge tree: a storage engine that buffers writes in memory, flushes them as sorted immutable files, and merges those files in the background. In: Storage engines: B-tree vs LSM-tree
- Lua script
- A script that Redis runs as one indivisible operation: no other command runs on that shard until it finishes. In: Design a rate limiter service
M
- Machine learning
- A way of building AI where the system learns patterns from examples instead of following rules a person wrote. In: What is artificial intelligence?
- Macro averaging
- Computing a metric for each class separately and taking the plain average, so every class counts equally regardless of size. In: Multi-class and multi-label classification
- majority quorum
- Any group containing more than half of the nodes in a cluster. In: Quorums and leader election
- mAP
- Mean average precision: AP averaged over all classes, and in COCO also over IoU thresholds from 0.50 to 0.95. In: Object detection
- MAPE
- Mean absolute percentage error: the average of absolute errors divided by the true values. Scale-free, but undefined at zero and explosive near it. In: Regression metrics
- margin
- The width of the empty band around an SVM's decision boundary, equal to 2 / ||w||. In: Support Vector Machines and kernels
- Markov decision process
- A model of sequential decision making: states, actions, transition probabilities P(s' | s, a), rewards, and a discount factor. Solving it means finding a policy that maximizes expected total reward. In: Sequential decisions: MDPs (bridge to RL) , RL fundamentals and MDPs
- masked
- Excluding some tokens, usually the prompt, from the training loss so the model learns only to produce the response. In: Supervised fine-tuning and instruction tuning
- materialization
- A job that reads the latest feature values from the offline store or a data source and writes them into the online store. In: Feature stores and training-serving skew
- materialized view
- The stored result of a query, kept up to date as the underlying data changes, so reading it is as cheap as reading a table. In: Denormalization, materialized views and CQRS
- matrix
- A rectangular grid of numbers with rows and columns. Multiplying a vector by it transforms the vector. In: Matrices and linear transformations
- Max pooling
- Pooling that keeps the largest value in each window: the strongest response of the feature in that region. In: Pooling and CNN architecture
- max-norm constraint
- A bound on the largest change to any single input feature, written as the infinity norm of the perturbation being at most epsilon. In: Robustness and adversarial examples
- Maximal marginal relevance
- Maximal marginal relevance: a greedy re-ranker that picks, at each step, the item with the best trade-off between relevance and dissimilarity to items already picked. In: Cold start, diversity and feedback loops
- maximum likelihood
- Choosing model parameters that maximise the probability (likelihood) of the observed data under the model. In: Gaussian mixture models and EM
- MCP client
- In MCP, the component inside the host that holds one dedicated connection to one MCP server and relays requests and results. In: Model Context Protocol (MCP)
- MCP server
- In MCP, a program that offers tools, resources and prompts to MCP clients, usually wrapping one system such as a file store, a database or a SaaS API. In: Model Context Protocol (MCP) , Capstone: a tool-using agent with MCP
- mean
- The average value you would get over many draws, also called the expectation: E[X]. In: Distributions you will meet
- Mean absolute error
- Mean absolute error: the average of the absolute residuals. In the target's units and equally sensitive to every minute of error. In: Regression metrics , Loss functions
- mean absolute scaled error
- Mean absolute scaled error: the forecast's MAE divided by the in-sample MAE of a naive or seasonal naive forecast, so values below 1 beat that baseline. In: Backtesting and forecast evaluation
- Mean squared error
- Mean squared error: the average of the squared residuals. The standard loss for regression. In: Linear regression intuition , Loss functions
- Mechanistic interpretability
- Interpretability work that tries to reverse-engineer the actual mechanisms inside a network, the internal features and the circuits that connect them, rather than only explaining outputs. In: Alignment and interpretability, the big picture
- mel spectrogram
- A picture of sound: how much energy each frequency band has in each short slice of time, with the frequency bands spaced the way human hearing perceives pitch. In: Speech: ASR and TTS
- membership
- The list of nodes currently in a cluster and their status (alive, suspected, dead, left), as each node believes it to be. In: Membership, gossip and failure detection
- membership inference
- An attack that decides, from a model's outputs, whether a given record was in the model's training set. In: Privacy-preserving ML
- memory-bound
- Limited by how fast data can be moved from memory rather than by how fast the chip can compute. Faster arithmetic does not help; moving fewer bytes does. In: How LLM inference works
- memoryless
- The property that the remaining wait does not depend on how long you have already waited. In: Distributions you will meet
- memtable
- An in-memory sorted structure that collects recent writes in an LSM-tree before they are flushed to disk. In: Storage engines: B-tree vs LSM-tree
- Merkle trees
- A tree of hashes where each parent is the hash of its children, so two replicas can find which ranges differ by comparing a few hashes. In: Multi-leader and leaderless replication , Design a distributed key-value store
- message queue
- A buffer that holds messages from producers until consumers are ready to process them, usually in first-in, first-out order. In: Why async? Queues explained
- meta-model
- The second-level model in stacking. Its inputs are the base models' predictions and its output is the final prediction. In: Stacking and blending
- method
- The action requested: GET reads, POST creates, PUT replaces, PATCH updates part, DELETE removes. In: HTTP/1.1, HTTP/2 and HTTP/3
- metric
- A numeric measurement aggregated over time, such as requests per second or 99th percentile latency, stored as a time series. In: Logs, metrics and traces , The ML project lifecycle
- Micro averaging
- Pooling all true positives, false positives and false negatives across classes before computing the metric, so frequent classes dominate. In: Multi-class and multi-label classification
- Microservices
- An architecture where a system is built as many small services, each owning its own data and deployed independently, usually by a single team, communicating over the network. In: Microservices
- mini-batch
- A small random subset of the training data (often 32 to 1024 examples) used to compute one gradient step. In: Autograd and your first PyTorch training loop
- Minimax
- The value of a game position assuming both players play optimally: leaves have their utility, MAX nodes take the maximum of their children, MIN nodes the minimum. In: Adversarial search: minimax and alpha-beta
- minimum detectable effect
- The smallest true effect worth detecting; it sets the required sample size. In: Offline vs online evaluation and A/B tests
- mIoU
- Mean intersection over union: IoU between predicted and true masks computed per class, then averaged over classes; the standard semantic segmentation metric. In: Image segmentation
- missing indicator
- An extra 0/1 column that records whether the original value was missing, so the model can learn from the missingness itself. In: Data cleaning
- Mixture of tokens
- A continuous variant of mixture of experts: tokens from different examples are blended into weighted mixtures, and each expert processes a mixture instead of being picked per token by a discrete router. In: Mixture of experts
- mixture-of-experts
- A layer with many parallel feed-forward networks (experts) and a router that sends each token to only a few of them, so total parameters can grow without growing the compute per token. In: Mixture of experts
- Mixup
- An augmentation that blends two training images and their one-hot labels with the same weight lambda, drawn from a Beta distribution. In: Data augmentation
- MMR
- Maximal marginal relevance: a greedy re-ranker that picks, at each step, the item with the best trade-off between relevance and dissimilarity to items already picked. In: Design a feed ranking system
- mode collapse
- A GAN failure where the generator produces only one or a few kinds of output, covering a small part of the data distribution, because those outputs currently fool the discriminator. In: Generative adversarial networks
- model
- An assignment of true or false to every proposition symbol: one possible way the world could be. In: Propositional logic , What is machine learning?
- model card
- A short document released with a model that states its intended uses, the uses it is not suited for, how it was evaluated, and how it performs across different groups and conditions. In: AI regulation and governance , How to read a model page
- model collapse
- The gradual, compounding loss of diversity and rare content when generative models are trained repeatedly on data produced by earlier models, without enough fresh real data. In: Distillation and synthetic data
- Model Context Protocol
- An open standard, introduced by Anthropic in November 2024, that defines how AI applications discover and use tools, data and prompt templates offered by separate programs called MCP servers. In: Model Context Protocol (MCP)
- model gateway
- A shared service in front of model providers that handles API keys, retries, fallbacks to other models, rate limits, caching and usage accounting. In: Anatomy of an LLM application
- model registry
- A versioned catalogue of trained models with the metadata needed to reproduce, compare and deploy them: training data reference, parameters, metrics and status. In: ML system architecture overview
- model servers
- Software that loads trained models and exposes them as network endpoints, handling versions, batching, hardware and metrics, such as TensorFlow Serving or NVIDIA Triton. In: Model serving patterns
- model states
- Everything that scales with the parameter count during training: weights, gradients and optimizer states such as Adam's momentum and variance. In: GPUs and distributed training
- modular monolith
- A single deployable application divided into modules with explicit public interfaces and enforced dependency rules, so each module can be owned and changed independently. In: Monolith and modular monolith
- momentum
- An optimizer tweak that keeps a running average of past gradients, so steps build up speed along consistent directions and damp out zig-zags. In: Gradient descent, the engine of ML , Optimizers: SGD, momentum, RMSProp, Adam, AdamW
- monolith
- An application built and deployed as a single unit that contains all of its features, typically with one codebase and one database. In: Monolith and modular monolith
- monotonic reads
- A guarantee that a user never sees data go backwards in time: once they have seen a value, later reads do not return an older one. In: Replication lag and read consistency guarantees
- Monte Carlo Tree Search
- A search algorithm that grows a game tree selectively, guided by statistics from many simulated games, balancing moves that look good with moves that are under-explored. In: Monte Carlo Tree Search and AlphaGo
- moving average
- A time series model that predicts a value from the current and previous q forecast errors (shocks); unrelated to a moving-average smoother. In: Stationarity and classical models
- MRR
- Mean reciprocal rank: the average over questions of 1 divided by the rank of the first relevant result, so 1 means always first and 0.5 means typically second. In: Evaluating RAG , Learning to rank
- MTBF
- Mean time between failures: the average time a component runs before it fails. In: Availability, reliability and the "nines"
- MTTR
- Mean time to recovery: the average time from a failure until service is restored, including detection, diagnosis and the fix. In: Availability, reliability and the "nines"
- multi-agent system
- A system in which several model instances, each running its own loop with its own prompt, tools and context, work on parts of one task and pass results between them. In: Multi-agent systems
- multi-armed bandit
- A sequential decision problem with several options (arms) of unknown average reward: each round you pick one, see its reward, and try to maximize the total. The simplest reinforcement learning problem: one state, no transitions. In: Multi-armed bandits
- multi-class
- Classification where each example belongs to exactly one of three or more classes. In: Multi-class and multi-label classification
- Multi-head attention
- Running several attention operations in parallel on the same tokens, each with its own query, key and value projections, then concatenating their outputs and projecting back to the model width. In: Multi-head attention
- multi-label
- Classification where each example can have any number of labels at the same time, including none. In: Multi-class and multi-label classification
- multi-layer network
- A neural network with one or more hidden layers between input and output, where each layer's neurons take the previous layer's outputs as inputs. Also called a multi-layer perceptron (MLP). In: Multi-layer networks
- multi-modal model
- A model that takes several input types (text, image, metadata) and fuses their embeddings into one prediction, so evidence in one modality can be weighed with the others. In: Design a spam / content moderation classifier
- multi-task model
- A model with a shared body and several output heads, each trained on a different label (click, like, share, hide), so related tasks share what they learn. In: Design a feed ranking system
- multi-tenant
- Serving many customers (tenants) from shared infrastructure while keeping each tenant's data and performance isolated from the others. In: Multi-tenancy and isolation
- multimodal models
- Models that take in, and sometimes generate, more than one kind of data, such as text, images, audio or video, in a single model. In: Multimodal models
- Mutual information
- A measure of how much knowing one variable reduces uncertainty about another. Zero means independent; it detects non-linear relationships too. In: Feature selection
- mutual TLS
- Mutual TLS: both client and server present certificates, so each proves its identity to the other. In: TLS and HTTPS , Service mesh
- MVCC
- Multi-version concurrency control: keeping several versions of each row so readers see a consistent snapshot without blocking writers. In: Isolation levels and anomalies
N
- N+1 problem
- Running one query for a list and then one more query for each item in it, so N items cost N + 1 queries. In: GraphQL
- Naive Bayes
- A classifier that applies Bayes' theorem while assuming every feature is independent of the others once the class is known. In: Naive Bayes
- narrow AI
- AI that performs well on a specific task or family of tasks, but does not have broad, flexible, human-level competence across domains. In: What is artificial intelligence?
- ndarray
- NumPy's fixed-type, n-dimensional grid of numbers, stored compactly so maths on it runs in fast compiled code. In: Your toolkit: Python, NumPy, pandas, scikit-learn, PyTorch
- NDCG
- Normalized DCG: DCG divided by the DCG of the best possible order of the same items, so 1.0 means a perfect ranking and queries with different numbers of relevant items can be averaged. In: Learning to rank
- NDCG@k
- Normalized discounted cumulative gain: the sum of graded relevance gains discounted by log of position, divided by the best possible sum, so 1.0 means a perfect order. In: Design a search ranking system
- near misses
- Examples chosen because they sit close to a decision boundary, such as two similar inputs with different correct answers, to teach the model where the boundary lies. In: Few-shot prompting and examples
- negative sampling
- A training trick for word2vec that turns word prediction into binary classification: score the true context word high and a few randomly drawn words low. In: Word embeddings
- negatives
- Items used during training as examples of what the user did not choose, so the model learns to score them below the positive item. In: Two-tower retrieval and embedding-based retrieval
- neighbourhood
- The set of solutions reachable from the current one by one small change, such as swapping two items or reversing a section of a route. In: Local search and optimization
- neighbourhood method
- Collaborative filtering that predicts from similar users or similar items, using a similarity measure such as cosine on co-rated items. In: Collaborative filtering and matrix factorization
- neighbouring
- Two datasets that differ in exactly one record: one includes a particular person's data and the other does not. In: Privacy-preserving ML
- Nested cross-validation
- Cross-validation with two loops: an inner loop that tunes hyperparameters and an outer loop that estimates the performance of the whole tuning procedure. In: Cross-validation
- newsvendor
- The single-period inventory problem: order the quantity at which the probability of selling out equals the cost of a lost sale divided by the sum of that cost and the cost of a leftover unit. In: Design a demand forecasting system
- noisy neighbour
- A tenant whose heavy usage consumes shared resources and degrades performance for other tenants on the same infrastructure. In: Multi-tenancy and isolation
- non-functional requirements
- How well the system must do it: scale, latency, availability, consistency, cost and so on. In: What is system design? , Functional and non-functional requirements
- Non-maximum suppression
- A post-processing step that keeps the highest-scoring box, removes every remaining box that overlaps it by more than an IoU threshold, and repeats with the next survivor. In: Object detection
- non-saturating loss
- The generator objective that maximizes log D(G(z)) instead of minimizing log(1 - D(G(z))): same fixed point, much stronger gradients when the discriminator confidently rejects fakes. In: Generative adversarial networks
- norm
- A function that measures a vector's length. The L2 norm is the straight-line length; the L1 norm adds the absolute values. In: Vectors and vector spaces
- normal equation
- A formula that computes the least squares weights of linear regression in one step: w = (X^T X)^-1 X^T y. In: Solving linear regression: normal equation and gradient descent
- Normalization
- Organizing tables so each fact is stored in exactly one place, removing duplication that could lead to inconsistent data. In: Relational databases and SQL , Weight initialization and normalization , Text preprocessing and tokenization basics
- normalized entropy
- Normalized entropy: the model's log loss divided by the log loss of always predicting the average CTR. Lower is better; 1.0 means no better than the base rate. In: Design an ad click-through-rate prediction system
- notebook
- An interactive document, such as Jupyter, that mixes code cells, their output and notes, and keeps variables in memory between cells. In: Your toolkit: Python, NumPy, pandas, scikit-learn, PyTorch
- novelty effect
- A temporary change in behaviour caused by something being new, which fades as users get used to it. In: Offline vs online evaluation and A/B tests
- null hypothesis
- The default claim that there is no effect or no difference, which a test tries to find evidence against. In: Hypothesis testing and p-values
- Numerical
- Data measured as a quantity, where arithmetic makes sense: price, age, temperature, number of clicks. In: Data types and datasets
O
- OAuth 2.0
- An authorization framework that lets an application obtain limited, revocable access to a user's resources on another service, through tokens issued by an authorization server, without seeing the user's password. In: OAuth 2.0, OpenID Connect and JWT
- Object detection
- The task of finding every object of interest in an image and returning, for each, a class label, a bounding box and a confidence score. In: Object detection
- object detector
- A model that finds objects in an image and returns a bounding box, a class and a confidence score for each one. In: Design a visual search system
- Object storage
- Storage for files as immutable objects in buckets, addressed by key and accessed over HTTP, with very high durability and practically unlimited capacity. In: Object storage, block storage and file systems
- objective function
- The function local search tries to maximize or minimize, such as tour length, profit or the number of satisfied constraints. In: Local search and optimization , Optimization basics
- Observability
- Recording what a system did, for every request, in enough detail to explain its behavior afterwards: logs, metrics and traces. In: Anatomy of an LLM application , Logs, metrics and traces
- observation
- The result of an agent's action, such as a tool's output or error message, appended to the context for the next reasoning step. In: The agent loop (ReAct)
- OCR
- Optical character recognition: turning text that appears in an image into machine-readable characters. In: Vision-language models in practice
- odds
- The ratio p / (1 - p): how many times more likely an event is to happen than not. Probability 0.8 is odds of 4. In: Logistic regression
- off-policy
- A learning method that evaluates or improves a different policy (here, the greedy one) from the one generating its experience (here, an exploring one). In: Q-learning
- Offline evaluation
- Evaluation on a fixed, historical dataset, without showing the model's output to real users. In: Offline vs online evaluation and A/B tests
- Offline RL
- Reinforcement learning from a fixed dataset of previously collected transitions, with no further interaction with the environment during training. In: RL in the real world
- offset
- The position of a message in a log partition; a consumer's offset records how far it has read. In: Message queues vs pub/sub vs event streams
- one-class SVM
- A support vector machine trained on normal data only. It learns a boundary that encloses most of the data in a kernel feature space; points outside are flagged. In: Anomaly detection
- One-cycle
- A schedule that raises the learning rate from a low value to a high peak over part of training, then lowers it to a very small value by the end. In: Learning rate schedules and warmup
- One-hot encoding
- Turning a category with k values into k columns of 0s and 1s, with a 1 in the column for the row's category. In: Feature engineering
- one-hot vector
- A vector with one slot per vocabulary item, all zeros except a single 1 in the slot of the item it represents. In: From symbols to vectors
- one-vs-rest
- Training one binary classifier per class (this class against all others) and predicting the class whose classifier is most confident. In: Multi-class and multi-label classification
- Online evaluation
- Measuring an AI system's quality on live traffic, using user feedback, behaviour signals and automatic grading of sampled outputs. In: Online evaluation and monitoring , Offline vs online evaluation and A/B tests
- online feature store
- A low-latency key-value store holding the latest feature values for serving, kept consistent with the offline feature definitions used in training. In: Design an ETA prediction system
- online joiner
- A streaming job that joins click events to the impressions they belong to, labelling an impression as a non-click only after a waiting window has passed. In: Design an ad click-through-rate prediction system
- ontology
- A formal description of the types of things in a domain and how they relate: classes, subclass hierarchies, the properties each class can have, and constraints on them. In: Knowledge graphs and ontologies
- open coding
- Writing free-form notes about each example without a predefined list of categories; a method borrowed from qualitative research. In: Error analysis
- OpenID Connect (OIDC)
- An identity layer on top of OAuth 2.0 that lets an app learn who the user is, through a signed ID token issued by the identity provider. In: OAuth 2.0, OpenID Connect and JWT
- optimal solution
- A solution with the lowest path cost among all solutions. In: Problem formulation and state spaces
- optimization
- Finding the inputs that make a function as small (or as large) as possible. In: Optimization basics
- optimizer
- The rule that turns gradients into parameter updates, such as SGD, momentum, RMSProp, Adam or AdamW. In: Optimizers: SGD, momentum, RMSProp, Adam, AdamW
- orchestration
- Coordinating a saga with a central component that tells each service which step to run next and which compensations to run on failure. In: Sagas: choreography and orchestration
- orchestration layer
- The application code that turns a user request into model calls: it gathers context, builds the prompt, runs the tool loop, validates outputs and decides what to return. In: Anatomy of an LLM application
- orchestrator
- A system that runs DAGs: it decides when a run starts, executes tasks in dependency order on workers, retries failures, and records the state and logs of every task. In: Training pipelines and orchestration
- ordered target statistics
- CatBoost's target encoding that, for each row, uses only the labels of rows that come before it in a random ordering, so no row sees its own label. In: XGBoost, LightGBM and CatBoost in practice
- orders of magnitude
- Powers of ten. Two values differ by an order of magnitude when one is about ten times the other. In: Latency numbers every engineer should know
- Ordinal
- Categorical data whose values have a meaningful order but uneven or unknown gaps, such as poor, good, excellent. In: Data types and datasets
- origin
- The server or storage that holds the authoritative copy of your content, which the CDN fetches from on a miss. In: Content Delivery Networks
- origin shield
- An intermediate CDN cache layer that all edge locations fetch through, so the origin sees one request per object instead of one per edge. In: Content Delivery Networks
- out-of-bag (OOB)
- An accuracy or error estimate for a bagged ensemble computed by predicting each training row only with the trees whose bootstrap sample did not include it. In: Bagging and random forests
- Out-of-distribution detection
- Detecting inputs that are unlike the training data, so the system can abstain, ask a human or fall back to a safer rule instead of predicting. In: Robustness and adversarial examples
- out-of-fold prediction
- A prediction for a training row made by a model that was trained without that row, as in k-fold cross-validation. In: Stacking and blending
- outliers
- A value far from the rest of the data. It may be an error to fix or a rare but real event to keep. In: Exploratory data analysis
- output gate
- The LSTM gate that scales tanh of the cell state to produce the hidden output h(t), deciding what part of the memory to expose. In: LSTM and GRU
- over-fetching
- Receiving fields the client does not need because the endpoint returns a fixed shape. In: GraphQL
- over-generalize
- When a model with dense embeddings predicts interactions for pairs that are actually rare or irrelevant, because similar-looking embeddings suggest a link the data does not support. In: Design an ad click-through-rate prediction system
- overconfident
- Predicting probabilities that are more extreme (closer to 0 or 1) than the observed frequencies justify. In: Probability calibration
- Overfitting
- When a model fits the training data, including its noise, so closely that it does worse on new data. In: Polynomial regression and basis functions , Bias, variance, underfitting and overfitting
- Overlap
- Repeating the end of one chunk at the start of the next, so a sentence or idea that falls on a boundary appears whole in at least one chunk. In: Document ingestion and chunking
- owner
- The node responsible for storing or serving a key. In: Consistent hashing
P
- p-value
- The probability, assuming the null hypothesis is true, of seeing data at least as extreme as what was observed. In: Hypothesis testing and p-values
- PACELC
- If there is a Partition, choose Availability or Consistency; Else, in normal operation, choose Latency or Consistency. In: PACELC and consistency models
- packets
- A small chunk of data, typically up to about 1,500 bytes, with a header saying where it comes from and where it is going. In: Networking basics: IP, TCP, UDP and ports
- Packing
- Concatenating several short training examples into one full-length sequence, with attention and loss masks so they do not see each other. In: Supervised fine-tuning and instruction tuning
- Padding
- Extra pixels, usually zeros, added around the border of the input so the kernel can be centred on edge pixels and the output keeps its size. In: Convolutions
- page token
- An encoded string returned with each page that the client passes back to get the next page. The server decides what is inside, usually the sort key of the last item. In: Pagination patterns
- PagedAttention
- An attention algorithm that stores each request's KV cache in fixed-size blocks anywhere in GPU memory, found through a per-request block table, like pages in an operating system's virtual memory. In: Batching and PagedAttention
- pages
- Returning a large result set in chunks, one page per request, with a way to ask for the next chunk. In: Pagination patterns
- pairwise
- Learning-to-rank methods that train on pairs of items from the same query, penalizing pairs in the wrong order. In: Learning to rank
- pairwise comparison
- Showing a judge two outputs for the same input and asking which is better, or whether they tie. In: LLM-as-judge
- Panoptic segmentation
- Segmentation that gives every pixel both a class and, for countable objects, an instance id, unifying semantic and instance segmentation. In: Image segmentation
- parallel tool calls
- Several tool calls returned by the model in a single reply, which your code can run concurrently before returning all the results together. In: Function calling and tool use
- parameter sharing
- Using the same weights at every position of the input, so a convolution layer has very few parameters and detects a pattern wherever it appears. In: Convolutions
- parameters
- The learned numbers inside a model. A model's size, such as 8B, is how many of them it has, in billions. In: Can I run it? What a model needs , Hyperparameter tuning , Multi-layer networks
- Parquet
- A column-oriented file format that stores each column together, compressed and with its type, so analytical queries read only the columns they need. In: Data types and datasets
- partial autocorrelation
- The correlation between a series and its value k steps earlier after removing the effects of the lags in between. In: Stationarity and classical models
- partial dependence plots
- A plot of the model's average prediction as one feature is varied over its range, with the other features left at their observed values. In: Explainability
- partial derivative
- The derivative of a many-input function with respect to one input, holding all the others fixed. In: Derivatives and the chain rule
- partial failure
- When some components of a system fail while others keep working, so the system as a whole is neither fully up nor fully down. In: Why distributed systems are hard
- partition key
- The value used to pick a message's partition; messages with the same key always go to the same partition and keep their relative order. In: Ordering, partition keys and parallelism
- Partitioning
- Splitting a dataset into disjoint pieces, each stored and served by a different database node. In: Partitioning and sharding
- partitions
- One ordered, append-only log that holds a subset of a topic's messages; the unit of parallelism, replication and ordering in Kafka. In: Kafka internals: the distributed log
- passive health checks
- Detecting failing servers by observing errors and timeouts on real requests, rather than only by separate health probes. In: What a load balancer does
- patch embedding
- The vector a vision transformer gets by multiplying a flattened image patch by a learned matrix; it plays the role of a word embedding. In: Vision transformers
- pCTR
- Predicted click-through rate: the model's estimate of the probability that this user clicks this ad if it is shown in this slot now. In: Design an ad click-through-rate prediction system
- PDDL
- Planning Domain Definition Language: the standard language for classical planning problems, split into a domain file (types, predicates, actions) and a problem file (objects, initial state, goal). In: Classical planning
- Probability density function: a curve whose area over an interval is the probability of landing in that interval. In: Probability fundamentals
- PEAS
- Performance measure, Environment, Actuators, Sensors: a four-part description of an agent's task. In: Agents and environments
- PEFT
- Parameter-efficient fine-tuning: methods that adapt a large model by training only a small number of new or selected parameters. In: Parameter-efficient fine-tuning: LoRA and QLoRA
- percentiles
- The value below which a given share of measurements fall. p99 is the latency 99 percent of requests beat. In: Latency vs throughput, and why percentiles matter
- percept
- What an agent senses at one moment, such as a camera frame, a reading or a message. In: Agents and environments
- perceptron
- The original learning neuron (1958): a weighted sum with a step activation, trained by nudging weights after each mistake. It can only separate classes with a straight line. In: The perceptron and the artificial neuron
- Permutation importance
- The drop in a model's score on held-out data when one feature's values are randomly shuffled, breaking its relation to the target. In: Feature importance and SHAP
- perplexity
- The main t-SNE setting: roughly the effective number of neighbours each point pays attention to. Typical values are 5 to 50. In: t-SNE and UMAP
- persona
- A described character or role the model is asked to play, such as a friendly tutor or a terse code reviewer, which shapes its tone and focus. In: System prompts and personas
- personalized PageRank
- A PageRank variant where the random surfer keeps jumping back to one start node, so scores measure how close every node is to that node in the graph. In: Graph-based recommendations
- PGD
- Projected gradient descent attack: repeat small FGSM-style steps, projecting back into the allowed epsilon box after each one. In: Robustness and adversarial examples
- phi accrual failure detector
- A failure detector that outputs a continuous suspicion level (phi) based on the observed distribution of heartbeat arrival times, instead of a yes/no timeout. In: Membership, gossip and failure detection
- pickle
- Python's built-in serialization format. A pickle file is a list of instructions that can import and call any function while it is being loaded. In: How to read a model page
- PII
- Personally identifiable information: any data that identifies a person directly (name, email, phone, card or ID number) or can identify them when combined with other data. In: Privacy and PII handling
- pinball loss
- An asymmetric loss for quantile forecasts: under-forecasts cost p times the miss and over-forecasts cost (1 - p) times the miss (up to a constant factor), so minimizing it gives the p-th quantile. In: Backtesting and forecast evaluation , Design an ETA prediction system , Design a demand forecasting system
- pipeline
- The Transformers function pipeline(task, model) that bundles preprocessing, the model and postprocessing, so one call goes from raw input (text, an image, audio) to a readable result. In: Hugging Face in ten minutes: the cool stuff first
- Pipeline parallelism
- Splitting a model's layers into consecutive stages on different GPUs and streaming micro-batches through them so the stages work at the same time. In: GPUs and distributed training
- pixel
- Picture element: one cell of the grid that makes up a digital image, holding a brightness value (grayscale) or several colour values. In: Images as tensors
- PKCE
- Proof Key for Code Exchange: the client sends a hash of a random secret when starting the flow and the secret itself when exchanging the code, so a stolen authorization code is useless to an attacker. In: OAuth 2.0, OpenID Connect and JWT
- Plan-space planning
- Planning that searches over partial plans instead of world states: it adds steps, ordering constraints and causal links, committing to an order only when it must. In: Classical planning
- Platt scaling
- Calibrating scores by fitting a logistic regression with one input, the model's raw score, on held-out data. In: Probability calibration
- PMF
- Probability mass function: the probability of each separate value of a discrete random variable. In: Probability fundamentals
- point-in-time correct
- Joining feature values exactly as they were at the time of each training example, never using later values. In: Data leakage
- point-to-point queue
- A messaging model where each message is delivered to exactly one of the consumers reading the queue, and is removed once that consumer acknowledges it. In: Message queues vs pub/sub vs event streams
- pointwise
- Learning-to-rank methods that score each item independently, by regression or classification on its relevance label. In: Learning to rank
- poison message
- A message that fails processing every time it is attempted, usually because of bad data or a bug, and can block or crash consumers if retried forever. In: Retries, dead letter queues and poison messages
- policy
- A rule that says which action to take in every state. In an MDP, an optimal policy maximizes expected total (discounted) reward from every state. In: Sequential decisions: MDPs (bridge to RL) , RLHF , RL fundamentals and MDPs
- Policy gradient
- Reinforcement learning methods that represent the policy directly as a parameterized distribution over actions and adjust its parameters by gradient ascent on expected return. In: Policy gradients and actor-critic, PPO
- policy network
- A neural network that takes a position and outputs a probability for each legal move, an estimate of which moves a strong player would choose. In: Monte Carlo Tree Search and AlphaGo
- polyglot persistence
- Using several different database technologies in one system, each for the data and queries it handles best. In: Choosing the right database
- Polynomial regression
- Linear regression on powers of a feature (x, x squared, x cubed and so on), which lets a linear model fit curves. In: Polynomial regression and basis functions
- Pooling
- A layer that downsamples a feature map by summarizing each small window, usually with its maximum (max pooling) or its mean (average pooling). In: Pooling and CNN architecture
- Popularity bias
- The tendency of a recommender to over-recommend already popular items, because they have the most interactions to learn from and win most comparisons. In: Cold start, diversity and feedback loops
- population
- Every item you care about, such as all customers or all future emails. Usually too big or impossible to measure completely. In: Statistics: sampling, CLT and confidence intervals
- population stability index
- Population stability index: a number that measures how much a variable's distribution has moved between a reference sample and a current sample, summed over bins. In: Monitoring, drift and retraining
- port
- A number from 0 to 65535 that identifies which program on a machine should receive traffic. Web servers usually listen on 443 for HTTPS. In: Networking basics: IP, TCP, UDP and ports
- position bias
- The tendency of users to interact more with items shown higher in a list, regardless of their relevance, which biases click logs. In: Design a feed ranking system
- position embedding
- A vector added to each token that encodes its position in the sequence; in ViT it tells the model where in the image a patch came from. In: Vision transformers
- positional encoding
- Information about each token's place in the sequence, added to or mixed into its vector so that order-blind attention can tell positions apart. In: Embeddings and positional encoding
- posting list
- The sorted list of document ids (and often term frequencies and positions) that contain a given term in an inverted index. In: Inverted indexes and full-text search
- Power
- The probability a test detects a real effect of a given size: 1 minus beta. In: Hypothesis testing and p-values , Offline vs online evaluation and A/B tests
- Power of two choices
- Pick two servers at random and send the request to the less loaded of the two. In: Load balancing algorithms
- PPO
- Proximal policy optimization: an actor-critic method that reuses each batch for several epochs while clipping the probability ratio between new and old policy, so no single update moves the policy far. In: Policy gradients and actor-critic, PPO
- pre-signed URL
- A URL to an object in storage that carries a signature and an expiry time, so anyone holding it can read that one object until it expires. In: Design a pastebin
- precision
- Of all the items a system flagged, the share that were actually correct. In: Capstone: AI code reviewer , Can I run it? What a model needs , Imbalanced data
- precision-recall curve
- Precision plotted against recall as the threshold sweeps. A random model's precision equals the positive rate, so improvements on rare classes are visible. In: Classification metrics
- prediction interval
- A range that should contain the future value with a stated probability, such as 80 percent, given the information at the forecast origin. In: Backtesting and forecast evaluation
- Predictive parity
- Fairness criterion requiring precision (positive predictive value) to be equal across groups: an approval means the same thing whatever the group. In: Fairness and bias
- preference list
- The ordered list of nodes that store copies of a key. In: Consistent hashing , Design a distributed key-value store
- Preference tuning
- Training that pushes a model toward responses people (or an AI judge) prefer, using comparisons between responses. RLHF and DPO are the main methods. In: The LLM training pipeline
- Prefill
- The first phase of generation: the whole prompt is processed in one parallel pass, filling the KV cache. Usually limited by compute. In: Context windows and the KV cache , How LLM inference works
- prefix cache
- Attention keys and values computed for a prompt prefix, kept in GPU memory so a later request that starts with the same text can skip recomputing them. In: Design an LLM chat service (ChatGPT-like)
- Prefix caching
- Keeping the KV cache of a prompt prefix after a request finishes so that later requests starting with the same tokens can reuse it and skip that part of prefill. In: Speculative decoding and other speedups
- presigned URL
- A URL that grants temporary permission for one specific operation on one object, signed by your backend so the storage service can verify it. In: Object storage, block storage and file systems
- pretrained model
- A network whose weights were already trained on a large dataset, such as ImageNet, and are published for reuse. In: Transfer learning and fine-tuning
- Pretraining
- The first and most expensive training stage of a language model: next-token prediction over a very large, broad text corpus, without human labels. In: Pretraining: next-token prediction , The LLM training pipeline
- primacy effect
- A temporary drop in a metric because users must adjust to a change, which recovers as they learn it. In: Offline vs online evaluation and A/B tests
- primary key
- A column, or set of columns, whose value uniquely identifies each row in a table. In: Relational databases and SQL
- primary metric
- The single metric that decides whether the treatment wins, chosen before the experiment starts. In: Offline vs online evaluation and A/B tests
- principal component
- One of the orthogonal directions found by PCA, ordered by how much variance the data has along it. They are the eigenvectors of the covariance matrix. In: Principal Component Analysis
- Principal Component Analysis
- Principal Component Analysis: a linear method that finds orthogonal directions of maximum variance in the data and projects the data onto the top few of them. In: Principal Component Analysis
- prior
- Your belief about something before seeing the new evidence, often the base rate in the population. In: Probability fundamentals , Naive Bayes
- privacy accountant
- The bookkeeping that adds up the privacy cost of every noisy training step to give the total (epsilon, delta) for the trained model. In: Privacy-preserving ML
- privacy budget
- The epsilon in differential privacy: a bound on how much any one person's data can change the probability of any output. Smaller is more private; budgets add up across repeated releases. In: Privacy-preserving ML
- Probabilistic data structures
- Data structures that answer questions such as membership, frequency or distinct count approximately, with bounded error, using far less memory than an exact answer requires. In: Probabilistic data structures
- Probability
- A number between 0 and 1 expressing how strongly an agent believes a statement, updated as evidence arrives. In: Probability for AI (recap) , Probability fundamentals
- problem formulation
- Writing a problem as an initial state, actions, a transition model, a goal test and a step cost, so that a search algorithm can solve it. In: Problem formulation and state spaces
- process
- A running program with its own private memory, file handles and at least one thread. The operating system isolates processes from each other. In: Processes, threads and concurrency
- producer
- A program or service that creates messages and sends them to a queue. In: Why async? Queues explained
- Product quantization
- A compression method that splits each vector into sub-vectors and replaces each with the ID of its nearest centroid in a small learned codebook, so a vector is stored in a few bytes. In: Vector databases and ANN indexes
- progressive validation
- Evaluating an online model on each new batch before training on it, so every example is used for both testing and training without leakage. In: Design an ad click-through-rate prediction system
- projections
- A read model built by processing events into a shape suited to specific queries, such as a table of balances or a search index; it can be deleted and rebuilt from the log. In: Event-driven architecture and event sourcing , Principal Component Analysis
- projector
- A small network, often a single linear layer or a two-layer MLP, that maps a vision or audio encoder's output vectors into the language model's embedding space so they can be used as tokens. In: Multimodal models
- Prolog
- A logic programming language whose programs are facts and rules (definite clauses) and whose execution is backward chaining with unification. In: First-order logic
- prompt
- The text you send to a language model: instructions, context, examples and the input to work on. The model continues from it, so it shapes everything the model produces. In: Prompting fundamentals
- Prompt caching
- A provider feature that stores the processed form (the KV cache) of a prompt prefix so later requests starting with exactly the same prefix can reuse it, at a lower price and latency. In: Prompt caching and cost control , Latency and cost budgets
- prompt injection
- An attack in which instructions hidden in content the model reads, such as a web page, file or email, try to override the user's instructions. In: Coding agents and computer-use agents , Prompt injection and data exfiltration
- prompt registry
- A store of prompt templates where each version is kept under an id, and configuration decides which version is live. In: LLMOps: shipping and iterating
- promptable segmentation
- A model that returns a segmentation mask for any object indicated by a prompt such as a click, a box or a rough mask, without being limited to a fixed class list. In: Image segmentation
- Prophet
- An open-source forecasting library from Meta that fits an additive model of trend with changepoints, Fourier seasonality and holiday effects. In: Prophet and modern forecasting libraries
- proposition
- A statement that is either true or false, written as a symbol such as P or Rain. In: Propositional logic
- Propositional logic
- A logic whose sentences are built from true-or-false propositions joined by not, and, or, implies and if-and-only-if. In: Propositional logic
- Protocol Buffers
- Google's language-neutral format for defining structured messages and encoding them as compact binary. In: gRPC and Protocol Buffers
- provenance
- The recorded history of a piece of media: who or what created it, with which tools, and what edits were made since. In: Video and audio generation overview
- provider
- Under the EU AI Act, the organisation that develops an AI system, or has it developed, and places it on the market or puts it into service under its own name. In: AI regulation and governance
- proxy metric
- A metric that is easy to measure quickly, such as click rate or log loss, used in place of the outcome you actually care about because that outcome is slow, rare or hard to attribute. In: Framing ML problems for products
- Pruning
- Removing weights, neurons or whole channels that contribute little, usually those with the smallest magnitude, and then fine-tuning the remaining network. In: Model compression
- pseudo-residuals
- The negative gradient of the loss with respect to the current predictions; for squared error it equals the ordinary residual. In: Boosting: AdaBoost to gradient boosting
- Pseudonymization
- Replacing each personal value with a consistent placeholder, such as EMAIL_1, while keeping a private mapping so the real value can be restored later. In: Privacy and PII handling
- pub/sub
- Publish/subscribe: a messaging model where publishers send messages to a topic and every subscriber of that topic receives its own copy. In: Message queues vs pub/sub vs event streams
- pull-based
- A design where consumers request work when they have capacity, so the rate of delivery is set by the consumer rather than the producer. In: Backpressure and flow control
Q
- Q-learning
- A reinforcement learning algorithm that learns Q(s, a), the value of each action in each state, from experienced transitions, by nudging each estimate toward the reward plus the discounted best value of the next state. In: Q-learning
- QLoRA
- LoRA fine-tuning on top of a base model stored in 4-bit precision, which cuts the memory for the frozen weights by about four times compared with 16-bit. In: Parameter-efficient fine-tuning: LoRA and QLoRA
- quadtree
- A tree where each node covers a square region and splits into four children when it contains more than a set number of points, so cell size adapts to data density. In: Geospatial indexing
- quantile
- The value below which a given fraction of outcomes fall; the 90th percentile travel time is a time that 90 percent of similar trips beat. In: Design an ETA prediction system
- quantization
- Storing numbers such as weights or activations in fewer bits, for example 8 or 4 instead of 16, by mapping them onto a small set of levels with a scale factor. In: Quantization for inference , Can I run it? What a model needs , Model compression
- query
- In attention, the vector a token uses to ask what it is looking for; it is compared with every key. In: Self-attention, step by step , Seq2seq and attention
- Query transformation
- Rewriting, expanding or splitting the user's question with a language model before retrieval, so the text sent to search matches how the answer is written. In: Contextual retrieval and query transformation
- QUIC
- A transport protocol built on UDP that provides reliable, encrypted, multiplexed streams; each stream is ordered on its own. In: HTTP/1.1, HTTP/2 and HTTP/3
R
- R squared
- The share of the target's variance explained by the model: 1 minus the model's MSE divided by the MSE of always predicting the mean. 1 is perfect, 0 matches the mean, negative is worse than the mean. In: Regression metrics
- RAG
- Retrieval-augmented generation: fetch documents relevant to a query from your own data and put them in the prompt so the model answers from them. In: Prompting vs RAG vs fine-tuning: choosing
- random forest
- Bagged decision trees where each split also considers only a random subset of the features, which makes the trees less alike and their average more accurate. In: Bagging and random forests
- Random search
- Sampling hyperparameter combinations at random from chosen ranges or distributions, for a fixed budget of trials. In: Hyperparameter tuning
- random seed
- The starting value of a pseudo-random number generator; the same seed produces the same sequence of random numbers. In: Experiment tracking and reproducibility
- random variable
- A quantity whose value depends on a random outcome, such as the number shown by a die or tomorrow's temperature. In: Probability fundamentals
- random walk
- A series where each value is the previous value plus an unpredictable shock; its best forecast is the last observed value. In: Stationarity and classical models
- Random-walk methods
- Recommending by simulating short random walks over the interaction graph from a query node and ranking other nodes by how often the walks visit them. In: Graph-based recommendations
- range allocator
- A service that hands out disjoint blocks of ids, such as 1,000 at a time, so servers can assign ids locally without coordinating on each one. In: Design a URL shortener
- rank
- In LoRA, the inner dimension r of the two adapter matrices. It caps how complex the learned weight update can be; common values range from 4 to 64. In: Parameter-efficient fine-tuning: LoRA and QLoRA
- rank-1 matrix
- A matrix that is one column pattern times one row pattern. It needs only m + n numbers to describe, instead of m times n. In: Eigenvectors, eigenvalues and SVD
- rate limiter
- A mechanism that caps how many requests a client (user, API key, IP address) may make in a period, rejecting or delaying the excess. In: Rate limiting algorithms
- rational agent
- An agent that chooses the action expected to maximize its performance measure, given what it has perceived and knows. In: Agents and environments
- RDF
- Resource Description Framework: a W3C standard that represents knowledge as triples, with entities and predicates identified by URIs. It is queried with SPARQL. In: Knowledge graphs and ontologies
- ReAct
- Reasoning and Acting: a prompting pattern in which a model interleaves written reasoning steps with actions such as tool calls, and reads each action's result before reasoning again. In: The agent loop (ReAct)
- read-your-writes
- A guarantee that after a user writes something, that same user's later reads will see the write. In: Replication lag and read consistency guarantees
- reality gap
- The difference between a simulator and the real system (dynamics, sensing, delays, appearance) that makes a policy trained in simulation behave differently on real hardware. In: RL in the real world
- reasoning model
- A language model trained to produce a long chain of intermediate reasoning (often hidden from the user) before its final answer, usually with reinforcement learning on problems whose answers can be checked. In: Reasoning models and test-time compute
- rebalances
- Redistributing a topic's partitions among the current members of a consumer group after a member joins, leaves or fails. In: Kafka internals: the distributed log
- recall
- For nearest neighbour search, the share of the true k nearest neighbours that the approximate search returns, for example 0.95 when it finds 95 of every 100. In: Vector databases and ANN indexes , Capstone: AI code reviewer , Imbalanced data
- Recall@k
- The share of a question's relevant chunks that appear in the top k results, averaged over questions. In: Evaluating RAG , Capstone: chat with your documents (RAG) , Design a visual search system
- receptive field
- The region of the original input image that can influence one value in a feature map. It grows with every convolution and pooling layer. In: Pooling and CNN architecture
- reciprocal rank fusion
- A way to merge ranked lists: each document scores the sum of 1 / (c + rank) over the lists it appears in, with c usually 60. It needs no score calibration. In: Semantic search basics , Hybrid search and reranking , Capstone: chat with your documents (RAG) , Design a search ranking system
- recommender system
- A system that predicts which items a user is likely to want and shows the best few, such as films, products, songs or posts. In: Recommender fundamentals
- Reconciliation
- Regularly comparing two independently produced records of the same activity, such as your ledger and a payment provider's report, and flagging every difference for investigation. In: Reconciliation and correctness jobs
- reconstruction error
- How far a model's rebuilt version of an input is from the input itself, often the mean squared difference. Used as an anomaly score. In: Anomaly detection , Autoencoders
- Recovery Point Objective (RPO)
- Recovery Point Objective: the maximum acceptable amount of data loss, measured as time before the failure, such as 5 minutes of writes. In: Disaster recovery: RPO and RTO
- Recovery Time Objective (RTO)
- Recovery Time Objective: the maximum acceptable time to restore service after a disaster. In: Disaster recovery: RPO and RTO
- recurrent neural network
- A neural network that processes a sequence one element at a time, updating a hidden state vector from the previous state and the current input with the same weights at every step. In: Recurrent neural networks
- Recursive feature elimination
- Recursive feature elimination: train a model, drop the least important feature or features, and repeat until the desired number remains. In: Feature selection
- Redis Cluster
- Redis's built-in sharding: the key space is split into 16,384 hash slots, each owned by one primary node with its replicas. In: Distributed caches: Redis and Memcached
- Redundancy
- Running more copies of a component than strictly needed, so that the failure of one does not take the service down. In: Redundancy, failover and multi-AZ
- regions
- A cloud provider's geographic area, such as eu-west or ap-southeast, containing several availability zones; regions are far apart and largely independent. In: Multi-region architecture
- regression target
- A supervised task whose label is a continuous number, here the travel time in minutes or seconds. In: Design an ETA prediction system
- regressions
- Something that used to work and breaks after a change. In: Why evals are the core skill , Types of learning
- regret
- The reward a bandit strategy lost compared with always playing the best arm: T times the best mean minus the expected reward of the arms actually pulled. In: Multi-armed bandits
- regularization
- Adding a penalty to the loss that discourages complex solutions, such as large weights, to improve generalization. In: Optimization basics , Regularization: ridge, lasso, elastic net , Regularization for deep nets
- regularization strength
- The strength of the regularization penalty, often written lambda or alpha. 0 means no penalty; larger values force smaller weights. In: Regularization: ridge, lasso, elastic net
- REINFORCE
- The basic Monte Carlo policy gradient algorithm (Williams, 1992): run an episode, then raise the log-probability of each action in proportion to the return that followed it. In: Policy gradients and actor-critic, PPO
- reinforcement learning
- Learning to act by trial and error: an agent interacts with an environment, receives rewards, and improves its policy without being given the MDP's model. In: Sequential decisions: MDPs (bridge to RL) , Types of learning , RL fundamentals and MDPs
- relational database
- A database that stores data in tables of rows and columns, links tables through keys, and is queried with SQL. In: Relational databases and SQL
- relevance labels
- Graded judgments by trained raters of whether each result is relevant to a query, used as ground truth for offline evaluation of search and recommendation. In: Design a visual search system
- reliability diagram
- A plot of the observed positive rate against the mean predicted probability for bins of predictions. A calibrated model lies on the diagonal. In: Probability calibration
- ReLU
- Rectified linear unit: max(0, z). It passes positive inputs unchanged and outputs zero for negative ones. In: Activation functions
- remainder
- What is left of a time series after the estimated trend and seasonal components are subtracted; ideally unpredictable noise. In: Time series fundamentals
- replicated state machine
- Several copies of a deterministic program that apply the same log of commands in the same order, so they stay identical. In: Consensus: Raft
- Replication
- Keeping copies of the same data on several machines, so the data survives failures and reads can be spread across copies. In: Replication: leader-follower
- reranker
- A model that scores how well each retrieved candidate answers the query, reading query and candidate together, and reorders the candidates by that score. In: Hybrid search and reranking , Capstone: chat with your documents (RAG)
- reset gate
- The GRU gate that scales how much of the previous state is used when computing the candidate; near 0 lets the unit start fresh from the current input. In: LSTM and GRU
- residual
- The difference between the true value and the model's prediction for one example: y minus y hat. In: Linear regression intuition , Boosting: AdaBoost to gradient boosting
- residual block
- A block that outputs its input plus a learned transformation of it, y = x + F(x), so it only has to learn a correction to the identity. In: Vanishing and exploding gradients, residual connections
- residual connection
- A connection that adds a layer's input to its output, so the layer learns a change to its input rather than a replacement. Makes deep networks trainable. In: Build a tiny GPT from scratch , Vanishing and exploding gradients, residual connections
- residual model
- A model trained to predict the error of another model or rule, so that the two added together are more accurate than either alone. In: Design an ETA prediction system
- residual stream
- The sequence of token vectors that flows through a transformer; each sub-layer reads from it and adds its output back, rather than replacing it. In: The transformer block
- resolver
- A function the GraphQL server calls to produce the value of one field, for example by querying a database or calling another service. In: GraphQL
- resource
- Any thing the API exposes, such as a user, an order or a collection of orders, identified by a URL. In: REST API design
- responsibility
- In a mixture model, the posterior probability that a given point was generated by a given component. In: Gaussian mixture models and EM
- REST
- Representational State Transfer: a style of API design that models data as resources identified by URLs and manipulated with standard HTTP methods. In: REST API design
- REST or gRPC
- Two common ways to call a service over the network: REST sends JSON over HTTP; gRPC sends compact binary messages over HTTP/2 using a schema, which is usually faster and is typed. In: Model serving patterns
- Retrieval
- Finding the passages from your own documents or databases that are relevant to the current question, so they can be added to the prompt. In: Anatomy of an LLM application , Two-tower retrieval and embedding-based retrieval
- Retrieval augmented generation
- Retrieval augmented generation: fetching relevant text from your own data at question time and giving it to the model as context, so its answer is grounded in that text. In: Why RAG?
- retry budget
- A cap on retries as a fraction of normal traffic or as tokens in a bucket, so retries cannot multiply load when a dependency is failing. In: Timeouts, retries, backoff and jitter
- return
- The total of future rewards from a time step on, usually discounted: G_t = r_t+1 + gamma r_t+2 + gamma^2 r_t+3 + ... In: RL fundamentals and MDPs
- reverse proxy
- A proxy that acts for servers: clients connect to it as if it were the server, and it forwards requests to the real servers behind it. In: Reverse proxies (Nginx, Envoy, HAProxy)
- reward
- A single number the environment returns after each action, saying how good the immediate outcome was. The agent's goal is to maximize the total over time. In: RL fundamentals and MDPs
- reward hacking
- When a model being optimised finds ways to score highly on a reward model that do not reflect real quality, such as excessive length or flattery. In: RLHF
- reward model
- A model that takes a prompt and a response and outputs a single score predicting how much people would prefer that response. In: RLHF
- Ridge regression
- Linear regression with an L2 penalty, the sum of squared weights. It shrinks all weights smoothly toward zero but rarely makes any exactly zero. In: Regularization: ridge, lasso, elastic net
- risk-based
- An approach to regulation in which the obligations on a system depend on how much harm its use could cause, with the strictest rules for the riskiest uses and few or none for low-risk ones. In: AI regulation and governance
- RLAIF
- Reinforcement learning from AI feedback: like RLHF, but the preference labels come from an AI model following written principles rather than from people. In: Constitutional AI and RLAIF
- RLHF
- Reinforcement learning from human feedback: train a reward model on human comparisons of responses, then optimise the language model with RL to score highly on it, with a penalty for drifting from the starting model. In: RLHF
- RLVR
- Reinforcement learning on verifiable rewards: training a model with rewards computed by a program that checks the final answer, such as exact match on a math result or passing unit tests, instead of a learned reward model. In: Reasoning models and test-time compute
- RMSE
- Root mean squared error: the square root of MSE, back in the target's units. Always at least as large as MAE, and much larger when there are a few big errors. In: Regression metrics
- RMSNorm
- Root-mean-square normalization: divides each example's features by their root-mean-square and applies a learned scale, without subtracting the mean. A cheaper variant of layer norm. In: Weight initialization and normalization
- RMSProp
- An adaptive optimizer that divides each parameter's gradient by a running root-mean-square of its recent gradients, giving every parameter its own effective step size. In: Optimizers: SGD, momentum, RMSProp, Adam, AdamW
- road graph
- A graph whose nodes are intersections and whose edges are road segments, each weighted by its expected traversal time. In: Design an ETA prediction system
- Robustness
- A model's ability to keep its accuracy when inputs are perturbed, corrupted or drawn from a different distribution than the training data. In: Robustness and adversarial examples
- ROC AUC
- Area under the ROC curve: the probability that the model scores a random positive above a random negative. 0.5 is random, 1.0 is perfect ranking. In: Classification metrics
- ROC curve
- Receiver operating characteristic curve: true positive rate (recall) plotted against false positive rate as the threshold sweeps from high to low. In: Classification metrics
- Role-based access control (RBAC)
- Role-based access control: users are assigned roles such as viewer or admin, and each role grants a set of permissions. In: Authentication vs authorization
- rolling deployment
- Replacing instances of the old version with the new one a few at a time, so capacity is maintained throughout the deployment. In: Deployment strategies
- rolling statistics
- A summary such as a mean or standard deviation over a past window of the series, computed only from data available at forecast time. In: ML and deep learning for forecasting
- rollout
- A fast simulated game from a position to the end, used to estimate who is likely to win. Also called a playout. In: Monte Carlo Tree Search and AlphaGo
- RoPE
- Rotary position embedding: encodes position by rotating each pair of query and key dimensions by an angle proportional to the token's position, so their dot product depends on relative distance. In: Embeddings and positional encoding
- RoPE scaling
- Techniques that extend a RoPE model's context by rescaling positions or rotation frequencies so longer sequences map into ranges the model has seen, usually followed by a short fine-tune on long data. In: Context windows and the KV cache
- router
- A small learned layer in a mixture-of-experts model that scores every expert for a token and picks the top few to process it. In: Mixture of experts , Routing, cascades and cost optimization
- rubric
- A written list of criteria that defines what a passing output must and must not do, used by human or model graders. In: Building an eval set
- rule
- A statement of the form 'if these conditions hold, then this conclusion holds', used to derive new facts from known ones. In: Knowledge representation
S
- saddle points
- A point where the slope is zero but the surface goes up in some directions and down in others, like the middle of a horse saddle. In: Optimization basics
- safe
- A method that does not change server state, so it can be cached, prefetched and retried freely. In: REST API design
- safetensors
- A weights format from Hugging Face: an 8-byte length, a JSON header naming each tensor's type, shape and byte range, then the raw numbers. Loading it cannot run code. In: How to read a model page
- saga
- A business transaction split into a sequence of local transactions, each with a compensating action that semantically undoes it if a later step fails. In: Sagas: choreography and orchestration
- sample
- The subset of the population you actually measure. Its statistics estimate the population's. In: Statistics: sampling, CLT and confidence intervals
- sampler
- The algorithm that decides how much predicted noise to remove at each step of diffusion generation, and how many steps to take (DDPM, DDIM and others). In: Image generation
- sampling distribution
- The distribution of a statistic, such as the mean, across many hypothetical samples of the same size. In: Statistics: sampling, CLT and confidence intervals
- sandbox
- An isolated environment, such as a container or virtual machine, that limits what a program can read, change and reach on the network. In: Coding agents and computer-use agents
- SARIMA
- Seasonal ARIMA, written ARIMA(p,d,q)(P,D,Q)m: adds seasonal differencing and AR and MA terms at multiples of the season length m. In: Stationarity and classical models
- SARSA
- An on-policy temporal-difference method that updates Q(s, a) toward r plus gamma times Q(s', a'), where a' is the action actually taken next. In: Q-learning
- SAT problem
- The Boolean satisfiability problem: deciding whether some assignment of true and false makes a formula true. It is NP-complete, yet modern solvers handle huge industrial instances. In: Propositional logic
- saturation
- When an activation function's input is in a flat region, so its derivative is close to zero and almost no gradient flows back through the neuron. In: Activation functions
- scalers
- Rescaling a numeric feature to a common range or spread, such as mean 0 and standard deviation 1, so no feature dominates by its units alone. In: Feature engineering
- scatter matrix
- A grid of scatter plots, one for every pair of columns, for scanning many relationships at once. In: Exploratory data analysis
- scatter plot
- A plot of one numeric column against another, one dot per row, showing how they move together. In: Exploratory data analysis
- scheme
- The protocol the browser should use. https means HTTP protected by TLS encryption. In: What happens when you type a URL and press Enter
- scratchpad
- The running record of an agent's thoughts, actions and observations, kept in the context so each step can see what came before. Also used for notes or a to-do list the agent maintains. In: The agent loop (ReAct)
- scree plot
- A plot of each principal component's explained variance in order. An elbow where the curve flattens suggests how many components carry signal. In: Principal Component Analysis
- SDK
- Software development kit. For a Space it decides how the app runs: gradio (a Python app), docker (any container) or static (plain HTML, CSS and JavaScript). In: Build and share your first Space
- Seasonality
- A repeating pattern of fixed, known period, such as day of week or month of year. In: Time series fundamentals
- second-price auction
- An auction where the winner pays the smallest amount that would still have won, which for one slot is the runner-up's value. In: Design an ad click-through-rate prediction system
- secret manager
- A service that stores credentials such as passwords, API keys and certificates, controls who can read them, audits access and supports rotation. In: Encryption, keys and secrets
- secure aggregation
- A cryptographic protocol that lets a server learn the sum of many clients' updates without seeing any individual update. In: Privacy-preserving ML
- Self-attention
- A mechanism in which every token computes a weighted average of information from the other tokens in the same sequence, with weights based on how relevant each one is. In: Self-attention, step by step
- self-consistency
- Sampling several reasoning paths for the same question and returning the most common final answer. Also called majority voting. In: Reasoning models and test-time compute , Chain of thought and step-by-step reasoning
- self-hosted model
- A model you run on your own servers or in your own cloud account, so prompts and outputs never go to an outside model provider. In: Privacy and PII handling
- Self-supervised learning
- Learning where the labels are created automatically from the data itself, for example by hiding part of an input and predicting it. In: Types of learning
- semantic cache
- A cache that returns a stored answer when a new question is similar in meaning to a cached one, not only when the text is identical, usually by comparing embeddings. In: Production RAG architecture
- Semantic caching
- Returning a stored answer when a new request is similar in meaning to an earlier one, judged by embedding similarity above a threshold, rather than identical in text. In: Routing, cascades and cost optimization
- semantic layer
- A curated layer of business definitions, such as metrics, dimensions and joins, that sits between users and raw tables so everyone computes a number the same way. In: Capstone: text-to-SQL analyst
- semantic network
- A graph whose nodes are concepts or objects and whose labelled edges are relationships between them, such as is-a and has-part. In: Knowledge representation
- Semantic search
- Search that embeds the query and the documents and returns the documents whose embeddings are nearest to the query's, so it matches meaning rather than exact words. In: Semantic search basics
- Semantic segmentation
- Segmentation that assigns a class label to every pixel without distinguishing separate objects of the same class. In: Image segmentation
- Semi-supervised learning
- Learning from a small labeled set plus a large unlabeled set, using the unlabeled data to improve what the labels alone teach. In: Types of learning
- sensitivity
- The largest amount one person's data can change a query's answer; noise is scaled to it. In: Privacy-preserving ML
- sequence-to-sequence
- A model that reads an input sequence with an encoder and generates an output sequence with a decoder, one token at a time; also called an encoder-decoder. In: Seq2seq and attention
- Server-Sent Events
- A simple standard for a server to push a stream of text events to a client over one long-lived HTTP response with the content type text/event-stream. In: Streaming responses to users , Design an LLM chat service (ChatGPT-like)
- serverless functions
- A cloud model where you deploy individual functions that the platform runs on demand in response to events, scaling automatically and billing only for execution time. In: Serverless and event-driven compute
- Service discovery
- The mechanism by which a service finds the current network addresses of healthy instances of another service. In: Service discovery
- service mesh
- An infrastructure layer that handles service-to-service communication through proxies deployed alongside every service, configured centrally, providing security, reliability and telemetry without application changes. In: Service mesh
- service registry
- A database of the currently running instances of each service, kept up to date by registration and health checks or heartbeats. In: Service discovery
- shadow
- Sending a copy of real requests to the new version without showing its output to users, to compare outputs, cost and latency safely. In: LLMOps: shipping and iterating
- shadow deployment
- Running a new model on a copy of live traffic without using its answers, to compare its predictions, latency and errors with the current model at no risk to users. In: Model serving patterns
- shadow models
- Models the attacker trains on data similar to the target's, so the attacker knows which records were members and can learn what member and non-member outputs look like. In: Privacy-preserving ML
- SHAP
- SHapley Additive exPlanations: attributes one prediction to features using Shapley values from game theory, so the contributions add up exactly to the prediction minus the average prediction. In: Feature importance and SHAP
- shape
- The size of an array along each dimension, such as (2, 3) for 2 rows and 3 columns. In: Your toolkit: Python, NumPy, pandas, scikit-learn, PyTorch , Forward pass by hand
- Shapley value
- A feature's average marginal contribution to the prediction, weighted over all subsets of the other features, as defined in cooperative game theory. In: Feature importance and SHAP
- shard
- One piece of a partitioned dataset, held by its own database node, usually with its own replicas. In: Partitioning and sharding
- shard key
- The column or value used to decide which shard a row belongs to. In: Partitioning and sharding
- shear
- A transformation that slides each row sideways in proportion to its height, turning squares into parallelograms. In: Matrices and linear transformations
- Short-term memory
- What an agent knows within the current task or conversation: the messages, tool results and working notes held in the context window. In: Memory for agents
- shrinkage
- Scaling each new tree's contribution by a learning rate below 1, which needs more trees but usually generalizes better. In: Boosting: AdaBoost to gradient boosting
- Shuffle sharding
- Assigning each customer a random combination of a few workers out of many, so that two customers rarely share exactly the same set and one bad customer can fully disable only a tiny fraction of others. In: Cell-based architecture and shuffle sharding
- sidecar
- A helper process deployed alongside each instance of a service, typically a proxy that handles its network traffic. In: Reverse proxies (Nginx, Envoy, HAProxy) , Service mesh
- sigmoid
- The function 1 / (1 + e^(-z)). It maps any real number to a value between 0 and 1, with sigmoid(0) = 0.5. In: Logistic regression , Activation functions
- signed token
- A piece of data signed by the server with a secret key, so the server can later verify it was not altered. JWTs are a common format. In: Stateless vs stateful services
- significance level
- The false positive rate you accept in advance, often 0.05: reject the null when p is below it. In: Hypothesis testing and p-values
- Simple exponential smoothing
- Forecasting with a weighted average of past observations whose weights decrease exponentially with age, controlled by a smoothing parameter alpha. In: Stationarity and classical models
- Simulated annealing
- Local search that always accepts improving moves and accepts a worsening move with probability exp(-delta / T), where the temperature T falls over time. In: Local search and optimization
- single point of failure
- A component whose failure alone brings down the whole system. In: Redundancy, failover and multi-AZ
- single-answer grading
- Grading one output on its own against a rubric, returning a score or pass or fail. In: LLM-as-judge
- singular value
- A non-negative number on the diagonal of Sigma in the SVD. Sorted from largest to smallest, they measure how much each direction matters. In: Eigenvectors, eigenvalues and SVD
- singular value decomposition
- Writing any matrix as U Sigma V transposed: a rotation, a scaling along perpendicular axes, and another rotation. In: Eigenvectors, eigenvalues and SVD
- skewed
- A distribution with a long tail on one side. Right-skewed data has a few very large values that pull the mean above the median. In: Exploratory data analysis
- SLA
- Service level agreement: a contract with customers that promises a level of service and states the penalty, often service credits, if it is missed. In: SLIs, SLOs, SLAs and error budgets
- SLI
- Service level indicator: a measured number describing one aspect of service quality, such as the share of requests that succeed within 300 ms. In: SLIs, SLOs, SLAs and error budgets
- sliding window
- Keeping only the most recent turns of a conversation that fit a token budget and dropping older ones. In: Memory for agents
- sliding-window attention
- Attention where each token attends only to the most recent W tokens instead of the whole prefix, so cost per token and KV cache size stay bounded by W. In: Context windows and the KV cache
- SLO
- Service level objective: a target for an SLI over a time window, such as 99.9 percent of requests succeed within 300 ms over 30 days. In: SLIs, SLOs, SLAs and error budgets
- sloppy quorum
- Accepting writes on whatever healthy nodes are reachable, even if they are not the key's usual replicas, and handing them back later. In: Multi-leader and leaderless replication
- SMOTE
- Synthetic Minority Over-sampling Technique: create new rare-class examples by interpolating between a rare example and one of its nearest rare neighbours. In: Imbalanced data
- snapshot
- A saved copy of an entity's state as of a specific event version, so replay can start there instead of from the first event. In: Event-driven architecture and event sourcing
- Snowflake
- A 64-bit id scheme that combines a timestamp, a generator (machine or shard) id and a per-millisecond sequence, so every machine can create unique, roughly time-ordered ids independently. In: Generating unique IDs at scale
- socket
- The operating system object a program uses to send and receive network data, bound to an address and port. In: Networking basics: IP, TCP, UDP and ports
- soft clustering
- Clustering in which each point gets a probability of belonging to each cluster instead of a single label. In: Gaussian mixture models and EM
- soft lookup
- A lookup that returns a weighted blend of all stored values according to how well each key matches the query, instead of exactly one value. In: Seq2seq and attention
- soft margin
- An SVM formulation that allows some points inside the margin or on the wrong side, paying a penalty for each, controlled by the parameter C. In: Support Vector Machines and kernels
- soft targets
- A teacher model's full probability distribution over possible outputs, used as a training target. It carries more information than a single correct label, such as which wrong answers are nearly right. In: Distillation and synthetic data
- Softmax
- A function that turns a list of scores into positive weights that sum to 1: exponentiate each score and divide by the total. In: Self-attention, step by step , Multi-class and multi-label classification
- solution
- A sequence of actions leading from the initial state to a state that passes the goal test. In: Problem formulation and state spaces
- Space
- A demo app hosted on the Hugging Face Hub, built with Gradio, Docker or static HTML, whose code lives in a Git repository like a model's. In: Hugging Face in ten minutes: the cool stuff first , Build and share your first Space
- spacetime patches
- Small blocks of a compressed video that span a few pixels in each direction and a few frames in time; video diffusion transformers use them as tokens. In: Video and audio generation overview
- span
- The record of one unit of work, such as a model call or tool call: its name, start time, duration, parent and attributes like tokens and errors. In: LLM observability and tracing , Distributed tracing
- Sparse attention
- Attention patterns where each token attends to a chosen subset of positions (local neighbours, strided positions, a few global tokens) rather than all of them, cutting the quadratic cost. In: Context windows and the KV cache
- sparse matrix
- A matrix format that stores only the non-zero entries and their positions, saving memory when almost all entries are zero. In: Bag of words and TF-IDF
- sparse, high-cardinality features
- Categorical features with millions or billions of distinct values, such as user id, ad id, advertiser id, page URL or query, where each row has only a few non-zero entries. In: Design an ad click-through-rate prediction system
- sparsely activated
- Only a small subset of a model's parameters is used for each input, as in mixture-of-experts layers where each token runs through a few experts. In: Mixture of experts
- sparsity
- The share of a matrix's cells that are empty; user-item matrices are usually more than 99 percent empty. In: Recommender fundamentals
- Specification gaming
- Behaviour that satisfies the literal specification of an objective without achieving the intended outcome, for example exploiting a loophole in a reward function. In: Alignment and interpretability, the big picture
- spectrogram
- A picture of sound: energy at each frequency over time, usually on a perceptual (mel) frequency scale. The standard input to speech models. In: Multimodal models
- Speculative decoding
- A decoding method in which a small, fast draft model proposes several tokens and the large target model checks them all in one parallel pass, keeping those it agrees with. The output distribution is the same as the target model's. In: Speculative decoding and other speedups
- split brain
- Two nodes both believing they are the leader and accepting conflicting writes, usually after a network partition or a false failure detection. In: Replication: leader-follower
- SSTable
- Sorted String Table: an immutable on-disk file of key-value pairs sorted by key, with an index and often a bloom filter. In: Storage engines: B-tree vs LSM-tree
- Stacking
- An ensemble in which a meta-model is trained to combine the predictions of several different base models, using their predictions as its input features. In: Stacking and blending
- standard error
- The standard deviation of a statistic's sampling distribution: how much the estimate would wobble across repeated samples. For a mean it is sigma over the square root of n. In: Statistics: sampling, CLT and confidence intervals
- State
- Data a system remembers between requests, such as who is logged in, what is in a cart or how far a long upload has got. In: Stateless vs stateful services , RL fundamentals and MDPs
- state space
- The set of all states reachable from the initial state, connected by the actions between them. Usually drawn as a graph. In: Problem formulation and state spaces
- stateless
- A service that keeps no client data in its own memory between requests, so any instance can handle any request. In: Stateless vs stateful services
- Static batching
- Grouping requests into a batch that runs until every member has finished, then starting the next batch. Simple, but slots of finished requests sit idle. In: Batching and PagedAttention
- stationary
- A time series whose statistical properties, such as mean, variance and autocorrelation, do not depend on the time at which it is observed. In: Stationarity and classical models
- Stemming
- Cutting words down to a common stem with suffix-stripping rules, without a dictionary; the stem need not be a real word. In: Text preprocessing and tokenization basics
- STL
- Seasonal and Trend decomposition using Loess: a robust, flexible decomposition that lets the seasonal pattern change slowly over time. In: Time series fundamentals
- Stochastic gradient descent
- Stochastic gradient descent: estimating the gradient from one example, or a small random batch, per step instead of the whole dataset. In: Gradient descent, the engine of ML , Optimizers: SGD, momentum, RMSProp, Adam, AdamW
- Stop words
- Very frequent words such as the, and or of that are often removed before counting because they carry little topic information. In: Text preprocessing and tokenization basics
- storage engine
- The part of a database that decides how data is laid out on disk and how reads and writes reach it. In: Storage engines: B-tree vs LSM-tree
- strangler fig pattern
- A migration pattern that gradually replaces a legacy system by routing one piece of functionality at a time to new code, until the old system can be retired. In: Migrating architectures: the strangler fig
- Stratified k-fold
- Cross-validation where each fold keeps the same class proportions as the full dataset. In: Cross-validation
- Stratified sampling
- Sampling separately within each group, such as each topic or customer type, so every group gets enough cases, instead of sampling the whole population at once. In: Building an eval set
- stratified split
- A split that keeps each class's share the same in every part, so a rare class is not over- or under-represented by chance. In: Train, validation and test splits
- stream processing
- Continuously computing results over unbounded data as each event arrives, instead of periodically over a fixed dataset. In: Stream processing
- Streaming
- Sending a model's output to the client piece by piece as it is generated, instead of waiting for the complete response. In: Streaming responses to users , UX patterns for AI
- streaming ASR
- Speech recognition that emits words while the person is still speaking, instead of waiting for the end of the recording. In: Speech: ASR and TTS
- Stride
- How many pixels the kernel moves between positions. Stride 2 skips every other position and roughly halves the output size. In: Convolutions
- STRIPS
- A classic action representation from 1971: each action has preconditions that must hold and effects given as an add list and a delete list of facts. In: Classical planning
- Student-t distribution
- A bell-shaped distribution with heavier tails than the normal distribution. With one degree of freedom it decays like 1 / (1 + d squared). In: t-SNE and UMAP
- stump
- A decision tree with a single split: one question, two leaves. The classic weak learner in boosting. In: Boosting: AdaBoost to gradient boosting
- subword tokenizer
- Tokenization that splits rare words into smaller frequent pieces, so any word can be represented without an unknown token. In: Text preprocessing and tokenization basics
- subwords
- Pieces of words: common words stay whole, rarer words are split into frequent fragments, and anything can still be spelled from small pieces. In: Tokenization
- superposition
- When a network represents more concepts than it has neurons by storing each concept as a direction across many neurons, so individual neurons respond to several unrelated concepts. In: Alignment and interpretability, the big picture
- Supervised fine-tuning
- Training a pretrained model on curated prompt and response examples so it learns to follow instructions in a consistent format. Often called SFT or instruction tuning. In: The LLM training pipeline , Supervised fine-tuning and instruction tuning
- supervised learning
- Learning from examples that each come with the correct answer, so the model learns to map inputs to that answer. In: Types of learning
- support
- The fraction of all transactions that contain every item in an itemset. In: Association rules and market basket analysis
- support vector machine
- A classifier that chooses the decision boundary with the largest margin to the nearest training points of each class, optionally in a kernel-induced feature space. In: Support Vector Machines and kernels
- support vectors
- The training points that lie on or inside the margin. They alone determine the SVM's boundary; removing any other point changes nothing. In: Support Vector Machines and kernels
- SWIM
- Scalable Weakly-consistent Infection-style process group Membership: a protocol that detects failures with direct and indirect pings and spreads membership changes by piggybacking them on those pings. In: Membership, gossip and failure detection
- switchback tests
- An experiment design that alternates treatment and control over time slices within a region, instead of splitting users, to avoid interference in shared marketplaces. In: Design an ETA prediction system
- symlinks
- A file that points to another file. Opening it opens the target. The Hugging Face cache uses them so one stored file can appear in many snapshot folders. In: Accounts, tokens, downloads and datasets
- symmetric (oblivious) trees
- Trees in which every node at the same depth uses the same split, so the tree is a lookup table of 2^depth cells; fast to evaluate and naturally regularized. In: XGBoost, LightGBM and CatBoost in practice
- synthetic data
- Training data generated by a model rather than collected from people or the world, for example instructions, answers or worked solutions written by a strong model. In: Distillation and synthetic data
- system
- All the parts that work together to serve users: clients, servers, databases, caches, queues and the network connecting them. In: What is system design?
- system prompt
- The first message in a conversation, written by the application developer, that sets the model's role, rules and constraints for every turn that follows. Users usually never see it. In: System prompts and personas
T
- t-SNE
- t-distributed Stochastic Neighbor Embedding: a non-linear method that places points in 2D or 3D so that each point's nearest neighbours in the original space stay its nearest neighbours in the map. In: t-SNE and UMAP
- tanh
- The hyperbolic tangent: an S-shaped function from -1 to 1, a rescaled sigmoid centred on zero. In: Activation functions
- Target encoding
- Replacing each category with the average target value for that category, smoothed toward the overall average, computed without using each row's own label. In: Feature engineering
- target network
- A frozen copy of the Q-network, refreshed every C updates, used only to compute the TD targets so they do not shift with every gradient step. In: Deep Q-Networks
- task tag
- The model card field (pipeline_tag) that names the model's task, such as text-generation or image-classification. The Hub uses it for search filters and to pick the page's widget and inference API. In: How to read a model page
- tautology
- A sentence that is true in every model, such as P or not P. In: Propositional logic
- TCP
- Transmission Control Protocol: a connection-based protocol that delivers a reliable, ordered stream of bytes over IP. In: Networking basics: IP, TCP, UDP and ports
- TD error
- The difference between the TD target and the current estimate; the learning signal of temporal-difference methods. In: Q-learning
- TD target
- The value an estimate is moved toward in a temporal-difference update: the observed reward plus the discounted estimated value of the next state. In: Q-learning
- teacher forcing
- Training a sequence decoder by feeding it the true previous token at each step instead of its own prediction, which makes training faster and more stable. In: Seq2seq and attention
- temperature
- A decoding setting that divides logits before softmax: below 1 sharpens the distribution toward the top tokens, above 1 flattens it, and 0 means greedy. In: Decoding and sampling
- temporal-difference
- Learning value estimates from other value estimates one step later, instead of waiting for the final outcome of an episode. In: Q-learning
- tensor
- PyTorch's array type: like a NumPy array, but it can live on a GPU and track operations so gradients can be computed automatically. In: Your toolkit: Python, NumPy, pandas, scikit-learn, PyTorch , Autograd and your first PyTorch training loop , Images as tensors
- tensor parallelism
- Splitting each layer's weight matrices across several GPUs, which compute their parts in parallel and exchange results at every layer. It divides weight and KV cache memory per GPU. In: GPU memory math , GPUs and distributed training
- term
- An election epoch: a number that increases with every election; messages from older terms are ignored, so stale leaders are recognized and rejected. In: Quorums and leader election
- term frequency
- How often a term occurs in one document; the TF in TF-IDF. In: Bag of words and TF-IDF
- test set
- Data held out until the very end and used once to estimate how the final model will perform on new data. In: Train, validation and test splits
- test-time compute
- Computation spent while answering a request, for example by generating a longer chain of reasoning or sampling several answers and picking one. Also called inference-time compute. In: Reasoning models and test-time compute
- text embedding model
- A model that maps a whole piece of text (sentence, paragraph, document) to a single fixed-length vector so that texts with similar meaning have high cosine similarity. In: Sentence and document embeddings
- Text-to-SQL
- Having a language model write a database query from a natural-language question and the database schema, running it, and using the results to ground the answer. In: GraphRAG and structured retrieval
- theoretical saturation
- The point in reviewing examples where new ones no longer reveal new categories or change existing ones. In: Error analysis
- Thompson sampling
- A bandit strategy that keeps a probability distribution over each arm's mean, draws one sample from each, and pulls the arm with the highest sample, so each arm is chosen with the probability that it is the best. In: Multi-armed bandits
- thought
- A short piece of reasoning the model writes before choosing an action, such as 'the search failed because of the date format; retry with an ISO date'. In: The agent loop (ReAct)
- thread
- A sequence of instructions the CPU runs. Threads in the same process share its memory, so they are lighter than processes but can step on each other's data. In: Processes, threads and concurrency
- threshold
- The probability above which a classifier predicts the positive class. 0.5 by default, but it should be chosen from the cost of each kind of error. In: Logistic regression
- Threshold moving
- Choosing the decision threshold from the costs of each type of error, instead of using the default 0.5. In: Imbalanced data
- Throughput
- The number of output tokens a server generates per second across all requests. It sets how many users one machine can serve and so the cost per token. In: How LLM inference works , Latency vs throughput, and why percentiles matter
- time series
- Measurements of the same thing taken in order over time, such as daily sales or a sensor reading every second. In: Data types and datasets , Time series fundamentals
- time to first token
- The time from sending a request until the first token of the answer arrives. In: UX patterns for AI , Design an LLM chat service (ChatGPT-like)
- Time-series cross-validation
- Cross-validation for ordered data where each round trains on an initial stretch of time and validates on the period right after it. In: Cross-validation
- Time-series data
- Data points that are each a timestamp and a value for a named series, written mostly in time order and queried by time range and aggregation. In: Time-series data
- timeout
- A limit on how long to wait for a response before treating the request as failed. It cannot distinguish a dead node from a slow one. In: Why distributed systems are hard
- TLS
- Transport Layer Security: a protocol that encrypts a connection and proves the server's identity using certificates. HTTPS is HTTP over TLS. In: TLS and HTTPS
- TLS termination
- Decrypting TLS at a load balancer or edge proxy and forwarding the request to backends, so certificates and crypto work live in one place. In: TLS and HTTPS
- token bucket
- A rate limiter that adds tokens to a bucket at a fixed rate up to a maximum; each request spends a token, so short bursts up to the bucket size are allowed while the long-run rate is capped. In: Rate limiting algorithms
- Tokenization
- Splitting text into units called tokens, such as words, pieces of words or characters, which are then mapped to integer ids. In: Text preprocessing and tokenization basics
- tokenizer
- The component that splits text into tokens from a fixed vocabulary and maps each token to an integer id; it is trained once, before the model. In: Tokenization
- tokens
- The unit of text a language model reads and writes: a whole word, part of a word, a single character or byte, or punctuation, each with an integer id. In: Tokenization
- tombstone
- A marker written to record that a key was deleted; the old value is removed later when compaction merges past it. In: Storage engines: B-tree vs LSM-tree
- tool calls
- A structured request from the model to run a named tool with JSON arguments, carrying an id that the matching result must refer to. In: Function calling and tool use
- tool schema
- The description of a tool given to the model: a name, a plain-language purpose, and a JSON Schema for its parameters. In: Function calling and tool use
- Top-k
- Sampling only from the k most likely tokens, after renormalising their probabilities to sum to 1. In: Decoding and sampling
- top-level domain
- Top-level domain: the last part of a name, such as .com, .org or .uk. In: DNS in depth
- Top-p
- Nucleus sampling: keep the smallest set of most likely tokens whose probabilities add up to at least p, then renormalise and sample from it. In: Decoding and sampling
- topic
- A named stream of messages in Kafka, such as 'orders', split into one or more partitions. In: Kafka internals: the distributed log
- TPOT
- Time per output token: the average time between consecutive output tokens for one request, which sets the streaming speed a user sees. Also called inter-token latency. In: How LLM inference works
- trace id
- A unique identifier attached to a request at the edge and passed to every service it touches, so logs, metrics and traces from that request can be linked. In: Logs, metrics and traces
- trace viewer
- A tool that displays a trace as a timeline of nested spans, with each span's inputs, outputs and attributes available on click. In: LLM observability and tracing
- traces
- The full record of one request or session: the user input, any retrieved context and tool calls, and the final output. In: Building an eval set , LLM observability and tracing
- Training
- The process of adjusting a model's internal numbers so its predictions on known examples match their labels. In: What is machine learning?
- training pipeline
- An automated, repeatable workflow that turns raw data into a validated, registered model: extract, validate, build features, train, evaluate, register. In: Training pipelines and orchestration
- training set
- The portion of data the model learns its parameters from. In: Train, validation and test splits
- training-serving skew
- A difference between how data is prepared or distributed at training time and at prediction time, which quietly hurts real-world accuracy. In: The ML project lifecycle , ML system architecture overview , Feature stores and training-serving skew , Design an ETA prediction system
- trajectory
- The full record of an agent run: every model message, tool call, tool result and the final outcome. In: Making agents reliable
- transaction
- A group of database operations that the database treats as one unit: either all of them take effect, or none do. In: Transactions and ACID , Association rules and market basket analysis
- transactional outbox
- A pattern where a service writes its business change and an event row to an outbox table in the same database transaction, and a separate relay publishes outbox rows to the message bus. In: Transactional outbox and change data capture
- Transfer learning
- Reusing a model trained on one task or dataset (the source) as the starting point for a different task (the target), instead of training from random weights. In: Transfer learning and fine-tuning
- transformer
- A neural network architecture built on attention, which lets every token look at every other token. It is the basis of modern large language models. In: A short history of AI
- transformer block
- The repeating unit of a transformer: multi-head attention followed by a position-wise feed-forward network, each wrapped in a residual connection and layer normalisation. In: The transformer block
- Transformers.js
- A JavaScript library from Hugging Face with the same pipeline API as transformers, running models with ONNX Runtime in the browser or in Node.js. In: Four ways to run a model
- transient
- A failure caused by a temporary condition, such as a timeout or overloaded server, that may succeed if the same request is tried again later. In: Making agents reliable
- trellis
- A grid with one column per time step and one node per hidden state, with edges for transitions; HMM algorithms fill it in column by column. In: Hidden Markov models
- trend-cycle
- The slowly changing level of a time series: its long-run increase or decrease, including slow cycles that are not of fixed length. In: Time series fundamentals
- trie
- A tree that stores strings character by character so that all strings sharing a prefix share a path from the root; also called a prefix tree. In: Typeahead and autocomplete
- triple
- A single fact in the form (subject, predicate, object), such as (Paris, capital_of, France). A knowledge graph is a large set of triples. In: Knowledge graphs and ontologies
- True negative rate
- Of the outputs a human labelled as good, the share the judge also passes. In: LLM-as-judge
- true positive
- A detection whose IoU with an unmatched ground-truth box of the same class reaches the evaluation threshold, such as 0.5. In: Object detection
- True positive rate
- Of the outputs a human labelled as failures, the share the judge also flags. Also called recall. In: LLM-as-judge
- truncated BPTT
- Running backpropagation through time over windows of a fixed number of steps, carrying the hidden state forward between windows but not the gradient. In: Recurrent neural networks
- trustworthiness
- A score from 0 to 1 for an embedding: how well the map's nearest neighbours are also nearest neighbours in the original space. It penalizes points that look close in the map but were far apart. In: t-SNE and UMAP
- TTFT
- Time to first token: how long a user waits from sending a request until the first output token appears. Driven mostly by prefill and queueing. In: How LLM inference works
- TTL
- Time to live: how many seconds a DNS answer may be cached before it must be looked up again. In: DNS in depth , Feature stores and training-serving skew
- TTS
- Text-to-speech: generating natural-sounding spoken audio from text. In: Speech: ASR and TTS
- Turing test
- Turing's proposal to judge machine intelligence by whether a person conversing in text can tell the machine from a human. In: A short history of AI
- Two-phase commit (2PC)
- A protocol that commits a transaction atomically across several participants: first every participant promises it can commit (prepare), then a coordinator tells all of them to commit or all to abort. In: Two-phase commit
- two-stage detector
- A detector that first proposes candidate regions (with a region proposal network) and then classifies and refines each region in a second stage. In: Object detection
- two-tower model
- A retrieval model with two separate networks: one turns user or query features into a vector, the other turns item features into a vector in the same space, and the score is their dot product. In: Two-tower retrieval and embedding-based retrieval
- type I error
- Rejecting the null when it is actually true: a false positive. Its rate is alpha. In: Hypothesis testing and p-values
- type II error
- Failing to reject the null when there really is an effect: a missed effect. Its rate is beta. In: Hypothesis testing and p-values
U
- U-Net
- An encoder-decoder segmentation network whose decoder concatenates the matching encoder feature maps through skip connections, giving a U-shaped diagram. In: Image segmentation
- ubiquitous language
- The precise shared vocabulary that domain experts and developers use within a bounded context, reflected directly in the code. In: Domain-driven design for architects
- UCB
- A bandit strategy that scores each arm by its average reward plus a bonus that is large when the arm has few pulls, then picks the highest score: optimism in the face of uncertainty. In: Multi-armed bandits
- UCT
- Upper Confidence bounds applied to Trees: choose the child maximizing w/n + c * sqrt(ln N / n), the win rate plus an exploration bonus that shrinks with visits. In: Monte Carlo Tree Search and AlphaGo
- UDP
- User Datagram Protocol: sends individual packets with no connection, acknowledgements, retries or ordering. In: Networking basics: IP, TCP, UDP and ports
- UMAP
- Uniform Manifold Approximation and Projection: a non-linear embedding method that builds a nearest-neighbour graph in the original space and lays it out in low dimensions; similar in spirit to t-SNE, usually faster. In: t-SNE and UMAP
- uncertainty sampling
- An active learning strategy that picks the examples whose predicted probability is closest to a coin flip, where the model is least confident. In: Data collection and labeling
- under-fetching
- Needing several requests to get all the data one screen needs. In: GraphQL
- Underfitting
- When a model is too simple to capture the real pattern, so it does badly on training and new data alike. In: Polynomial regression and basis functions , Bias, variance, underfitting and overfitting
- Unification
- Finding a substitution of terms for variables that makes two expressions identical, such as {x: Alice, y: Bob} for Parent(x, y) and Parent(Alice, Bob). In: First-order logic
- unified diff
- The standard text format for code changes: file headers, then hunks starting with @@ that list removed lines with - and added lines with +, surrounded by unchanged context lines. In: Capstone: AI code reviewer
- unified memory
- On Apple Silicon Macs, a single pool of memory shared by the CPU and the GPU, so a model can use most of the machine's RAM for GPU work. In: Can I run it? What a model needs
- Uninformed search
- Search that uses only the problem definition, with no estimate of how close a state is to the goal. In: Uninformed search: BFS, DFS, uniform cost
- unit root
- A property of a series, like a random walk, where shocks never die out, so the level wanders without returning to a fixed mean; differencing removes it. In: Stationarity and classical models
- universal approximation theorem
- The result that a network with one hidden layer and a non-linear activation can approximate any continuous function on a bounded region as closely as you want, given enough hidden neurons. In: Multi-layer networks
- unrolling
- Redrawing a recurrent network as a feed-forward chain with one copy of the cell per time step, all copies sharing the same weights. In: Recurrent neural networks
- Unstructured
- Data without a fixed set of columns, such as free text, images, audio and video. Models must learn their own features from it. In: Data types and datasets
- unsupervised learning
- Learning from data with no labels, to discover structure such as groups, patterns or a simpler representation. In: Types of learning
- update gate
- The GRU gate that mixes the old state and the candidate: near 1 keeps the old state, near 0 replaces it. In: LSTM and GRU
- upstream gradient
- During backpropagation, the gradient of the loss with respect to a node's output, passed back from the nodes that consumed it. In: Backpropagation
- user interface
- The screens, chat boxes, buttons and notifications through which a person uses the product. In: Anatomy of an LLM application
- user-item matrix
- A table with one row per user and one column per item, holding ratings or interactions; almost all cells are empty. In: Recommender fundamentals
- utility
- A number scoring how desirable an outcome is, letting an agent trade off goals such as speed against safety. In: Agents and environments , Decision theory and utility
- UUID
- A 128-bit universally unique identifier; version 4 is almost entirely random, version 7 starts with a millisecond timestamp so values sort by creation time. In: Generating unique IDs at scale
V
- VAD
- Voice activity detection: a small model that classifies short slices of audio as speech or not speech, used to find where speech starts and stops. In: Speech: ASR and TTS
- validation set
- Data held out from training and used to compare models and tune choices such as hyperparameters and features. In: Train, validation and test splits
- value
- The expected total discounted reward an agent collects starting from a state and acting optimally afterwards. In: Sequential decisions: MDPs (bridge to RL) , Self-attention, step by step
- value model
- A hand-set formula that combines a model's predicted probabilities into one ranking score, with weights chosen by the product team and tuned with experiments. In: Design a feed ranking system
- value network
- A neural network that takes a position and outputs an estimate of the probability of winning from it. In: Monte Carlo Tree Search and AlphaGo
- value of information
- How much the expected utility of the best decision would improve if you learned something before deciding. It is never negative, and is zero if the information could not change the decision. In: Decision theory and utility
- vanishing gradients
- When gradients shrink exponentially as they are propagated back through many layers, so early layers learn extremely slowly or not at all. In: Vanishing and exploding gradients, residual connections , Recurrent neural networks
- variance
- The average squared distance from the mean, measuring spread. Its square root is the standard deviation. In: Distributions you will meet , Bias, variance, underfitting and overfitting
- variational autoencoder
- An autoencoder whose encoder outputs a probability distribution over codes, trained so that sampled codes reconstruct the input and the code distribution stays close to a simple prior, usually a standard normal. In: Variational autoencoders
- vector
- An ordered list of numbers. Geometrically, an arrow from the origin to the point those numbers describe. In: Vectors and vector spaces
- vector clock
- A list of counters, one per node that has coordinated writes to an item; comparing two lists tells whether one version descends from the other or whether they are concurrent. In: Design a distributed key-value store
- vector database
- A database or index specialized in storing embedding vectors and answering approximate nearest-neighbour queries, usually with metadata filtering. In: Vector search in system design
- vector index
- A store built for fast nearest-neighbour search over vectors, usually with approximate indexes such as HNSW. In: Why RAG?
- vector space
- A set of vectors you can add together and scale by numbers, staying inside the set. Every ML feature space is one. In: Vectors and vector spaces
- Vertical scaling
- Making one machine more powerful: more CPU cores, memory, faster disks. Also called scaling up. In: Vertical vs horizontal scaling
- vibe check
- Judging an AI system by trying a few examples by hand and deciding whether the outputs feel right. In: Why evals are the core skill
- virtual environment
- An isolated folder of Python packages for one project, so projects do not break each other's dependencies. In: Your toolkit: Python, NumPy, pandas, scikit-learn, PyTorch
- virtual IP
- An IP address not tied to one machine; it can move between machines, so clients keep using the same address when the machine behind it changes. In: What a load balancer does
- virtual nodes
- Many positions on the ring for one physical server, created by hashing names like cache-a#0, cache-a#1, and so on. In: Consistent hashing , Design a distributed key-value store
- vision transformer
- A transformer applied to images by cutting the image into fixed-size patches, flattening each patch into a vector, and treating the patches as a token sequence. In: Multimodal models , Vision transformers
- vision-language model
- A language model that also accepts images as input and answers in text about what they show, including any text inside them. In: Vision-language models in practice
- visual features
- Vectors produced by a vision encoder that describe the content of image patches in a form a downstream model can use. In: Multimodal models
- visual token
- A token that stands for one patch of an image in a vision-language model's input. Images cost context and money through these tokens, like text does. In: Vision-language models in practice
- Viterbi algorithm
- Dynamic programming over the trellis that keeps, for each state and time, the probability of the best path ending there and a back-pointer, then traces back the single most likely hidden sequence. In: Hidden Markov models
- vocabulary
- The fixed set of tokens a model knows, each mapped to an integer id. In: Text preprocessing and tokenization basics
- vocoder
- The part of a speech synthesis system that turns an acoustic representation, such as a mel spectrogram, into an audio waveform you can play. In: Speech: ASR and TTS
- Voice-to-voice latency
- The time from the moment the caller stops speaking to the moment they hear the first sound of the reply, measured on the caller's side. In: Capstone: voice assistant
- VRAM
- Video RAM: the memory on a graphics card. Models run fast only when their weights fit in it. In: Can I run it? What a model needs
W
- WAPE
- Weighted absolute percentage error: the sum of absolute errors divided by the sum of actuals, so large series count more and zero-demand days do not divide by zero. In: Design a demand forecasting system
- warm pool
- A set of pre-started, idle instances that can take traffic immediately, avoiding the startup delay. In: Autoscaling
- warmup
- A short phase at the start of training in which the learning rate rises from near zero to its peak value, avoiding unstable early updates. In: Learning rate schedules and warmup
- watch
- A subscription to a key or prefix in a coordination service; the service pushes an event to the client whenever the key changes. In: Coordination services (ZooKeeper, etcd)
- watermark
- A signal hidden in the content itself (pixels, audio samples or word choices) that a detector can find later, used to identify media as AI-generated. In: Video and audio generation overview , Stream processing
- Weak supervision
- Creating approximate labels automatically from rules, heuristics or other sources, and combining many such noisy labelers instead of labeling by hand. In: Data collection and labeling
- web application firewall
- Web application firewall: a filter in front of an application that inspects HTTP requests and blocks those matching attack patterns or custom rules. In: DDoS protection and abuse prevention
- webhook
- An HTTP request a provider sends to a URL you registered whenever a specified event happens, carrying the event as JSON. In: Webhooks
- weight
- A number the model learns that multiplies a feature. In linear regression it is the change in prediction per unit of that feature. In: Linear regression intuition , The perceptron and the artificial neuron
- Weight and activation quantization
- Quantizing both the weights and the activations flowing between layers, for example W8A8 (INT8) or FP8, so the matrix multiplications themselves run in low precision. In: Quantization for inference
- Weight decay
- Shrinking every weight slightly towards zero at each step, a form of L2 regularization that discourages large weights. In: Optimizers: SGD, momentum, RMSProp, Adam, AdamW , Regularization for deep nets
- Weight initialization
- Choosing the random starting values of a network's weights, scaled so that activations and gradients keep roughly the same size from layer to layer. In: Weight initialization and normalization
- weight sharing
- Using one set of weights at many positions (every time step in an RNN, every image location in a CNN), so the model size does not grow with the input and what is learned in one place applies everywhere. In: Recurrent neural networks
- Weight-only quantization
- Quantizing only the stored weights (for example to 4 or 8 bits) while activations and the arithmetic stay in 16-bit. Written W4A16 or W8A16. Saves memory and bandwidth. In: Quantization for inference
- Weighted A*
- A* with the heuristic multiplied by a weight w greater than 1. Faster, and the path found costs at most w times the optimum when h is admissible. In: Informed search and A*
- Whitening
- A transform that decorrelates features and scales each to unit variance, so the covariance matrix becomes the identity. In PCA it means dividing each component score by the square root of its eigenvalue. In: Principal Component Analysis
- Wide and Deep
- A model that jointly trains a linear part over sparse features and crosses (memorization) and a deep network over embeddings (generalization), summing their logits before a single sigmoid. In: Design an ad click-through-rate prediction system
- Width
- The number of neurons in a layer. In: Multi-layer networks
- window
- A finite slice of an unbounded stream, defined by time or activity, over which an aggregation such as a count or sum is computed. In: Stream processing
- Winkler score
- An interval score: the interval's width plus a penalty of 2/alpha times the distance by which the actual value falls outside it. In: Backtesting and forecast evaluation
- winsorizing
- Capping extreme values at a chosen percentile, such as the 1st and 99th, so a few huge values do not dominate. In: Data cleaning
- word embedding
- A dense vector of real numbers, typically 50 to 300 dimensions, learned for each word so that words used in similar contexts get similar vectors. In: Word embeddings
- word-level timestamps
- Timing information saying when each word starts and ends in generated or recognised audio. In: Speech: ASR and TTS
- word2vec
- A family of shallow neural models (skip-gram and CBOW) that learn word vectors by predicting words from their neighbours in a sliding window. In: Word embeddings
- worker pool
- A group of identical consumer processes reading from the same queue, so work is shared among them and capacity grows by adding workers. In: Why async? Queues explained
- workflow
- A system in which language model calls and tools are arranged along code paths you defined in advance. In: Workflows vs agents: the patterns
- workflow engine
- A system that runs long-lived business processes durably: it records each step's outcome so a process survives crashes and restarts and resumes exactly where it left off. In: Workflow engines
- working context
- A fixed-size block of text inside the prompt where an agent keeps key facts about the user and task, edited only through memory functions. In: Memory for agents
- working set
- The set of items a workload actively uses over a period of time. In: Eviction policies
- write-ahead log
- An append-only log where every change is recorded and flushed to disk before the data pages themselves are modified, so the database can recover after a crash. In: Transactions and ACID
X
- Xavier (Glorot) initialization
- Initialization with weight variance about 1/n (or 2 / (n_in + n_out)), which keeps signal scale steady through layers with tanh or sigmoid-like activations. Also called Glorot initialization. In: Weight initialization and normalization
- XGBoost
- An open-source gradient boosting library known for regularized objectives, sparsity-aware splits and scalable distributed training. In: XGBoost, LightGBM and CatBoost in practice
Z
- z-score
- How many standard deviations a value lies from the mean: z = (x - mean) / std. In: Anomaly detection
- zero-shot
- Applying a pretrained model to a new task or data set without any training on it. In: ML and deep learning for forecasting
- ZeroGPU
- Hugging Face's shared GPU pool for Spaces. A GPU is attached only while a request runs, so one GPU serves many Spaces. In: Hugging Face in ten minutes: the cool stuff first , Build and share your first Space