Design a search ranking system

The problem

A shopper types "quiet headphones for flights". Another types "wh-1000xm5 case". A third types "red trail running shoes size 10". The catalog has 500 million items. Each query needs a ranked page in a couple of hundred milliseconds, and the first three results decide whether the shopper buys or leaves.

Unlike a feed ( Design a feed ranking system ), search has an explicit request: the query. That makes relevance measurable, and it makes failure obvious. The hard parts are that queries are short and ambiguous, the words a shopper uses rarely match the words in the listing, exact tokens such as model numbers must match exactly, and the clicks you learn from are biased toward whatever you ranked first last time.

Think of a skilled shop assistant. They first work out what you mean ("for flights" means noise cancelling, "size 10" is a filter, not a word to match). They pull everything plausible from the stockroom, both items labelled with your words and items that fit what you described. Then they line up the best few on the counter, putting the ones people like you usually buy first. That is query understanding, retrieval, and ranking.

Requirements

Functional

• Given a query, optional filters and a user context, return a ranked page of results with facets (brand, price, size).
• Handle spelling mistakes, synonyms, plurals, other languages and exact identifiers (model numbers, ISBNs).
• Respect hard constraints: in stock, ships to the user's country, matches selected filters.
• Log each query, the results shown with positions, and clicks, add-to-carts and purchases.

Non-functional

• Latency: P99 under 200 ms end to end; the learning-to-rank step gets perhaps 30 to 50 ms.
• Freshness: new listings and price or stock changes searchable within minutes.
• Scale: thousands of queries per second at peak over hundreds of millions of items.
• Quality: measured offline with graded relevance judgements and online with A/B tests on purchases.

Out of scope

• Autocomplete and query suggestions ( Design search autocomplete ).
• Sponsored results and their auction ( Design an ad click-through-rate prediction system ).
• Indexing infrastructure (sharding, replication of the inverted index), covered in Inverted indexes and full-text search .

Estimates

Assumed numbers for a large marketplace, not any company's figures:

# Back-of-envelope sizing for e-commerce search. Every input is an assumption; change them and re-run.
queries_per_day = 200_000_000
peak_factor = 3
catalog = 500_000_000            # searchable items
embedding_dims = 256
bytes_per_dim = 1                # int8-quantized vectors
candidates = 1_000               # from keyword + vector retrieval, merged
reranked = 200                   # scored by the learning-to-rank model
rerank_cpu_us = 20               # CPU microseconds per item for a boosted-tree ranker

peak_qps = queries_per_day / 86_400 * peak_factor
print(f"peak queries per second      {peak_qps:,.0f}")
print(f"vector index size            {catalog * embedding_dims * bytes_per_dim / 1e9:,.0f} GB before graph overhead")
print(f"items scored by the ranker   {peak_qps * reranked / 1e6:,.1f} million per second at peak")
print(f"ranker CPU cores at peak     {peak_qps * reranked * rerank_cpu_us / 1e6:,.0f}")
print(f"candidates merged per second {peak_qps * candidates / 1e6:,.1f} million at peak")

peak queries per second      6,944
vector index size            128 GB before graph overhead
items scored by the ranker   1.4 million per second at peak
ranker CPU cores at peak     28
candidates merged per second 6.9 million at peak

• About 7,000 queries per second at peak. Search QPS is lower than feed QPS, but each query touches a much larger index.
• The vector index is large but feasible: 500 million items at 256 dimensions in 8-bit form is 128 GB of raw vectors before the graph links of an approximate nearest neighbour index. It is sharded across machines like the keyword index.
• Ranking is cheap per item: scoring 200 candidates per query with boosted trees at 20 microseconds each needs about 28 CPU cores at peak. A cross-encoder transformer over the same candidates would need GPUs, which is why it is usually applied only to the top few dozen.

Query understanding

Before any retrieval, turn the raw string into a structured request:

1. Normalize and correct: lowercase, Unicode normalization, spelling correction trained on query logs ("hedphones" becomes "headphones"), but never "correct" a rare exact token like a model number.
2. Segment and tag: "red trail running shoes size 10" becomes color = red, category = trail running shoes, size = 10. A sequence tagger trained on queries with known attributes does this; the tagged parts become filters or boosts instead of words to match.
3. Classify intent and category: predict the product category (shoes, not shoe care), whether the query is navigational (a brand or a specific product) or exploratory, and whether it has commercial intent.
4. Rewrite and expand: add synonyms ("sneakers" for "trainers"), mined from query reformulations (users who searched X and then Y and bought) and from embeddings. Expansion raises recall and can hurt precision, so expanded terms get lower weight.

Language models now handle much of this: a single fine-tuned model can correct, tag and classify. They are expensive per query, so results are cached for frequent queries (a small fraction of distinct queries covers most traffic) and the model runs only on the long tail. AI-08.06 (Contextual retrieval and query transformation) covers query rewriting with language models.

High-level design

Deep dives

Deep dive 1: Hybrid retrieval, BM25 plus embeddings

BM25A keyword ranking function that scores a document by query-term frequency, saturating repeated terms and normalizing for document length, weighted by how rare each term is (IDF). is the workhorse of keyword search. For a query term $t$ in document $d$:

where $f_{t,d}$ is how often $t$ appears in $d$, $|d|$ the document length and $k_1\approx 1.2$, $b\approx 0.75$ the usual constants. Worked example: a term with IDF 2.0 appearing once in a document of average length scores $2.0\times \frac{1\times 2.2}{1+1.2}=2.0$; appearing three times it scores $2.0\times \frac{3\times 2.2}{3+1.2}\approx 3.1$, not 6. Repeats help, but with diminishing returns, so keyword stuffing does little.

BM25 is precise on exact tokens (brand names, model numbers, sizes) and needs no training. It fails when the shopper's words differ from the listing's: "sneakers" against "trainers", "quiet" against "noise cancelling".

Dense retrievalRetrieval by nearest neighbours in a learned vector space, where the query and items are embedded by models trained so that relevant pairs are close. embeds the query and every item with a two-tower model trained on query and purchase pairs, then finds the nearest items with an approximate nearest neighbour index. It handles synonyms and descriptive queries, but blurs exact tokens: to an embedding, "WH-1000XM5" and "WH-1000XM4" are almost the same thing.

Run both and fuse. Reciprocal rank fusionReciprocal rank fusion: combine ranked lists by summing 1 / (k + rank) for each document across the lists, using only ranks so no score calibration is needed. adds $1/(k+\text{rank})$ from each list a document appears in, which needs no score calibration. Or feed both scores as features to the ranker and let it learn when to trust which.

The sample below implements BM25, a stand-in embedding (words mapped to shared concepts, so synonyms match but model numbers mean nothing) and RRF on a ten-item catalog, scored by NDCG@3 against hand-graded relevance.

# Hybrid retrieval on a toy catalog: BM25, a stand-in embedding model, reciprocal rank fusion, NDCG@3.
import numpy as np

docs = ["bose quietcomfort noise cancelling headphones", "cheap wired earbuds", "running shoes for men",
        "sony 55 inch tv", "sony wh-1000xm4 wireless headphones", "leather office chair",
        "trail sneakers waterproof", "sony wh-1000xm5 wireless headphones", "ergonomic desk seat",
        "bluetooth over-ear headset with mic"]
# Stand-in for a learned embedding: words map to concepts, synonyms share one; model numbers map to nothing.
concept = dict(headphones="audio", headset="audio", earbuds="audio", noise="anc", cancelling="anc",
               quietcomfort="anc", wireless="wireless", bluetooth="wireless", shoes="shoe", sneakers="shoe",
               running="sport", jogging="sport", trail="sport", chair="seat", seat="seat", office="work",
               desk="work", work="work", tv="tv")
toks = [d.split() for d in docs]
avg_len, n = np.mean([len(t) for t in toks]), len(docs)

def bm25(q, k1=1.2, b=0.75):
    s = np.zeros(n)
    for w in q.split():
        df = sum(w in t for t in toks)
        idf = np.log(1 + (n - df + 0.5) / (df + 0.5))
        s += [idf * t.count(w) * (k1 + 1) / (t.count(w) + k1 * (1 - b + b * len(t) / avg_len)) for t in toks]
    return s

def dense(q):  # concept overlap (Jaccard): understands synonyms, blind to exact model numbers
    e = lambda text: {concept[w] for w in text.split() if w in concept}
    return np.array([len(e(q) & e(d)) / max(1, len(e(q) | e(d))) for d in docs])

def rrf(*scores, k=10):  # reciprocal rank fusion: sum of 1 / (k + rank) over the documents each list returned
    return sum((s > 0) / (k + np.argsort(np.argsort(-s, kind="stable")) + 1) for s in scores)

def ndcg3(scores, rel):
    dcg = lambda g: sum((2 ** x - 1) / np.log2(r + 2) for r, x in enumerate(g))
    return dcg([rel.get(i, 0) for i in np.argsort(-scores, kind="stable")[:3]]) / dcg(sorted(rel.values())[::-1][:3])

queries = {"wh-1000xm5": {7: 3, 4: 1}, "jogging sneakers": {2: 3, 6: 3},
           "comfy chair work": {5: 3, 8: 3}, "noise cancelling bluetooth headset": {0: 3, 9: 2, 7: 2, 4: 2}}
for q, rel in queries.items():
    print(f"{q:35s} NDCG@3  BM25 {ndcg3(bm25(q), rel):.2f}  dense {ndcg3(dense(q), rel):.2f}"
          f"  hybrid {ndcg3(rrf(bm25(q), dense(q)), rel):.2f}")

wh-1000xm5                          NDCG@3  BM25 0.92  dense 0.00  hybrid 0.92
jogging sneakers                    NDCG@3  BM25 0.61  dense 1.00  hybrid 1.00
comfy chair work                    NDCG@3  BM25 0.61  dense 1.00  hybrid 1.00
noise cancelling bluetooth headset  NDCG@3  BM25 0.86  dense 1.00  hybrid 1.00

• "wh-1000xm5": BM25 finds the exact model (0.92); the embedding has nothing to match on (0.00).
• "jogging sneakers" and "comfy chair work": BM25 finds only the listing that shares a word (0.61); the embedding knows sneakers are shoes and work chairs are desk seats (1.00).
• Hybrid matches the better of the two on every query. That is the case for running both: each covers the other's blind spot.

Deep dive 2: Learning to rank and NDCG

Retrieval returns a few hundred plausible items. A learning-to-rankA model trained to order a list of items for a query, using pointwise, pairwise or listwise losses on graded relevance labels. model orders them using far richer features than retrieval can afford:

• Query-item match: BM25 per field (title, brand, description), embedding similarity, attribute matches (color, size), category match with the predicted query category.
• Item quality: rating, review count, return rate, price relative to the category, image quality, seller reliability.
• Behaviour: historical click-through and conversion for this query and item (debiased), popularity.
• User and context: past brands and price range, location, device (personalization, below).

Gradient-boosted trees with a listwise loss such as LambdaMART are a strong default; a cross-encoder transformer that reads the query and item text together is more accurate and much more expensive, so it re-ranks only the top few dozen. See Learning to rank for the losses.

Measure with NDCG@kNormalized discounted cumulative gain: the sum of graded relevance gains discounted by log of position, divided by the best possible sum, so 1.0 means a perfect order.. With grades $g_i$ at positions $i=\mathrm{1..}k$:

Worked example: results graded 3, 0, 2 give $\text{DCG@3}=7/1+0+3/2=8.5$. The ideal order 3, 2, 0 gives $7+3/1.585=8.89$. NDCG@3 $=8.5/8.89=0.96$: one misplaced item near the top costs only a little, because the discount at position 2 is mild; a perfect item at position 10 is worth far less than at position 1.

Graded labels come from two places: human judgements (raters grade query-item pairs on a guideline, expensive but unbiased, the basis of offline evaluation) and behaviour (clicks, carts, purchases, cheap and plentiful but biased).

Deep dive 3: Click bias and personalization

Clicks are biased by position (people click the top result more), by presentation (images and prices change clicks) and by what you showed at all (items never shown get no clicks). A click modelA probabilistic model of how users scan and click a results list, used to separate the effect of position from the item's real relevance. separates these. The simplest, the position-based model, says $P(\text{click})=P(\text{examine position }k)\times P(\text{relevant})$. Estimate the examination curve with small randomized swaps of adjacent results, then weight training clicks by its inverse ( Design a feed ranking system shows this on feed data). Cascade models add that users scan top-down and stop after a satisfying click, so a click at position 4 also says positions 1 to 3 were seen and judged worse.

Purchases are a stronger and less biased signal than clicks, but rarer. A common label is a graded mix: purchase 3, add-to-cart 2, click 1, skip 0, with skips counted only above the last clicked position.

Personalization adds user features to the ranker: preferred brands, price band, sizes, past purchases, location. Two cautions. Keep query relevance first: a shopper who searches "running shoes" and usually buys one brand should see that brand boosted, not only that brand. And personalize less for navigational queries ("wh-1000xm5" means exactly that product for everyone) than for exploratory ones ("gift for dad").

Try it

1. With the embedding weight at 0 (keyword only), "wh-1000xm5 case" scores 0.93, but switch the query to "quiet headphones for flights": 0.06. None of its best products contain the words "quiet" or "flights".
2. Move the weight to 1 (embedding only). "quiet headphones for flights" rises to 0.98, while "wh-1000xm5 case" falls to 0.74: the embedding no longer tells the XM5 case from the XM4 case.
3. Find the best blend for the mean line: about 0.65, where the mean NDCG@5 is 0.92, higher than keyword only (0.54) or embedding only (0.86). Now select "red trail running shoes": its own best is at 0.30 (0.89). A single fixed weight is a compromise; a ranker that sees the query type can choose per query.

Going deeper

Evaluation and experiments

Offline: NDCG@10 on a fixed set of human-judged queries, stratified by head (frequent), torso and tail queries, because a change that helps head queries often hurts the tail. Also recall@1000 for retrieval alone: the ranker cannot rescue what retrieval missed. Track zero-result rate and the share of queries with spelling corrections or rewrites.

Online: A/B tests on purchases or revenue per search, click-through on the first page, time to first click, and the reformulation rate (searching again right away, a direct sign of failure). InterleavingAn online evaluation that merges the result lists of two rankers into one list per query and credits each ranker with the clicks its items receive, detecting differences with far less traffic than an A/B test. compares two rankers with far less traffic than an A/B test, which makes it a good filter before a full test.

Monitoring

Zero-result and reformulation rates by query segment, latency per stage, index freshness (time from listing update to searchable), the share of traffic answered from the query cache, retrieval recall on a sampled audit set, and click-through by position (a sudden change in the curve often means a front-end change, not a ranking one).

Bottlenecks and failure modes

Trade-offs we made

Extensions

• Multilingual search with a shared embedding space across languages.
• Conversational search: follow-up queries that refine the previous one ("cheaper", "in blue").
• Generative answers above the results for informational queries, grounded in retrieved listings (see AI-08.05, Hybrid search and reranking).
• Learning facets: which filters to show for a query, learned from what shoppers use.

In the wild

Where you will see this: every marketplace, travel site, app store, job board and web search engine runs query understanding, keyword plus embedding retrieval and a learned ranker, and almost all of them learn from clicks only after correcting for position. The same hybrid retrieval is the first stage of retrieval-augmented generation in the AI track ( Hybrid search and reranking ).

Check yourself

Recap