Search lives or dies on the first few results. Ranking is about putting the best ones there, and measuring whether you did.
4 minute read
In short
Search ranking retrieves candidates, then ranks them with query, document and behaviour signals.
NDCG rewards putting relevant results near the top; 1.0 is the ideal order.
Labels from clicks are biased by position; confirm with A/B tests.
In plain words
Search has the same funnel as a recommender, but driven by a query: retrieve candidates with keyword and vector search, then rank them with a model that uses the query, the document and signals like freshness, popularity and the user's past behaviour.
To measure ranking, the standard metric is NDCG. Each result has a relevance grade (say 0 to 3). Results near the top count more: each grade is divided by log₂(position + 1). Divide the total by the best possible total, and you get a score between 0 and 1.
Worked example: NDCG for five results
A query returns five results with relevance grades 3, 2, 0, 1, 2.
position
1
2
3
4
5
grade
3
2
0
1
2
divide by log₂(position + 1)
1.000
1.585
2.000
2.322
2.585
contribution
3.000
1.262
0.000
0.431
0.774
1
Add up the contributions
DCG = 3.000 + 1.262 + 0 + 0.431 + 0.774 = 5.466
2
The best possible order
Sort the same grades: 3, 2, 2, 1, 0.
ideal = 3.000 + 1.262 + 1.000 + 0.431 + 0 = 5.693
3
Divide
Close to perfect. The loss comes from the irrelevant result at position 3 pushing a grade-2 result down to 5.
NDCG = 5.466 ÷ 5.693 = 0.960
Your turn
Why does NDCG divide by the log of the position?
What to remember
Get relevance labels from human raters or from behaviour (clicks, purchases), knowing that behaviour is biased towards whatever was already shown on top. Test offline with NDCG, then online with A/B tests. Keep a fast path for exact matches like product codes, where keyword search must win.
Common mistakes
Treating clicks as perfect relevance labels; top positions get clicked more regardless of quality.
Ignoring exact-match queries like product codes, where keyword search must win.
Optimising relevance only and ignoring business needs like stock and margin.
Check yourself
Answer in your own words, out loud or on paper, before you open the model answer.
Design the ranking for an e-commerce search.
I'd use two stages. Retrieval finds a few hundred candidates fast, combining keyword search, which is essential for brand names and product codes, with vector search for meaning, plus filters like category and stock. Then a learning-to-rank model, often gradient-boosted trees, scores each candidate using query-product relevance features, product quality and popularity, price, and the user's history. Labels come from behaviour, like clicks, add-to-cart and purchases, corrected for position bias, plus some human relevance ratings. I'd measure offline with NDCG and online with A/B tests on conversion and revenue per search, and watch guardrails like zero-result rate and latency.