epochs

Learn · Module 14 · ML system design · Lesson 2 of 5

Designing a recommender

You can't score millions of items for every user in a few hundred milliseconds. Every large recommender solves that with a funnel.

3 minute read

In short
  • Recommenders use a funnel: candidate generation, ranking, re-ranking.
  • Each stage is more expensive per item and sees fewer items.
  • Choose the predicted signal carefully, and handle cold start and feedback loops.

In plain words

A recommender picks a few items for a user from a huge catalogue. The standard design has stages, each narrowing the field with a more expensive model: candidate generation finds a few thousand plausible items quickly, ranking scores those carefully, and re-ranking applies business rules like variety and freshness.

Worked example: a video app's home feed

  1. 1
    Catalogue

    10 million videos. Far too many to score with a heavy model in time.

    10,000,000
  2. 2
    Candidate generation

    Several cheap sources: videos similar to recently watched ones (embedding search), popular in the user's region, new from followed creators.

    → about 1,000 candidates in ~30 ms
  3. 3
    Ranking

    A richer model scores each candidate using user history, video features and context, predicting things like 'will they watch more than half?'

    → 1,000 scores in ~50 ms, keep the top 100
  4. 4
    Re-ranking

    Mix in variety (not 20 videos from one creator), remove already-seen ones, add a little exploration of new videos.

    → final 20 shown

Decisions to explain

decisionoptions and trade-off
What to predictClicks are easy but reward clickbait; watch time and satisfaction are better signals
New users (cold start)Popular items, onboarding questions, region and device
New itemsUse the item's content (title, embedding) until it has interactions; give it some exposure
Feedback loopsThe model only learns from what it showed; keep a little random exploration
Your turn

Why not run the ranking model on all 10 million videos for each request?

What to remember

Measure offline (does ranking put watched items near the top?) and online with A/B tests on real behaviour like watch time and return visits. Name the risks: feedback loops, popularity bias and clickbait. Good designers question the metric, not just optimise it.

Common mistakes

  • Proposing one big model that scores the whole catalogue for every request.
  • Optimising for clicks without discussing clickbait and long-term satisfaction.
  • Forgetting new users and new items, which have no history.
Check yourself

Answer in your own words, out loud or on paper, before you open the model answer.

Design a recommendation system for a video app.

I'd clarify scale and the goal first, say 10 million videos and a goal of meaningful watch time, not just clicks. Then a funnel. Candidate generation pulls about a thousand items from cheap sources: embedding similarity to recently watched videos, popular videos in the user's region, and new uploads from followed creators. A ranking model then scores those candidates using user history, video features and context, predicting outcomes like watching more than half. A re-ranking step adds variety, removes already-seen videos and mixes in some exploration. For new users I'd use popularity and onboarding choices; for new videos, their content features. I'd evaluate offline with ranking metrics, then with A/B tests on watch time and return visits.

More to think about
  • How would you handle new users and new items?
  • Why might optimising for clicks be a bad idea?

Go further

Was this lesson clear?