epochs

Learn · Module 16 · ML from scratch · Lesson 5 of 5

Sampling strategies in code

Temperature, top-k and top-p in one function, and a quick check that it matches the maths.

4 minute read

In short
  • Divide by temperature, softmax, filter with top-k or top-p, renormalise, sample.
  • Temperature 0 is greedy decoding: always the top token.
  • Test random code by fixing the seed and checking frequencies over many draws.

This lesson is part of Pro

The first two lessons of every module are free. Pro opens every lesson and every project.

See Pro plansFrom ₹119 a month. No auto-renewal.