epochs

Learn · Module 10 · Agents · Lesson 5 of 6

Evaluating agents

An agent can reach the right answer by a terrible route, or a sensible route to a wrong answer. Measure both.

3 minute read

In short
  • Measure outcome (did it succeed) and trajectory (were the steps sensible and safe).
  • Run each task several times; pass@k and pass^k tell very different stories.
  • Track cost and safety violations alongside success.

This lesson is part of Pro

The first two lessons of every module are free. Pro opens every lesson and every project.

See Pro plansFrom ₹119 a month. No auto-renewal.