An agent can reach the right answer by a terrible route, or a sensible route to a wrong answer. Measure both.
3 minute read
In short
Measure outcome (did it succeed) and trajectory (were the steps sensible and safe).
Run each task several times; pass@k and pass^k tell very different stories.
Track cost and safety violations alongside success.
In plain words
Evaluating an agent means running it on a set of realistic tasks and checking two things. The outcome: did it succeed? The trajectory: were the steps sensible, safe and efficient? Outcome is usually checked by code (do the tests pass, is the database in the right state).
Agents are random, so run each task several times. Two summaries matter: pass@k, the chance it succeeds at least once in k tries, and pass^k, the chance it succeeds every time.
Worked example: is it reliable enough to ship?
On 50 tasks, the agent succeeds 34 times: 68%.
1
pass@3: succeeds at least once in 3 tries
Fine if a person can retry or pick the best result.
1 − (1 − 0.68)³ = 0.967
2
pass^3: succeeds all 3 times
What matters if it runs unattended and users expect it to just work.
0.68³ = 0.314
3
Same agent, very different stories
97% sounds ready; 31% doesn't. Choose the measure that matches how the agent will be used.
What to check on the trajectory
check
why
Steps and tokens used
Cost and speed
Any unsafe or forbidden tool calls
Safety, even if the outcome was fine
Repeated or pointless calls
Signs of confusion
Did it verify before claiming success
Reliability
Your turn
An agent books meetings for users with no one checking. Which number best tells you if it's ready?
What to remember
Build tasks from real use, with a clear automatic check of success. Run each several times. Track success, cost and safety violations together, and read failed trajectories to learn what to fix.
Common mistakes
Scoring only the final answer and ignoring dangerous or wasteful actions along the way.
Running each task once; agents vary a lot between runs.
Testing against live systems where results change, making runs impossible to compare.
Check yourself
Answer in your own words, out loud or on paper, before you open the model answer.
How would you evaluate an AI agent?
I'd build a set of realistic tasks, each with a clear way to check success, ideally automatic, like 'the tests pass' or 'the right record was updated'. I'd run them in a sandbox with fixed tools and data, so results are repeatable, and run each task several times, because agents are inconsistent. I'd measure success rate, but also cost, number of steps, time, and safety: did it take any action it shouldn't have, even on tasks it completed? Then I'd read failed traces to group failures, like wrong tool choice or giving up early, which tells me what to fix. And I'd compare against a simpler workflow baseline to justify using an agent at all.