epochs

Learn · Module 11 · Evaluation · Lesson 4 of 5

Comparing two systems

System B scored 82% and system A 78%. Is B actually better, or did it get lucky on this test set?

3 minute read

In short
  • Small score differences on small test sets are often luck.
  • Compare systems on the same questions and focus on where they disagree.
  • Use hundreds of examples, and keep a separate test set from the one you tuned on.

This lesson is part of Pro

The first two lessons of every module are free. Pro opens every lesson and every project.

See Pro plansFrom ₹119 a month. No auto-renewal.