Accuracy is the first number everyone reports and often the least useful one.
3 minute read
In short
A confusion matrix counts true and false positives and negatives.
Precision measures how trustworthy the flags are; recall measures how much was caught.
With rare classes, accuracy is misleading; choose the metric from the cost of each kind of mistake.
In plain words
A classifier makes two kinds of mistakes: false alarms and misses. A confusion matrix counts each outcome: true positives (caught), false positives (false alarms), false negatives (missed) and true negatives (correctly ignored).
From those counts come three key numbers. Precision: of everything flagged, how much was right? Recall: of everything that should have been flagged, how much was caught? F1 combines them into one score.
Worked example: a fraud detector
A model checks 1,000 transactions, of which 60 are fraud.
flagged as fraud
not flagged
actually fraud
40 (true positive)
20 (false negative)
actually fine
10 (false positive)
930 (true negative)
1
Accuracy: share of all answers that were right
(40 + 930) ÷ 1000 = 0.97
2
The trap
A useless model that never flags anything is right on every non-fraud transaction.
940 ÷ 1000 = 0.94 accuracy, catching zero fraud
3
Precision: of 50 flagged, how many were fraud?
40 ÷ 50 = 0.80
4
Recall: of 60 frauds, how many were caught?
40 ÷ 60 = 0.667
5
F1: the harmonic mean of the two
2 × 0.80 × 0.667 ÷ (0.80 + 0.667) = 0.727
Accuracy says 97%. Recall says a third of the fraud got through. When one class is rare, accuracy hides exactly the mistakes you care about.
Your turn
You're building a screening test for a serious disease. A positive result leads to a cheap second test. Which number matters most?
Where you'll see this
Most classifiers output a probability, and moving the threshold trades precision for recall. Curves like ROC and precision-recall show the whole trade-off at once. Pick the metric from the cost of each mistake, then pick the threshold.
Common mistakes
Reporting accuracy on imbalanced data, where predicting the majority class looks excellent.
Comparing precision or recall without fixing the threshold; both change as the threshold moves.
Choosing a metric without asking which mistake is more expensive for the business.
Check yourself
Answer in your own words, out loud or on paper, before you open the model answer.
Explain precision and recall with an example.
Precision asks: of everything the model flagged, how much was actually positive? Recall asks: of all the real positives, how many did the model catch? Take a spam filter. If it flags 100 emails and 90 are spam, precision is 90%. If there were 120 spam emails in total, recall is 90 out of 120, 75%. There's a trade-off controlled by the threshold: flag more and recall rises but precision falls. Which matters more depends on the cost. For spam, a false positive hides a real email, so precision matters. For cancer screening, missing a case is worse, so recall matters. F1 combines both when you need one number.