Beyond Accuracy: Precision, Recall and the Cost of an Error
On imbalanced data a model that always says no can be right almost every time and be useless. The confusion matrix separates the two kinds of mistake, and precision, recall and the curves over thresholds say which model suits which cost.
What a learner can do afterwards
- Build a confusion matrix and compute precision and recall from it
- Show a case where high accuracy hides a model that never finds the rare class
- Choose a threshold from a stated cost of a miss against a cost of a false alarm
1 · Read
Accuracy says how often the model is right overall, counting every guess the same way. You already keep one set of data untouched, the testing set, and score on it once at the end so the number stays honest. A model great on training but poor on new data is overfitting: it memorised the practice. What it still hides is which mistakes the model makes, and on rare events that is everything.
Picture 100 patients where 99 are healthy and 1 is sick. A lazy model that always says healthy scores 99 percent, yet finds zero sick patients. Overall accuracy looks superb while the rare class is missed entirely. That is the trap: on imbalanced data, always guessing the common answer is accurate and useless.
The confusion matrix escapes the trap by separating the two kinds of mistake: the miss, a sick patient called healthy, and the false alarm, a healthy patient called sick. Precision is the share of yes calls that are truly sick, and recall is the share of sick patients found. Threshold curves then say which model suits which cost. Binary problems like spam or not spam live here, since there are exactly two mistakes to weigh.
Choose the threshold from the stated costs. When a miss costs far more than a false alarm, catch more cases even at the price of extra alarms.
Split train from test, distrust lone accuracy on rare events, and set the threshold from the cost of each mistake.
2 · Watch
Take it off screen
Where it sits
8 questions wait behind this lesson, each with its answer explained. Every answer feeds the sky: stars light as they are learned, and dim when it is time to come back.