Beyond Accuracy: Precision, Recall and the Cost of an Error · seed 1 · A4, ink-friendly. The answer key prints on its own page for grown-ups.

Accuracy can lie, mistakes tell truth

Computing · Machine Learning · ages 20-22
Name ______________________   Date ____________

  1. In a group of 100 where 99 are healthy, an always-healthy model is right how many times?

    Answer: ______________

  2. What does accuracy measure?

    • How often it is right
    • How fast the model trains
    • How rare the positive cases are

  3. The testing set is data the model has never seen, and it tells the truth about real-world behaviour.

    Circle one:   True   False

  4. An always-healthy model scores 99 percent on rare-illness data. Why is it useless?

    • It never finds the rare cases
    • It trains too slowly to deploy
    • It uses too much memory

  5. What does the confusion matrix separate?

    • Training rows from testing rows
    • Fast models from slow models
    • The miss from the false alarm

  6. Missing an illness costs far more than a false alarm. Where should the threshold go?

    • Toward flagging nobody, to avoid all alarms
    • Thresholds cannot move once training ends
    • Move the threshold toward catching

  7. A model scores great on training data but poorly on new data. What is wrong?

    • Underfitting: too simple and missing the pattern
    • Overfitting: memorised practice
    • Balanced fitting: exactly right for deployment

  8. A team reports 99 percent accuracy and celebrates. You learn the model never flags the rare event. Verdict?

    • Success, accuracy settles everything
    • Failing, it misses the whole purpose
    • Success, rare events never matter
LightMySky · lightmysky.comW1-mt_BHGgZM7tak-s1

Answer key

For grown-ups. Fold this page away before handing over the rest.

Accuracy can lie, mistakes tell truth W1-mt_BHGgZM7tak-s1

  1. 99 · It matches all 99 healthy patients and misses the single sick one.
  2. How often it is right · Accuracy counts every guess the same way across the whole set.
  3. True · Unseen data cannot have been memorised, so the score generalises.
  4. It never finds the rare cases · Accuracy counts the easy common guesses and hides the total miss on the rare class.
  5. The miss from the false alarm · It splits the two kinds of mistake so each can be counted and costed.
  6. Move the threshold toward catching · Costly misses justify extra alarms, so the threshold moves toward catching.
  7. Overfitting: memorised practice · The train-test gap is the signature of memorised practice that cannot generalise.
  8. Failing, it misses the whole purpose · High accuracy paired with zero rare finds is the classic failing pattern.
Worksheet · LightMySky