Beyond Accuracy: Precision, Recall and the Cost of an Error · seed 1 · A4, ink-friendly. The answer key prints on its own page for grown-ups.
En anglais

Accuracy can lie, mistakes tell truth

Computing · Machine Learning · ages 20-22
Name ______________________   Date ____________

  1. In a group of 100 where 99 are healthy, an always-healthy model is right how many times?

    Answer: ______________

  2. What does accuracy measure?

    • How often it is right
    • How fast the model trains
    • How rare the positive cases are

  3. The testing set is data the model has never seen, so a score on it is not inflated by memorised training examples.

    Circle one:   True   False

  4. An always-healthy model scores 99 percent on rare-illness data. Why is it useless?

    • It never finds the rare cases
    • It trains too slowly to deploy
    • It uses too much memory

  5. What does the confusion matrix separate?

    • Training rows from testing rows
    • Fast models from slow models
    • The miss from the false alarm

  6. Missing an illness costs far more than a false alarm. Where should the threshold go?

    • Toward flagging nobody, to avoid all alarms
    • Thresholds cannot move once training ends
    • Move the threshold toward catching

  7. A model scores great on training data but poorly on new data. What is wrong?

    • Underfitting: too simple and missing the pattern
    • Overfitting: memorised practice
    • Balanced fitting: exactly right for deployment

  8. A team reports 99 percent accuracy and celebrates. You learn the model never flags the rare event. Verdict?

    • Success, accuracy settles everything
    • Failing, it misses the whole purpose
    • Success, rare events never matter
LightMySky · lightmysky.comW1-mt_BHGgZM7tak-s1

Answer key

For grown-ups. Fold this page away before handing over the rest.

Accuracy can lie, mistakes tell truth W1-mt_BHGgZM7tak-s1

  1. 99 · It matches all 99 healthy patients and misses the single sick one.
  2. How often it is right · Accuracy counts every guess the same way across the whole set.
  3. True · Unseen data cannot have been memorised, so the score is not flattered by practice. How far it carries to new data depends on that data resembling the test set.
  4. It never finds the rare cases · Accuracy counts the easy common guesses and hides the total miss on the rare class.
  5. The miss from the false alarm · It splits the two kinds of mistake so each can be counted and costed.
  6. Move the threshold toward catching · Costly misses justify extra alarms, so the threshold moves toward catching.
  7. Overfitting: memorised practice · The train-test gap is the signature of memorised practice that cannot generalise.
  8. Failing, it misses the whole purpose · High accuracy paired with zero rare finds is a failing result: the model misses its purpose.
Worksheet · LightMySky