Regularisation: Charging for Complexity · seed 1 · A4, ink-friendly. The answer key prints on its own page for grown-ups.

A charge for big numbers

Computing · Machine Learning · ages 20-21
Name ______________________   Date ____________

  1. Which penalty drives some coefficients exactly to zero?

    • The squared penalty
    • No penalty can do that
    • The absolute penalty

  2. What happens to the coefficients as the penalty grows?

    • They grow without limit
    • They shrink toward zero
    • They freeze immediately

  3. Features must be scaled before a penalty is fair.

    Circle one:   True   False

  4. Why must features be scaled before penalising?

    • Scaling speeds up the computer clock
    • Large-unit features would take unfair penalty share
    • Scaling removes the need for any penalty

  5. Your regression fits training data perfectly with giant coefficients but flops on new data. What do you add, and what should you see?

    • A penalty, with smaller coefficients and better new-data error
    • More features, and even bigger coefficients
    • A smaller training set, with identical coefficients

  6. You need automatic feature selection, not just smaller weights. Which penalty and why?

    • Absolute, because exact zeros drop features out
    • Squared, because smooth shrinking selects features
    • Neither, because penalties cannot select

  7. Someone penalises without scaling: one feature in millimetres dominates the penalty and gets crushed, though it predicts well. What went wrong?

    • The penalty was too small to matter
    • Units posed as guilt, so a good feature was punished
    • Absolute penalties always crush good features

  8. After an absolute penalty, three of ten coefficients are exactly zero. A teammate says those three features were useless all along. What is the careful reading?

    • They were useless, and the proof is final
    • The penalty failed, since zeros mean errors
    • They added nothing given the rest, at this penalty strength
LightMySky · lightmysky.comW1-mt_RVgsDP0XzP-s1

Answer key

For grown-ups. Fold this page away before handing over the rest.

A charge for big numbers W1-mt_RVgsDP0XzP-s1

  1. The absolute penalty · Its sharp corner at zero lets coefficients land on it exactly.
  2. They shrink toward zero · A bigger charge on size pulls every coefficient back.
  3. True · Raw units would decide guilt instead of usefulness.
  4. Large-unit features would take unfair penalty share · Units are not importance, so unscaled penalties punish the wrong features.
  5. A penalty, with smaller coefficients and better new-data error · The penalty trades a little training fit for much better generalisation.
  6. Absolute, because exact zeros drop features out · Exact zeros remove features from the model on the way.
  7. Units posed as guilt, so a good feature was punished · Unscaled units distort the charge, so size beat usefulness.
  8. They added nothing given the rest, at this penalty strength · Zero means unneeded beside the others, not useless in every possible model.
Worksheet · LightMySky