The Transformer Block: Heads, Residuals and Normalisation · seed 1 · A4, ink-friendly. The answer key prints on its own page for grown-ups.

Inside one transformer block

Computing · Machine Learning · ages 22-23
Name ______________________   Date ____________

  1. What does normalisation do inside the block?

    • It deletes rare words
    • It holds the scale steady
    • It shuffles the positions

  2. Without position codes, what can the block not do?

    • Tell an order from a shuffle
    • Add two numbers
    • Store any weights

  3. The position-wise network mixes information across positions.

    Circle one:   True   False

  4. A deep stack trains poorly after residuals are cut. Why?

    • The vocabulary shrank
    • The data ran out
    • The signal and gradient degrade with depth

  5. Trace a vector: after attention mixes, what happens next?

    • The input is added back through the residual
    • The vector is deleted
    • Positions are shuffled randomly

  6. Why prefer four narrow heads over one wide head of equal size?

    • Narrow heads train with less data
    • Each head can track a different relation
    • Wide heads are forbidden

  7. A student claims the residual copies attention's output forward unchanged. What is wrong?

    • Nothing, that is exactly right
    • It adds the sub-layer input back, not the output
    • Residuals carry gradients only, no values

  8. Shuffling the input barely changes a trained model output. What failed?

    • The residuals are too strong
    • The data is too clean
    • Position encoding is missing or ignored
LightMySky · lightmysky.comW1-mt_eKC2IehaDT-s1

Answer key

For grown-ups. Fold this page away before handing over the rest.

Inside one transformer block W1-mt_eKC2IehaDT-s1

  1. It holds the scale steady · Steady scale keeps the deep stack trainable.
  2. Tell an order from a shuffle · Attention alone treats the sequence as an unordered set.
  3. False · It transforms each position alone; mixing is attention's job.
  4. The signal and gradient degrade with depth · Residuals carry signal through depth; without them training stalls.
  5. The input is added back through the residual · Residual plus norm closes the attention half before the network half.
  6. Each head can track a different relation · Division of watching beats one undifferentiated glance.
  7. It adds the sub-layer input back, not the output · The residual restores the input alongside the transformed result.
  8. Position encoding is missing or ignored · Order-blindness is the signature of missing position codes.
Worksheet · LightMySky