The Transformer Block: Heads, Residuals and Normalisation · seed 1 · A4, ink-friendly. The answer key prints on its own page for grown-ups.

Inside one transformer block

Computing · Machine Learning · ages 22-23
Name ______________________   Date ____________

  1. What does normalisation do inside the block?

    • It deletes rare words
    • It holds the scale steady
    • It shuffles the positions

  2. Without position codes, what can an unmasked block not do?

    • Tell an order from a shuffle
    • Blend values from other positions
    • Score a query against the keys

  3. The position-wise network mixes information across positions.

    Circle one:   True   False

  4. A deep stack trains poorly after residuals are cut. Why?

    • The heads can no longer watch different relations
    • Normalisation now runs on the wrong scale
    • The signal and gradient degrade with depth

  5. Trace a vector: after attention mixes, what happens next?

    • The input is added back through the residual
    • The vector is passed to the next block untouched
    • The other positions are mixed in a second time

  6. Why prefer four narrow heads over one wide head of equal size?

    • Narrow heads train with less data
    • Each head can track a different relation
    • One wide head cannot share a residual path

  7. A student claims the residual copies attention's output forward unchanged. What is wrong?

    • Nothing, that is exactly right
    • It adds the sub-layer input back, not the output
    • Residuals carry the gradient backward but no values forward

  8. In an unmasked model, shuffling the input only shuffles the output, and nothing else changes. What failed?

    • The residual path is carrying too much signal
    • The normalisation is holding the scale too steady
    • Position encoding is missing or ignored
LightMySky · lightmysky.comW1-mt_eKC2IehaDT-s1

Answer key

For grown-ups. Fold this page away before handing over the rest.

Inside one transformer block W1-mt_eKC2IehaDT-s1

  1. It holds the scale steady · Steady scale keeps the deep stack trainable.
  2. Tell an order from a shuffle · Attention alone treats the sequence as an unordered set.
  3. False · It transforms each position alone; mixing is attention's job.
  4. The signal and gradient degrade with depth · Residuals carry signal through depth; without them deep stacks become very hard to train.
  5. The input is added back through the residual · Residual plus norm closes the attention half before the network half.
  6. Each head can track a different relation · Splitting the watching among several heads beats one head watching everything at once.
  7. It adds the sub-layer input back, not the output · The residual restores the input alongside the transformed result.
  8. Position encoding is missing or ignored · Order-blindness is the signature of missing position codes.
Worksheet · LightMySky