---
title: "The Transformer Block: Heads, Residuals and Normalisation"
description: "One block stacks multi-head attention and a position-wise network around a residual path, with normalisation holding the scale steady. Attention treats a sequence as a set, so position has to be suppl"
canonical: https://lightmysky.com/learn/computing/the-transformer-block-heads-residuals-and-normalisation-mt_eKC2IehaDT
source: https://lightmysky.com/learn/computing/the-transformer-block-heads-residuals-and-normalisation-mt_eKC2IehaDT.md
retrieved: 2026-11-12
---

> **Agent view.** This is the Markdown twin of the page, for tools and assistants.
> When to use this site, and the call that answers each job: https://lightmysky.com/agent-instructions.md
> API description (OpenAPI 3.1): https://lightmysky.com/openapi.json · Authentication: https://lightmysky.com/auth.md
> Pricing: https://lightmysky.com/pricing.md · Catalog: https://lightmysky.com/llms.txt · Full catalog: https://lightmysky.com/llms-full.txt
> Every machine-readable file on this domain: https://lightmysky.com/.well-known/ai-catalog.json
> Ask for Markdown with `Accept: text/markdown`, a `.md` address, or `?mode=agent`.

# The Transformer Block: Heads, Residuals and Normalisation

One block stacks multi-head attention and a position-wise network around a residual path, with normalisation holding the scale steady. Attention treats a sequence as a set, so position has to be supplied explicitly or an unmasked block cannot tell an order from a shuffle.

Subject: Computing · Area: Machine Learning · Ages 22 to 23
Page: https://lightmysky.com/learn/computing/the-transformer-block-heads-residuals-and-normalisation-mt_eKC2IehaDT

## Ready when they can

- Trace a single vector through one block and name what each sub-layer changed about it
- Say why several narrow heads are used instead of one wide one
- Predict what fails if the residual path is cut, and what fails if position is never encoded

## Lesson: Inside one transformer block

Attention treats a sequence as a set, so position must be supplied explicitly. Without position codes, an unmasked block is order-blind: shuffle the inputs and the outputs shuffle the same way, with nothing else changed. A causal mask, which lets each position see itself and the positions before it but nothing after, breaks that symmetry, so some order gets through even without codes, and whether the final answer moves depends on how the outputs are read out.

**Example.** Follow one vector through the block. Attention mixes in the other positions, the residual adds the input back, normalisation steadies the scale, and the position-wise network transforms each position alone. Residual plus norm closes each half of the block.

Each piece earns its place. Cut the residual path and deep stacks become very hard to train, because the signal and its gradient degrade layer by layer. Several narrow heads are used instead of one wide head because each head can watch a different relation within the same budget.

For an unmasked block, run the shuffle test when outputs look order-blind. If shuffling the input only shuffles the output, position was never encoded or the codes are being ignored.

**Recap.** Position codes supply order, residuals carry the signal deep, and narrow heads divide the watching.

## Practice

8 questions on this page, each with its working shown.

## Needs first

- [Attention as a Learned Lookup](https://lightmysky.com/learn/computing/attention-as-a-learned-lookup-mt_rrdtS1lHvC)

## Optional connections

- [From Perceptron to Multilayer Network](https://lightmysky.com/learn/computing/from-perceptron-to-multilayer-network-mt_TTa7KN1-IO)

## Opens up

- [Tokenisation and What a Model Actually Reads](https://lightmysky.com/learn/computing/tokenisation-and-what-a-model-actually-reads-mt_PW7s8qEiJD)