---
title: "The Transformer Block: Heads, Residuals and Normalisation"
description: "One block stacks multi-head attention and a position-wise network around a residual path, with normalisation holding the scale steady. Attention treats a sequence as a set, so position has to be suppl"
canonical: https://lightmysky.com/learn/computing/the-transformer-block-heads-residuals-and-normalisation-mt_eKC2IehaDT
source: https://lightmysky.com/learn/computing/the-transformer-block-heads-residuals-and-normalisation-mt_eKC2IehaDT.md
retrieved: 2026-09-12
---

> **Agent view.** This is the Markdown twin of the page, for tools and assistants.
> When to use this site, and the call that answers each job: https://lightmysky.com/agent-instructions.md
> API description (OpenAPI 3.1): https://lightmysky.com/openapi.json · Authentication: https://lightmysky.com/auth.md
> Pricing: https://lightmysky.com/pricing.md · Catalog: https://lightmysky.com/llms.txt · Full catalog: https://lightmysky.com/llms-full.txt
> Every machine-readable file on this domain: https://lightmysky.com/.well-known/ai-catalog.json
> Ask for Markdown with `Accept: text/markdown`, a `.md` address, or `?mode=agent`.

# The Transformer Block: Heads, Residuals and Normalisation

One block stacks multi-head attention and a position-wise network around a residual path, with normalisation holding the scale steady. Attention treats a sequence as a set, so position has to be supplied explicitly or the block cannot tell an order from a shuffle.

Subject: Computing · Area: Machine Learning · Ages 22 to 23
Page: https://lightmysky.com/learn/computing/the-transformer-block-heads-residuals-and-normalisation-mt_eKC2IehaDT

## Ready when they can

- Trace a single vector through one block and name what each sub-layer changed about it
- Say why several narrow heads are used instead of one wide one
- Predict what fails if the residual path is cut, and what fails if position is never encoded

## Lesson: Inside one transformer block

Attention treats a sequence as a set, so position must be supplied explicitly. Without position codes the block cannot tell an order from a shuffle, and swapping two words would barely change its answer.

**Example.** Follow one vector through the block. Attention mixes in the other positions, the residual adds the input back, normalisation steadies the scale, and the position-wise network transforms each position alone. Residual plus norm closes each half of the block.

Each piece earns its place. Cut the residual path and deep stacks stop training, because the signal and its gradient degrade. Several narrow heads beat one wide head because each head can watch a different relation within the same budget.

**Tip.** Run the shuffle test when outputs look order-blind. If shuffling the input barely moves the output, position was never encoded.

**Recap.** Position codes supply order, residuals carry the signal deep, and narrow heads divide the watching.

## Practice

8 questions on this page, each with its working shown.

## Needs first

- [Attention as a Learned Lookup](https://lightmysky.com/learn/computing/attention-as-a-learned-lookup-mt_rrdtS1lHvC)
- [From Perceptron to Multilayer Network](https://lightmysky.com/learn/computing/from-perceptron-to-multilayer-network-mt_TTa7KN1-IO)

## Opens up

- [Tokenisation and What a Model Actually Reads](https://lightmysky.com/learn/computing/tokenisation-and-what-a-model-actually-reads-mt_PW7s8qEiJD)