---
title: "Tokenisation and What a Model Actually Reads"
description: "A model never sees letters or words. It sees ids drawn from a vocabulary built by merging frequent pieces of text, and that vocabulary decides what is cheap to express, which spellings are fragile, an"
canonical: https://lightmysky.com/learn/computing/tokenisation-and-what-a-model-actually-reads-mt_PW7s8qEiJD
source: https://lightmysky.com/learn/computing/tokenisation-and-what-a-model-actually-reads-mt_PW7s8qEiJD.md
retrieved: 2026-09-12
---

> **Agent view.** This is the Markdown twin of the page, for tools and assistants.
> When to use this site, and the call that answers each job: https://lightmysky.com/agent-instructions.md
> API description (OpenAPI 3.1): https://lightmysky.com/openapi.json · Authentication: https://lightmysky.com/auth.md
> Pricing: https://lightmysky.com/pricing.md · Catalog: https://lightmysky.com/llms.txt · Full catalog: https://lightmysky.com/llms-full.txt
> Every machine-readable file on this domain: https://lightmysky.com/.well-known/ai-catalog.json
> Ask for Markdown with `Accept: text/markdown`, a `.md` address, or `?mode=agent`.

# Tokenisation and What a Model Actually Reads

A model never sees letters or words. It sees ids drawn from a vocabulary built by merging frequent pieces of text, and that vocabulary decides what is cheap to express, which spellings are fragile, and why digits and rare names behave badly.

Subject: Computing · Area: Machine Learning · Ages 22 to 23
Page: https://lightmysky.com/learn/computing/tokenisation-and-what-a-model-actually-reads-mt_PW7s8qEiJD

## Ready when they can

- Segment a sentence into subword pieces and count the tokens it costs
- Show two spellings of the same thing that tokenise differently and predict the effect on the model
- Say why a request is priced and limited in tokens rather than in words

## Lesson: What the model actually reads

A model never sees letters or words. It sees ids drawn from a vocabulary built by merging frequent pieces of text. Frequent strings cost one token, while rare names split into many pieces.

**Example.** Take a toy vocabulary where common words stay whole and rare ones split. The common word dog costs 1 token, while an unfamiliar name splits into 4 pieces and costs 4 tokens. Two spellings of the same thing can split differently, and the model then treats them as different inputs.

Requests are priced and limited in tokens rather than words because tokens are the true input units. Cost follows piece count, so digits and rare spellings run up the bill.

**Tip.** To estimate cost, count pieces not words. A short word in characters can still be many tokens when the vocabulary never merged it.

**Recap.** Models read token ids, frequent text is cheap, and rare spellings cost pieces.

## Practice

8 questions on this page, each with its working shown.

## Needs first

- [The Transformer Block: Heads, Residuals and Normalisation](https://lightmysky.com/learn/computing/the-transformer-block-heads-residuals-and-normalisation-mt_eKC2IehaDT)

## Opens up

- [Pretraining by Predicting the Next Token](https://lightmysky.com/learn/computing/pretraining-by-predicting-the-next-token-mt_D8b2phZJfp)