--- title: "The Memory Hierarchy and Why Caching Works" description: "A typical memory hierarchy combines small, fast registers and caches with larger main memory and storage. Temporal locality reuses recent locations; spatial locality accesses nearby locations. Cache p" canonical: https://lightmysky.com/learn/computing/the-memory-hierarchy-and-why-caching-works-mt_0f5ArFPlxP source: https://lightmysky.com/learn/computing/the-memory-hierarchy-and-why-caching-works-mt_0f5ArFPlxP.md retrieved: 2026-11-12 --- > **Agent view.** This is the Markdown twin of the page, for tools and assistants. > When to use this site, and the call that answers each job: https://lightmysky.com/agent-instructions.md > API description (OpenAPI 3.1): https://lightmysky.com/openapi.json · Authentication: https://lightmysky.com/auth.md > Pricing: https://lightmysky.com/pricing.md · Catalog: https://lightmysky.com/llms.txt · Full catalog: https://lightmysky.com/llms-full.txt > Every machine-readable file on this domain: https://lightmysky.com/.well-known/ai-catalog.json > Ask for Markdown with `Accept: text/markdown`, a `.md` address, or `?mode=agent`. # The Memory Hierarchy and Why Caching Works A typical memory hierarchy combines small, fast registers and caches with larger main memory and storage. Temporal locality reuses recent locations; spatial locality accesses nearby locations. Cache performance depends on access order, data layout and cache capacity. Subject: Computing · Area: Computer Systems · Ages 18 to 19 Page: https://lightmysky.com/learn/computing/the-memory-hierarchy-and-why-caching-works-mt_0f5ArFPlxP ## Ready when they can - Order the levels of the hierarchy by size and by access time - Identify temporal and spatial locality in a piece of loop code - Predict which of two loops over the same array has fewer cache misses under a stated model and say why ## Lesson: Small fast memory, big slow memory *(drawing: Below the ISA, a typical hierarchy has increasing access time in this order. Capacity usually increases too.)* Temporal locality means accessing the same memory location again soon. A loop repeatedly reading a lookup entry shows temporal locality. The benefit depends on whether that entry remains available in a fast level. Reuse of a location is different from visiting several neighbouring locations. A cache line is a fixed block of neighbouring bytes copied into a cache on a miss. A miss means the requested data is absent. In row-major storage, each row's elements are consecutive and rows follow one another. Row-first traversal then visits neighbours; column-first jumps between rows. On large arrays, those jumps can reduce use of each line before eviction. These access patterns do not guarantee a particular speed ratio. To count misses, specify line size, alignment, initial contents and capacity. Aligned line starts are multiples of the line size in bytes. In a cache holding one line, the first access misses. Another address in that resident line hits. An address in a different line misses and replaces it. Compare complete address sequences, not just drawings. Real runtime also depends on other work and hardware. Spatial locality means accessing nearby memory locations close together in time. Consecutive one-byte elements occupy addresses differing by 1. Reading them in order uses neighbouring bytes. Array indices alone do not prove this layout: state that the elements are stored consecutively before treating their indices as neighbouring addresses. In the typical hierarchy used here, total capacity increases from registers to caches, main memory and disk. This is a size ordering, not an access-time measurement. Register storage is limited but close to the execution hardware; larger levels provide room for more data. Particular devices can differ from this typical ordering. Read the layout before choosing a traversal. Column-major storage puts each column's elements together, so column-first visits neighbours. A drawing's row or column length does not set memory order. For a two-row, three-column example, the columns below occupy consecutive addresses. Access order suggests locality, but does not alone guarantee which traversal will run faster. **Recap.** Check layout and address order. Recent reuse is temporal locality; nearby accesses are spatial locality. Predict misses using a stated cache model, not the drawing alone. *Lesson, questions and flashcards written for this dataset using stated memory-layout and cache models.* ## Practice 8 questions on this page, each with its working shown. ## Needs first - [Memory and Storage: RAM, ROM and the Backing Store](https://lightmysky.com/learn/computing/memory-and-storage-ram-rom-and-the-backing-store-mt_AiXGJioO_S) - [Instruction Set Architecture as a Contract](https://lightmysky.com/learn/computing/instruction-set-architecture-as-a-contract-mt_pbwHRePY5M) ## Opens up - [Indexes: B-Trees and the Cost of a Lookup](https://lightmysky.com/learn/computing/indexes-b-trees-and-the-cost-of-a-lookup-mt_1Z5EQTqooI) - [Pipelining and the Cost of a Hazard](https://lightmysky.com/learn/computing/pipelining-and-the-cost-of-a-hazard-mt_GI_fPY9d6s) - [Paging, Page Faults and Replacement Policies](https://lightmysky.com/learn/computing/paging-page-faults-and-replacement-policies-mt_nEEuGL89Ze) - [Processes, Context Switches and What They Cost](https://lightmysky.com/learn/computing/processes-context-switches-and-what-they-cost-mt_oiOwHKUoXG)