---
title: "Log-Structured Storage and Compaction"
description: "Writing sequentially to a log and merging sorted runs in the background turns random writes into sequential ones. Reads then have to consult several runs, so the design trades read amplification and b"
canonical: https://lightmysky.com/learn/computing/log-structured-storage-and-compaction-mt_MDZ7MPNEhO
source: https://lightmysky.com/learn/computing/log-structured-storage-and-compaction-mt_MDZ7MPNEhO.md
retrieved: 2026-09-12
---

> **Agent view.** This is the Markdown twin of the page, for tools and assistants.
> When to use this site, and the call that answers each job: https://lightmysky.com/agent-instructions.md
> API description (OpenAPI 3.1): https://lightmysky.com/openapi.json · Authentication: https://lightmysky.com/auth.md
> Pricing: https://lightmysky.com/pricing.md · Catalog: https://lightmysky.com/llms.txt · Full catalog: https://lightmysky.com/llms-full.txt
> Every machine-readable file on this domain: https://lightmysky.com/.well-known/ai-catalog.json
> Ask for Markdown with `Accept: text/markdown`, a `.md` address, or `?mode=agent`.

# Log-Structured Storage and Compaction

Writing sequentially to a log and merging sorted runs in the background turns random writes into sequential ones. Reads then have to consult several runs, so the design trades read amplification and background work against write throughput.

Subject: Computing · Area: Data & Databases · Ages 22 to 24
Page: https://lightmysky.com/learn/computing/log-structured-storage-and-compaction-mt_MDZ7MPNEhO

## Ready when they can

- Trace a key through a memory table, a flushed run and a compaction
- Say which of write, read and space amplification each compaction policy favours
- Explain why a read may have to touch several runs, and what a membership filter saves

## Lesson: Write fast now, tidy up later

Every write lands first in a memory table, which later flushes to disk as one sorted run. Each key may then own several copies across runs, and the newest copy wins. A background job called compaction merges runs into bigger sorted runs and drops the shadowed copies.

**Example.** Key k is written twice. The older value rests in a flushed run while the newer value sits in the memory table. A read checks newest first and stops at the fresh copy. After the next compaction, only one copy of k remains.

Two policies split the trade. Tiered merging keeps several runs per level and merges rarely: writes stay cheap, but reads check more runs and old copies linger, so read and space amplification grow. Leveled merging keeps about one run per level and merges often: reads stay short and space stays tight, but data is rewritten more, so write amplification grows.

**Tip.** A read may have to touch several runs, so each run carries a small membership filter. A filter that says no lets the read skip that run entirely. The price is a little memory plus rare false positives that cost one wasted look.

**Recap.** Sequential writes plus background merges trade read and space costs against write speed.

## Practice

8 questions on this page, each with its working shown.

## Needs first

- [Partitioning and Replication for Scale](https://lightmysky.com/learn/computing/partitioning-and-replication-for-scale-mt_Xvzz5Hnuvu)

## Opens up

- [Stream Processing: Windows, Watermarks and Late Events](https://lightmysky.com/learn/computing/stream-processing-windows-watermarks-and-late-events-mt_CfzXL_cL0D)