--- title: "Data & Databases" description: "21 topics in Computing, in the order they build on each other." canonical: https://lightmysky.com/learn/computing/areas/data-and-databases source: https://lightmysky.com/learn/computing/areas/data-and-databases.md retrieved: 2026-09-02 --- > **Agent view.** This is the Markdown twin of the page, for tools and assistants. > When to use this site, and the call that answers each job: https://lightmysky.com/agent-instructions.md > API description (OpenAPI 3.1): https://lightmysky.com/openapi.json · Authentication: https://lightmysky.com/auth.md > Pricing: https://lightmysky.com/pricing.md · Catalog: https://lightmysky.com/llms.txt · Full catalog: https://lightmysky.com/llms-full.txt > Every machine-readable file on this domain: https://lightmysky.com/.well-known/ai-catalog.json > Ask for Markdown with `Accept: text/markdown`, a `.md` address, or `?mode=agent`. # Data & Databases 21 topics in Computing, in the order they build on each other. Page: https://lightmysky.com/learn/computing/areas/data-and-databases - [The Relational Model and Relational Algebra](https://lightmysky.com/learn/computing/the-relational-model-and-relational-algebra-mt_unDBd_Ug8T): A table is a set of tuples over named attributes, and queries are built from a handful of operations on those sets: select, project, join, union, difference. Every query language sits on top of this algebra, which is why the same question can be asked several ways and mean one thing. - [Functional Dependencies and Boyce-Codd Normal Form](https://lightmysky.com/learn/computing/functional-dependencies-and-boyce-codd-normal-form-mt_PfWLq1aEQz): A functional dependency says one set of attributes fixes another. Computing the closure of a set of dependencies finds the candidate keys, and Boyce-Codd normal form is the condition that every dependency starts from one of those keys. - [SQL at Depth: Subqueries, Set Operations and Views](https://lightmysky.com/learn/computing/sql-at-depth-subqueries-set-operations-and-views-mt_SLKUlC-7or): A query can use another query as a table, as a value, or as a test of existence, and set operations combine results that share a shape. A view names a query so the rest of the system can treat it as a table. - [Window Functions and Running Calculations](https://lightmysky.com/learn/computing/window-functions-and-running-calculations-mt_aaVxFPjWQv): A window function computes over a set of neighbouring rows while keeping every row in the output, which grouping cannot do. Ranking, running totals and comparisons with the previous row all come from partitioning and ordering the window. - [Indexes: B-Trees and the Cost of a Lookup](https://lightmysky.com/learn/computing/indexes-b-trees-and-the-cost-of-a-lookup-mt_1Z5EQTqooI): An index is a second structure that makes some lookups cheap and every write more expensive. Most are B-trees, whose wide nodes suit a disk that reads a page at a time, and whose ordering also serves ranges and sorts. - [Reading a Query Plan](https://lightmysky.com/learn/computing/reading-a-query-plan-mt_spbbzsCoBB): The database chooses how to run a query: which index to use, which join method, in what order. The plan is that choice written down with estimated row counts, and reading it is how a slow query stops being a mystery. - [Transactions and the ACID Promise](https://lightmysky.com/learn/computing/transactions-and-the-acid-promise-mt_8Egjmafq-L): A transaction groups statements so they take effect together or not at all, keep the database's rules true, do not see each other half-done, and survive a crash once committed. Those four promises are what let application code stay simple. - [Isolation Levels and the Anomalies They Allow](https://lightmysky.com/learn/computing/isolation-levels-and-the-anomalies-they-allow-mt_utxPNgxveB): Full isolation is expensive, so databases offer weaker levels, each defined by which anomalies it permits: dirty reads, non-repeatable reads, phantoms. Choosing a level is choosing which wrong answers the application can tolerate. - [Concurrency Control: Locks and Snapshots](https://lightmysky.com/learn/computing/concurrency-control-locks-and-snapshots-mt_gOd17LRKel): Two-phase locking makes transactions wait for each other and can deadlock. Multi-version concurrency control gives each transaction a consistent snapshot instead, so readers never block writers, at the price of keeping old versions around. - [Crash Recovery and Write-Ahead Logging](https://lightmysky.com/learn/computing/crash-recovery-and-write-ahead-logging-mt_JFPafSWOWF): Durability is achieved by writing the intention to a log before touching the data pages. After a crash the system replays committed work and undoes the rest, which is how a commit can be trusted the moment it returns. - [Key-Value and Document Stores](https://lightmysky.com/learn/computing/key-value-and-document-stores-mt_PfBDSgfeQX): Some workloads want a single key lookup and nothing else, and pay for the freedom by giving up joins and schema checks. A document store keeps whole nested objects together, which suits reading one thing at a time and makes any cross-cutting question harder. - [Wide-Column and Graph Stores](https://lightmysky.com/learn/computing/wide-column-and-graph-stores-mt_4a4yDq83fy): A wide-column store arranges data by the queries it must serve, so the design starts from the access pattern rather than the entities. A graph store makes relationships first-class, which turns a chain of joins into a traversal. - [Partitioning and Replication for Scale](https://lightmysky.com/learn/computing/partitioning-and-replication-for-scale-mt_Xvzz5Hnuvu): Beyond one machine, data is split by key across nodes and copied for safety. The partition key decides which queries stay fast and which become scatter-and-gather, and the replication scheme decides what a read may return after a recent write. - [Data Pipelines: Batch, Streaming and Doing It Twice](https://lightmysky.com/learn/computing/data-pipelines-batch-streaming-and-doing-it-twice-mt_o0zBEPLbmF): Data moves from where it is produced to where it is analysed through stages that extract, transform and load it. Any stage can be retried after a failure, so each one has to be safe to run twice, and late or out-of-order records have to be handled on purpose. - [The Data Science Workflow](https://lightmysky.com/learn/computing/the-data-science-workflow-mt_99QNfiyQ55): Useful analysis runs in a loop: state the question, find and check the data, build something, look at what it says, then report it in a form someone can act on. Most of the work is in the first two stages, and skipping them is what produces confident nonsense. - [Cleaning Real Data: Missing, Duplicated and Wrong](https://lightmysky.com/learn/computing/cleaning-real-data-missing-duplicated-and-wrong-mt_qIsLXA-7EC): Real datasets have blanks, repeats, impossible values and three spellings of the same city. Every repair is a decision that changes the result, so a cleaning step has to be written down and defended rather than done quietly. - [Exploratory Analysis and an Honest Chart](https://lightmysky.com/learn/computing/exploratory-analysis-and-an-honest-chart-mt_Jpb78Z6Ytu): Before modelling anything, look: distributions, outliers, relationships between pairs. Then the chart that reports it has to make the truth easy to read, which rules out a truncated axis, a misleading area, and a colour scale nobody can order. - [Distributed Transactions and the Blocking Case of Two-Phase Commit](https://lightmysky.com/learn/computing/distributed-transactions-and-the-blocking-case-of-two-phase-commit-mt_MP_B_a_EwW): A transaction spanning several stores needs one atomic decision. Two-phase commit gets it by making participants promise before the coordinator decides, and its known weakness is that a coordinator failure at the wrong moment leaves participants holding locks with nobody to ask. - [Conflict-Free Replicated Data Types](https://lightmysky.com/learn/computing/conflict-free-replicated-data-types-mt_Bhf1nu-9jz): If every update is commutative and idempotent, replicas that receive the same set of updates in any order end in the same state, and no coordination is needed at write time. The cost is that the merge rule, not the application, decides what concurrent edits mean. - [Log-Structured Storage and Compaction](https://lightmysky.com/learn/computing/log-structured-storage-and-compaction-mt_MDZ7MPNEhO): Writing sequentially to a log and merging sorted runs in the background turns random writes into sequential ones. Reads then have to consult several runs, so the design trades read amplification and background work against write throughput. - [Stream Processing: Windows, Watermarks and Late Events](https://lightmysky.com/learn/computing/stream-processing-windows-watermarks-and-late-events-mt_CfzXL_cL0D): An unbounded stream has no end at which to compute an answer, so results are cut by windows over event time. A watermark is the system's claim that event time has passed a point, and every record arriving after it forces a choice between waiting longer and being wrong.