Failure Models and Replication
A distributed system has to say what kinds of failure it tolerates: a machine that stops, a message that never arrives, a machine that stops and comes back with old state. Replication is the usual answer, and it turns one copy's truth into a question about which copy to believe.
What a learner can do afterwards
- Distinguish a crashed node from a slow one from the outside, and say why it is hard
- Explain what a primary and its replicas each promise a client
- Say what a replica that rejoins with stale state has to do before serving reads
1 · Read
You already know messages carry order between machines. Now ask what can go wrong. A machine can stop dead, a message can vanish, or a stopped machine can return later still believing old state. First you must name which failure you tolerate.
You call a node and hear nothing. It may have crashed, or it may just be slow. From outside both look identical: silence. That is why systems use timeouts and still cannot tell the two apart for sure.
Replication is the usual answer: keep copies on several machines. The primary takes writes and promises one order, while replicas copy it and promise to serve only what they have caught up with. One copy's truth becomes a question about which copy to believe.
A replica that rejoins with stale state must catch up before serving reads. Let it copy the missed changes first, and only then answer clients, or it will hand out yesterday's truth.
Name the failures you tolerate, remember silence hides crash and slowness alike, and never let a stale replica answer.
2 · Watch
Take it off screen
Where it sits
8 questions wait behind this lesson, each with its answer explained. Every answer feeds the sky: stars light as they are learned, and dim when it is time to come back.