Distributed Systems Reference/Consensus & Coordination

Consensus & Raft

The consensus problem, why it's hard, Raft leader election and log replication — enough to explain etcd, ZooKeeper, and Kafka controller behavior.

4/5Overview: 30m

The consensus problem

Multiple nodes must agree on a single value (or log order) despite crashes and partitions. Used for:

  • Leader election — who is primary right now?
  • Metadata stores — etcd, ZooKeeper, Consul
  • Distributed logs — Kafka controller, replicated state machines

FLP impossibility — no deterministic consensus in asynchronous system with one faulty process. Real systems use timeouts (partial synchrony) and majority quorums.

Paxos vs Raft

Paxos — proven, hard to teach. Know it exists; senior interviews increasingly expect Raft instead.

Raft — decomposed into leader election, log replication, safety. Majority of nodes must agree; typical cluster size 3 or 5.

Raft at interview depth

States

Each node: follower, candidate, or leader.

Leader election

Followers timeout → become candidate → request votes. Majority wins → new leader. Term numbers monotonically increase; stale leaders ignored.

Log replication

Client requests go to leader. Leader appends to local log, replicates to followers, commits when majority ack. Followers apply committed entries in order.

Safety intuition

If two leaders in same term — impossible with majority. If old leader reappears — higher term exists, old leader steps down.

Where you see Raft

  • etcd — Kubernetes control plane
  • Consul — service mesh coordination
  • CockroachDB / TiKV — distributed SQL storage layer

Not PhD depth

You don't prove Raft safety. You walk through election + replication and connect to failover in Topic 3.

For joint consensus, snapshots, and pre-vote — see Topic 9 Raft Advanced (infra-oriented optional module).

Further Reading

Hands-On Tasks (Optional)

Low-setup exercises — browser visualizers, paper drills, or optional Docker. No autograding; the goal is interview fluency.

  • Walk through the Raft visualizer

    Complete the interactive Raft tutorial at thesecretlivesofdata.com/raft — trigger leader election and a partition. Note which nodes can accept writes in each state.

    25m