Math and simulation without a cluster
Not every distributed concept needs Kafka in Docker. Two high-value local drills:
Quorum arithmetic — Given N replicas, write quorum W, read quorum R, does a read necessarily see the latest write? Implement the predicate; test the classic counterexamples (R+W≤N).
At-least-once consumer — Feed a list of messages, some duplicated ids. Count how many would have been applied vs skipped if you maintain a dedup set. This is exactly-once effect without transactional Kafka.
What this does not replace
- Raft implementation labs (6.824) — optional infra-heavy depth, not required here.
- Docker Kafka exercise — still in Topic 8 if you want one broker locally.
These tasks close the gap between reading about delivery semantics and reasoning under retry.
Practice order
- Quorum staleness function + unit tests (~15 min).
- At-least-once consumer simulation (~20 min).
- Verbal follow-up: what changes for exactly-once end-to-end? (idempotent sink + transactions + offset commit order — point to Topic 7/8 theory.)
Further Reading
Coding Exercises (Optional)
In-memory specs — implement locally with unit tests. No cluster, Docker, or cloud setup unless a task says otherwise.
- Code: Quorum read staleness check15m
Implement `can_read_stale(N, R, W) -> bool` returning true when a read quorum might miss the latest write (classic R+W≤N case). Add `min_nodes_for_strong_read(N, W) -> R` returning the smallest R that guarantees overlap with any successful write quorum when W=(N//2)+1 style majority writes. Unit-test: N=5,W=3,R=3 → false; N=5,W=2,R=2 → true; edge N=1. ~15 minutes.
- Code: At-least-once consumer with dedup20m
Simulate `process(messages: {id, payload}[])` where the broker may redeliver. Maintain a processed-id set (assume bounded memory — evict oldest after 10k ids). Return `{processed_count, duplicates_skipped}`. Messages with duplicate ids must not double-apply side effects (increment a `applied` counter only once per id). ~20 minutes.