Distributed Systems Reference/Foundations & System Design Assembly

Capacity Estimation & Design Assembly

Back-of-the-envelope QPS, storage, and bandwidth; the 45-minute design recipe; and a track wiring table for pulling depth from this roadmap under interview pressure.

3/5Overview: 35m

What this subtopic owns

The core Distributed Systems topics teach correctness primitives — CAP, replication, consensus, partitioning. System Design Use Cases indexes external case walkthroughs. This subtopic bridges them: back-of-the-envelope math and how to assemble building blocks from this roadmap in a 45-minute moderated design.

You are not memorizing diagrams. You are practicing structured thinking under ambiguity — the Staff bar.

Back-of-the-envelope cheat sheet

Memorize powers of two (rough):

PowerApprox value
2^10~1 thousand (1 KB)
2^20~1 million (1 MB)
2^30~1 billion (1 GB)
2^40~1 trillion (1 TB)

Latency anchors (order of magnitude, same datacenter):

OperationBallpark
L1 cache reference~1 ns
RAM~100 ns
SSD random read~100 μs
Same-DC RPC~0.5–2 ms
Cross-region RTT~50–150 ms

DAU → QPS

peak QPS ≈ DAU × actions_per_user_per_day / 86,400 × peak_factor

Use peak_factor 2–5× average unless given otherwise. Example: 10M DAU, 20 reads/user/day → ~2,300 avg read/s → ~5–10k peak QPS.

Storage

storage ≈ records × record_size × retention_multiplier

Account for replicas (×3), indexes (+30–100%), and growth headroom. Example: 500M users × 2 KB profile × 3 replicas ≈ 3 TB — fits a managed DB cluster; say when you'd shard.

Bandwidth

bandwidth ≈ QPS × payload_size

1 Gbps ≈ 125 MB/s. A 50 KB response at 10k QPS ≈ 500 MB/s — CDN and compression matter.

Interview habit: state assumptions aloud ("assuming 1 KB average post, 10M DAU, 5 reads/day…") before multiplying. Wrong math with clear assumptions beats silent guessing.

The 45-minute assembly recipe

  1. Clarify (5–8 min) — functional requirements, non-functional (scale, latency, consistency, availability), out of scope
  2. Estimate (3–5 min) — QPS, storage, bandwidth; one sanity check ("one server can't hold this")
  3. High-level (10 min) — clients, load balancer, API, core services, data stores, async pipeline; draw data flow for one read and one write
  4. Deep dives (15–20 min) — interviewer picks 2–3: DB schema, caching, sharding, fan-out, failure modes
  5. Wrap (3–5 min) — bottlenecks, monitoring, trade-offs you'd revisit at 10× scale

Staff signal: drive trade-offs, not box count. "We could fan-out on write for fresher celebrity feeds, but write amplification is O(followers) — I'd start fan-out on read with caching unless proven insufficient."

Track wiring — where to pull depth

When the interviewer pushes on a thread, jump to the in-repo home — don't invent from scratch:

Interview threadPull from
Partial failure, CAP, consistency choiceThis track → Topics 1–2
Replication, quorums, read-your-writesReplication
Sharding, hot keys, rebalancingPartitioning
Ordering, Kafka, delivery guaranteesMessaging & Streams
Distributed transactions, sagas, outboxDistributed Transactions
Application caching, stampedeDatabases → Application Caching
B-tree vs LSM, SQL vs NoSQL pickDatabases track
API shape, pagination, idempotencyCommunication
Gateway, BFF, feed fan-outMicroservices → API Gateway & BFF
TLS, CDN, L4/L7 LBNetworking
OAuth, service-to-service trustSecurity
SLOs, tracing, golden signalsObservability
Managed vs self-hosted, SQS vs KafkaMicroservices → Cloud Primitives

Worked estimation drills (math only)

URL shortener — 100M new URLs/month, 5-year retention, 500 bytes/row with indexes → ~60B rows lifetime → tens of TB — need sharding by hash prefix; cache hot redirects.

Chat — 50M DAU, 50 messages/user/day, 200 B/message → ~29k write msg/s average, ~100k+ peak — partition by conversation_id; WebSocket connection servers separate from message store.

Full timed case practice → System Design Use Cases (methodology section first, then cases).

Common Staff-level mistakes

  • Jumping to microservices before clarifying scale
  • One database for everything without access-pattern justification
  • Ignoring hot keys and celebrity problems on partitioned systems
  • Caching without invalidation story
  • No failure mode ("what if Redis dies?" → degrade to DB with rate limit)
  • Forgetting async for notifications, search index, analytics

How to practice with this roadmap

  1. Read methodology links in System Design Use Cases
  2. Pick one case; do estimates on paper using formulas above
  3. Map each box to a subtopic you've studied — note gaps
  4. Verbalize trade-offs linking to CAP/PACELC and PACELC-style latency vs consistency

Link forward

Partitioning — hot key mitigation you estimated. Messaging — async legs in your diagram. Microservices → BFF — read-path aggregation for feed-style cases.

Further Reading

Hands-On Tasks (Optional)

Low-setup exercises — browser visualizers, paper drills, or optional Docker. No autograding; the goal is interview fluency.

  • Estimate a chat workload

    50M DAU, 50 messages/user/day, 200 B average message. Compute average and peak message write QPS (state peak_factor), rough 5-year storage with 3× replication, and whether a single Postgres primary is plausible.

    20m