Partition Keys, Skew & Resharding

Why partition

Replication copies the same data; partitioning splits different data across nodes so each shard fits on disk and CPU. Most large systems use both: partitions, each replicated.

Partitioning strategies

Strategy	How	Risk
Key-range	Keys A–M on shard 1, N–Z on shard 2	Hot spots if keys cluster (e.g. recent timestamps)
Hash of key	shard = hash(key) mod N	Even spread, but range queries expensive
Consistent hashing	Ring with virtual nodes	Minimal remapping when nodes added/removed

See Networking roadmap for consistent hashing in load balancing — same idea applied to data placement.

Hot partitions

Skewed access patterns create hot shards — one node serves disproportionate traffic. Mitigations:

• Salted keys — user_id#random_bucket spreads writes, complicates reads
• Split hot key — sub-shards for celebrity accounts
• Separate read path — cache in front of hot partition
• Async aggregation — don't make one row the write bottleneck

Secondary indexes across shards

Global secondary index either:

• Scatter-gather — query all shards, merge (slow, expensive)
• Co-located index — index partitioned by same key (limited lookup patterns)
• Dedicated index service — separate system (Elasticsearch, etc.)

Know the trade-off; system design covers product choices.

Resharding

When you add capacity, move partitions between nodes. Goals: minimal downtime, limited double-writes, routing table updates. DDIA §6.4 covers fixed vs dynamic partitioning.

Interview one-liner

"Pick partition key from the query pattern you can't afford to scatter-gather — usually tenant_id or user_id — and watch for hot keys before you need a fire drill."