Multicore, Cache Affinity & CPU Pinning

One socket, many cores

Modern servers have dozens of cores, often grouped into NUMA nodes with local vs remote memory. The scheduler must balance load (spread work) against affinity (keep threads near warm caches). Every migration trades fairness for cache misses.

Cache hierarchy matters

L1 (per core, ~32KB) → L2 (per core) → L3 (shared) → RAM (NUMA-local vs remote)

A thread that ran on core 4 has hot L1/L2 lines there. Move it to core 37 on another socket and every memory access may pay remote NUMA latency (~2× local).

Affinity and pinning

Technique	What it does	When to use
sched_setaffinity	Bind thread to CPU set	Latency-sensitive, known cache footprint
NUMA-aware alloc (numactl --membind)	Allocate on local node	Large in-memory databases
Hyperthreading awareness	2 logical CPUs share one core	CPU-bound: treat as ~1.3× not 2× cores

CPU pinning helps when workload is cache-stable and migration cost exceeds load-imbalance cost. It hurts when pinned threads idle while others queue — you've disabled the scheduler's load balancer.

Hyperthreading (SMT)

Two logical CPUs share one physical core's execution units. Two CPU-bound threads on sibling hyperthreads fight for the same pipeline — throughput may be less than one thread alone. Two memory-bound threads may benefit from hiding latency.

Senior-level signal

Tail latency spikes on NUMA hosts often trace to cross-node memory or noisy neighbors on shared cores — not application logic. Check numastat, perf stat -e node-loads,node-load-misses, and whether Kubernetes placed your pod without topology spread constraints.

Where this goes next

Virtual Address Spaces begins memory virtualization — how each of those scheduled threads sees its own private memory regardless of which core it lands on.