Threads vs. Async: Choosing the Right Model

Language-specific mechanics: Concurrency Language Manual — Async/Await, Futures & Non-Blocking I/O →

The question underneath the question

"Should I use threads or async here?" is really asking: where does time actually go in this workload, and does that time represent the CPU working, or the CPU waiting? Everything else in this subtopic is downstream of answering that one question honestly for your specific task.

I/O-bound work spends almost all of its wall-clock time waiting — for a network response, a disk read, a database query, another service to reply. The CPU is idle during that wait; it has nothing to compute.
CPU-bound work spends its wall-clock time actually computing — parsing, hashing, sorting, running a model, compressing data. The CPU is busy the whole time.

Async (event-loop-based) concurrency, from this topic's first subtopic, is fundamentally a way to make waiting cheap: instead of dedicating a whole OS thread (with its own stack, its own kernel scheduling overhead) to sit idle until data arrives, you register interest and let one thread's event loop juggle thousands of pending waits at once. That's a huge win precisely because waiting was the expensive, wasteful part.

But async buys you nothing for CPU-bound work. A single-threaded event loop still only has one thread physically executing instructions at any instant — running a CPU-heavy function inside an async function doesn't make the arithmetic go faster or run on another core; it just occupies the loop's one thread while every other pending callback (including ones whose I/O already completed) sits frozen. If the bottleneck is genuinely computation, the only way to actually go faster is real parallelism: multiple OS threads (for work that can release a language-level lock like Python's GIL during native calls, or in languages without a GIL like Java/Kotlin/C++/Go) or multiple processes (to sidestep a GIL entirely, at the cost of serialization overhead to move data between processes).

The decision framework

Workload shape	Right tool	Why
I/O-bound, many concurrent operations (thousands of sockets, many outstanding API calls)	Async / event loop	Waiting is nearly free on an event loop; OS threads would waste memory and context-switch overhead per idle connection
I/O-bound, but using a blocking library with no async equivalent	A thread (or a thread pool) dedicated to that call, kept off the main event loop	You can't make a blocking call non-blocking from the outside; isolate it so it doesn't freeze everything else
CPU-bound	Threads with true parallelism (JVM/Kotlin, C++, Go), or processes (Python's `multiprocessing`, to route around the GIL)	Only genuine parallel execution on multiple cores speeds up computation; async and (GIL-bound) threads don't help here at all
Mixed: an I/O-heavy service with occasional CPU-heavy steps	Event loop for the I/O, explicitly offloading the CPU-heavy steps to a thread or process pool	Keeps the loop responsive for everything else while the heavy step runs elsewhere

Two nuances worth stating precisely, because interviewers often probe exactly here:

"Threads help with I/O-bound work" is true, but for a different reason than async does. A thread blocked on I/O releases the CPU (and, in Python's case, releases the GIL during the actual blocking syscall) so other threads can run — it's just that each thread carries real OS overhead (a stack, kernel scheduling, a context switch to resume it) that an event loop's per-task bookkeeping doesn't. Both models achieve I/O concurrency; async does it far more cheaply at very high connection counts, which is the whole reason it exists as a separate paradigm rather than everyone just using thread pools forever.
Async doesn't just fail to help CPU-bound work — it actively hurts if you get it wrong. Dropping a CPU-heavy function into an async handler without offloading it doesn't merely fail to speed things up; it freezes the entire event loop (and every other in-flight request sharing it) for the duration, which is strictly worse than the equivalent blocking-thread version where at least other threads keep running.

Mixed workloads: combining both models

Most real systems aren't purely one or the other, and the mature answer to "threads or async?" is frequently "the event loop, with an escape hatch to a pool for the CPU-bound parts":

Node.js does exactly this internally: your JavaScript runs on one event-loop thread, but libuv's worker thread pool handles filesystem calls, DNS, and certain crypto functions that have no non-blocking OS primitive to hook into the loop's readiness notification.
Python's asyncio exposes this pattern directly to you via loop.run_in_executor(pool, blocking_fn, *args), handing a blocking or CPU-bound call to a ThreadPoolExecutor (for I/O-bound blocking libraries) or ProcessPoolExecutor (for genuinely CPU-bound work, to escape the GIL) while await-ing the result without freezing the loop.
A JVM/Kotlin server commonly separates dispatchers by workload: Dispatchers.IO (a large pool sized for blocking I/O calls) versus Dispatchers.Default (a pool sized to the number of CPU cores, for actual computation) — the coroutine machinery makes it a one-line withContext(...) switch rather than a structural rewrite.

The pattern generalizes: pick the cheap concurrency model (async/event loop) as your default for the I/O-bound bulk of the work, and reach for real parallelism (a bounded thread or process pool — see Topic 9 for sizing that pool correctly) only for the specific pieces that are actually CPU-bound, so the expensive resource (threads, processes, cores) is spent only where it does something an event loop fundamentally cannot.

Pitfalls and interview gotchas

"Async is just faster than threads" — stated with no qualification. Wrong as a blanket claim. Async is more efficient per unit of I/O-bound concurrency (lower memory, no context-switch overhead), but it provides zero speedup for CPU-bound work, where real threads/processes win outright.
Forgetting that "single-threaded" doesn't mean "no other threads exist in the process." Node.js and Python's asyncio both commonly delegate to background thread/process pools even though your application code runs on one loop thread — conflating "my code is async" with "this process uses exactly one OS thread" is a common and easily-corrected misconception.
Choosing threads by default "to be safe," even for a purely I/O-bound, high-fan-out workload. At high concurrency (thousands of simultaneous outstanding requests), OS thread overhead (stack memory, kernel scheduling) becomes the actual bottleneck — this is precisely the scenario async concurrency was invented to solve, and defaulting to threads here throws away the win.
Not having a concrete answer for "what happens if a CPU-bound task lands in my event loop by mistake." The correct answer is "it blocks the entire loop, not just that task" — if you can't state that clearly, revisit the first subtopic in this topic.
Assuming the GIL makes Python threads useless for everything. It makes them useless for parallelizing pure-Python CPU-bound code, but they're still perfectly effective for I/O-bound blocking calls (the GIL is released during the actual blocking syscall) and for CPU-bound work done inside native extensions that release the GIL (e.g., much of NumPy).

A one-line rule to reach for under pressure

I/O-bound and high concurrency count → async/event loop. CPU-bound → real parallelism (threads/processes). Mixed → an event loop that explicitly offloads its CPU-heavy slice to a pool. If you can state that sentence and justify each clause with the "where does the time go" question from the top of this page, you've covered what most interviewers are actually testing when they ask "threads or async?"

The shared shape across ecosystems: keep the event loop (or a request-handling thread) free, and hand genuinely CPU-heavy work to a pool sized for parallel computation.

// There is no event loop to protect. CPU-bound work already runs
// in parallel up to GOMAXPROCS. Bound it only so compute doesn't
// starve runnable I/O goroutines:
sem := make(chan struct{}, runtime.GOMAXPROCS(0))
resultCh := make(chan []byte, 1)
sem <- struct{}{}
go func() {
    defer func() { <-sem }()
    resultCh <- compress(payload)
}()
result := <-resultCh

The Node/Python move — 'offload CPU so the event loop stays free' — doesn't apply. Goroutines already multiplex blocking I/O via the netpoller, and CPU work already uses real cores. The remaining knob is bounding CPU-heavy goroutines so they don't occupy every P with busy work; that's a semaphore or a small worker pool, not worker_threads.

Further Resources (Optional)

Practice Problems

Interview relevancy:!!!Critical — must-have!!Important — highly recommended!Good to have~Niche — rarely asked

Promise PoolMediumPremium!!!3/525m