LoRA & Multi-Adapter Serving

LoRA in one paragraph

LoRA (Low-Rank Adaptation) freezes base weights and trains small rank-decomposition matrices injected into attention layers. Fine-tuning updates megabytes instead of gigabytes — fast to train, fast to swap.

W' = W + BA (B, A are low-rank; W frozen)

Production use cases

Scenario	LoRA vs RAG vs full fine-tune
Per-customer tone/style	LoRA adapter per tenant
New product vocabulary	RAG first; LoRA if retrieval isn't enough
Domain reasoning shift	Full fine-tune or larger adapter — higher cost
Frequent knowledge updates	RAG (LoRA is stale until retrained)

Assumes AI Engineering covered when to RAG. Here: how to serve many adapters.

Multi-adapter serving

vLLM multi-LoRA loads one base model and switches (or batches) adapter weights per request.

Design questions:

Question	Options
Adapter storage	Object store (S3/GCS), synced to GPU nodes on deploy
Routing	Tenant ID → adapter ID in request metadata
Memory	Preload top-N adapters; LRU evict cold ones
Isolation	Adapter weights are tenant-scoped; no cross-tenant leakage in weights (prompt injection is still a policy problem — Topic 5)

Cold start: loading a new adapter adds latency — warm pools or pre-warm on tenant signup.

Adapter lifecycle

1. Train on curated dataset (offline pipeline — not this track's depth)
2. Version and store artifact (customer-123/v3)
3. Register in serving config; canary 5% traffic
4. Rollback = route to previous adapter version (like model rollouts in Topic 8)

Link to Topic 8 (Shipping & Operations)

Adapter rollouts mirror model version rollouts — feature flags, canary, eval regression gates.

What we deliberately skip

• Training loop hyperparameters (learning rate, rank selection sweeps)
• RLHF / DPO alignment training
• Pretraining from scratch

Those are ML research / training engineer territory.

Interview answer template

"100 enterprise tenants on one 70B base — LoRA adapters in object storage, vLLM multi-LoRA with LRU preload of top 20 by traffic. Request carries adapter_id; cold loads hit a warm queue with 2s timeout fallback to base model plus RAG."