Choosing & Routing Models

Not every task needs the frontier model

Mid-2026 model tiers (representative — check current pricing):

Tier	Examples	Best for	Cost ratio
Frontier	Claude Opus, GPT-4.1, Gemini 2.5 Pro	Architecture, complex reasoning, agent orchestration	1× (baseline)
Balanced	Claude Sonnet, GPT-4.1-mini, Gemini 2.5 Flash	Code generation, review, most daily tasks	~0.2–0.3×
Fast/cheap	Claude Haiku, GPT-4.1-nano, Gemini Flash-Lite	Classification, triage, formatting, simple Q&A	~0.05–0.1×

Using Opus for "format this JSON" is like using a senior architect to fix typos. Route by task complexity.

Task-based routing

Task	Recommended tier	Why
Architecture design	Frontier	High reasoning, low frequency
Code implementation	Balanced	Good enough quality, high frequency
Code review (first pass)	Balanced	Speed matters; human does final pass
Triage/classification	Fast	Simple decision, high volume
Test generation	Balanced	Pattern-following, not reasoning
Agent orchestration	Frontier	Tool selection quality is critical
Doc summarization	Fast/cheap	Low stakes, high volume
Customer-facing answers	Frontier	Quality and safety critical

Routing implementation

Simple (most teams): manual selection in IDE settings or per-request model picker.

Programmatic (production agents):

Classifier-based: fast model classifies task complexity → routes to appropriate tier. Meta, but cost-effective at scale.

Cost optimization without quality loss

Strategy	Savings	Risk
Route easy tasks to cheap models	50–80% on those tasks	Quality drop on misclassified tasks
Cache common queries	100% on cache hits	Stale answers
Reduce context size	Linear with input tokens	Missing context → wrong answers
Batch API requests	~50% discount	Latency increase
Distillation	Run cheap model trained on frontier outputs	Upfront training cost

Measure, don't assume. Run your eval suite on the cheap model before switching.

Latency considerations

Factor	Impact
Model tier	Frontier: 2–10s TTFT; Fast: 0.5–2s
Context length	Longer input = slower prefill
Output length	Streaming helps perceived latency
Tool calls	Each call adds round-trip
Provider load	Time-of-day variance

For interactive IDE use, time-to-first-token matters more than total generation time. Streaming is non-optional for UX.

Model upgrade strategy

Models change monthly. Your workflow shouldn't break:

1. Pin model versions in production (claude-sonnet-4-20250514, not claude-sonnet-4-latest)
2. Re-run eval suite before upgrading
3. A/B test new model on 10% traffic before full switch
4. Monitor regression metrics for 48h after upgrade
5. Keep rollback path — previous model version available for 30 days

Cursor handles model selection in the IDE; for custom agents, you own this lifecycle.

Multi-provider strategy

Avoid single-vendor lock-in for production agents:

Approach	Trade-off
Primary + fallback	Route to provider B if A is down/slow
Best-of-breed	Claude for code, GPT for structured output, Gemini for long context
Abstraction layer	LiteLLM, Portkey — unified API, harder to optimize per-provider

For IDE work, use whatever Cursor supports. For production agents, abstract the provider interface.

Interview framing

"I route by task complexity: frontier for architecture and agent orchestration, balanced for code generation, fast for triage and classification. I pin model versions in production, re-run evals before upgrades, and measure cost per task type monthly."

Senior signal: Show a routing matrix with cost estimates. Explain why you wouldn't use the frontier model for CI code review. Mention eval-gated model upgrades.