Production Agents & Evals

You can't ship agents without evals

Every other software domain has tests. Agent systems that "feel right" in demo and fail in production need the same discipline. Evals are unit tests for non-deterministic systems — and they're non-optional for production agents.

Eval types for agents

Eval type	What it measures	Grading
Task completion	Did the agent achieve the goal?	Automated (check output state) or human
Tool selection	Did it call the right tools in the right order?	Automated (compare to expected sequence)
Output quality	Is the response correct, complete, safe?	Human or LLM-as-judge
Regression	Did a model/prompt change break known-good cases?	Automated (CI gate)
Adversarial	Does it handle edge cases, injection, ambiguity?	Human-designed test cases
Cost/latency	Does it stay within budget?	Automated (metrics)

Building an eval suite

Start with real failures, not hypothetical cases:

1. Collect 20-50 real tasks your agent should handle 2. Include 5-10 known failure cases from production/testing 3. Define pass/fail criteria BEFORE running (not after seeing output) 4. Mix difficulty: 40% easy, 40% medium, 20% adversarial 5. Automate what you can; human-grade the rest

Example eval case:

LLM-as-judge (careful)

Using an LLM to grade another LLM's output scales evals but introduces its own biases:

Use LLM-as-judge for	Don't use for
Subjective quality (clarity, completeness)	Factual correctness (judge may agree with wrong answer)
Ranking multiple outputs	Safety-critical pass/fail
Approximate regression detection	Compliance/legal decisions

Pattern: LLM judge for triage → human review on flagged cases. Never LLM-judge as sole gate for production.

Production checklist

Before any agent workflow goes live:

#	Gate	Pass criteria
1	Eval suite ≥ 85% pass rate	On held-out test set, not training examples
2	Adversarial cases handled	Prompt injection, ambiguous input, missing context
3	Cost budget defined	Max tokens/call, max calls/task, daily budget
4	Latency SLA	p95 response time acceptable for use case
5	Failure mode documented	What happens when agent fails/loops/times out
6	Human escalation path	User can always reach a human
7	Audit logging	Every tool call, LLM input/output logged
8	Rollback plan	Can disable agent without breaking workflow
9	Model upgrade path	Re-run eval suite before switching models
10	Access control	Agent inherits minimum necessary permissions

Failure recovery patterns

Agents fail differently than traditional services:

Failure	Detection	Recovery
Tool loop (repeats same call)	Same tool+args N times	Max iteration cap; break loop
Context overflow	Truncated input errors	Summarize and retry with smaller context
Wrong tool selection	Unexpected tool in trace	Fallback tool; ask user for clarification
Partial completion	Timeout mid-task	Checkpoint state; resume or rollback
Hallucinated tool args	Server-side validation fails	Return error to model; retry with schema hint

Design for graceful degradation: if the agent fails, the user can still complete the task manually.

Cost and latency budgets

Agents are expensive. Define budgets upfront:

Per task: max 10 tool calls, max 50k tokens, max 60s Per user/day: max 100 tasks, max 500k tokens Per month: $X budget → alert at 80%

Track and alert. An agent loop bug can burn through a monthly budget in hours.

Continuous eval in CI

Wire evals into your pipeline:

Model/prompt change → Run eval suite in CI → Pass rate ≥ threshold → Allow deploy → Pass rate < threshold → Block + alert

Same pattern as unit test coverage gates. OpenAI and Anthropic both provide eval APIs/frameworks.

Interview framing

"I don't ship agent features without an eval suite of 20+ real-world cases, including adversarial inputs. I automate tool-selection and regression checks, use human grading for quality. CI blocks deploys below 85% pass rate. Every agent has cost caps, timeout, and human escalation."

Senior signal: Describe a specific eval case that caught a regression. Explain your grading strategy (automated vs human). Mention cost budget numbers for a real or hypothetical agent.