Capabilities, Limitations & Hallucinations

Capabilities in mid-2026 (what to trust)

Frontier models (Claude Opus/Sonnet, GPT-4.1/o-series, Gemini 2.5) reliably handle:

Task	Reliability	Caveat
Code generation in familiar languages	High	May use deprecated APIs; always verify against docs
Refactoring within a single file	High	Cross-file refactors need explicit context
Explaining existing code	High	Confident wrong explanations happen
Writing tests from specs	Medium-High	Edge cases and mocks often missed
Multi-file architecture decisions	Medium	Needs full context; prone to oversimplification
Precise numerical reasoning	Medium	Use tools/calculators for money and dates
Real-time facts (versions, CVEs)	Low	Knowledge cutoff; use RAG or web search

Treat the model as a fast junior engineer with perfect recall of public docs and imperfect recall of your repo.

Limitations that won't prompt away

These are architectural, not tuning problems:

1. No persistent memory — each call starts fresh unless you inject history
2. No true grounding — it generates text that looks like facts
3. Non-determinism — same prompt can yield different outputs
4. Context truncation — long inputs lose middle sections silently
5. Tool hallucination — may call functions that don't exist or with wrong args
6. Sycophancy — agrees with your framing even when you're wrong

Engineering response: verify, don't trust. Every AI output touching production code goes through the same review as a human PR.

Hallucinations: mechanism and taxonomy

A hallucination is the model generating a high-confidence continuation that is factually wrong. It's not a bug — it's the model doing exactly what it was trained to do: produce plausible text.

Type	Example	Detection
Fabricated API	db.fastQuery() that doesn't exist	Compile/lint; grep codebase
Wrong signature	Correct method name, wrong params	Type checker, IDE
Invented citation	Fake paper title or URL	Click the link
Stale syntax	Python 2 patterns in 2026 code	CI with current runtime
Confident nonsense	Plausible but wrong architecture rationale	Domain expert review

Guardrails that actually work

Prompting "don't hallucinate" is worthless. These patterns work:

Verify loop: Generate → Lint/Compile → Feed errors back → Retry (max N) Grounding: RAG retrieval → Cite sources → Reject if no source found Tool use: Force factual lookups via MCP/API instead of memory Human gate: AI drafts, human approves before merge Eval suite: Regression tests on known failure cases

Anthropic and OpenAI both recommend structured verification over prompt pleading.

The "confident wrong" problem

The most dangerous failure mode isn't obvious nonsense — it's plausible wrong code that passes a quick glance. Examples:

• Using async incorrectly but syntactically valid
• Importing from a package that exists on PyPI but isn't in your requirements.txt
• Implementing the wrong edge case with clean, idiomatic code

Your defense: tests, types, and CI — the same gates you'd use for any contributor.

When to escalate vs abort

Signal	Action
Model contradicts itself in same thread	Start fresh with tighter spec
Same error after 2 retry loops	You're missing context; add files or docs
Output references APIs not in your stack	Switch to RAG or paste the actual API docs
Architectural suggestion feels too clean	It probably ignores your constraints — push back

Interview framing

"Hallucinations are a property of autoregressive generation, not a configuration bug. I mitigate with retrieval grounding, tool use for facts, compile-time verification, and eval suites for regression. I never ship AI-generated code without the same review bar as human code."

Senior signal: Distinguish knowledge cutoff (fix with RAG) from reasoning failure (fix with decomposition) from instruction drift (fix with rules/specs). Don't lump them as "AI was wrong."