Guardrails & Content Policy

Defense in depth for LLM products

Vendor safety (OpenAI moderation, Anthropic constitutional AI) is necessary, not sufficient. Your product context — internal data, brand voice, regulated domains — needs application-layer guardrails.

OWASP LLM Top 10: prompt injection, insecure output handling, training data poisoning — design assuming attackers probe your surface.

Policy stack layers

User input → [Client filter] → [Gateway] → [Pre-model policy] → Model → [Post-model policy] → User

Layer	Examples	Latency budget
Input	PII redaction, prompt injection heuristics, length caps	< 10ms (rules)
Pre-model	Moderation API, blocklist, jailbreak classifier	50–200ms
Model	Vendor safety tuning	(in inference)
Post-model	Output moderation, citation check, format validation	50–200ms
Audit	Log policy decisions (not always full prompt)	Async

Fail closed for high-risk domains (medical, financial advice); fail open with flag only when UX demands it and legal approves.

Prompt injection vs jailbreaking

Attack	Vector	Mitigation
Direct injection	User text overrides system prompt	Delimiter discipline, instruction hierarchy, input sandboxing
Indirect injection	Malicious content in retrieved docs	Sanitize retrieval, separate system/user channels
Tool abuse	Model tricked into calling dangerous tools	Allowlisted tools, human-in-loop for writes

No silver bullet — combine rules, classifiers, and least-privilege tools.

PII and data leakage

• Scan inputs for SSN, credit cards, secrets before sending to vendor
• Redact logs — prompts often contain user PII
• Block model from echoing training-like secrets (API keys in user paste)

Enterprise contracts require zero retention — verify vendor tier, don't assume.

Content policy dimensions

Beyond "toxicity":

• Competitor mentions
• Legal claims without disclaimer
• Copyrighted text reproduction
• Off-brand tone

Maintain a policy config versioned like code — rollbacks when new rules over-block.

Human review queues

For borderline scores, route to human before display. Queue depth is an SLO — backlog = delayed UX.

Interview framing

"Secure a customer support bot with tool access":

1. Read-only tools by default; writes need confirmation
2. Retrieved tickets sanitized; metadata ACL enforced
3. Post-model check for PII in outbound email tool args
4. Rate limit and anomaly detection on policy blocks

Senior signal: Distinguish security (injection → data exfil) from safety (harmful content) — different teams, different metrics.

Link forward

Failure Modes & Degradation covers runtime behavior when policy or model layers fail — not just what to block.