How to use this reference
- Work through topics top to bottom — evals and rollouts assume you understand serving and degradation patterns from earlier topics.
- This track treats the LLM as a probabilistic dependency in a distributed system — idempotency, timeouts, and fallbacks are not optional.
- We deliberately do not repeat AI Engineering (prompting, Cursor, MCP basics), generic SLO/incident playbooks (Observability), or circuit-breaker theory (Distributed Systems) — only how they apply to AI features at scale.
- Optional tasks are architecture and rollout design drills — paper or whiteboard level, not model training labs.
- Topic 9 — Deep Cuts (Inference Infra) is optional. Skip it for product and backend system design; do it for ML platform, inference, or AI infrastructure roles (vLLM, GPU fleet sizing, disaggregated serving). Adds ≈10h on top of the core path.
The Reference
- 1
The production lens — why LLM features are distributed systems with probabilistic cores.
- 2
Sync, async, streaming, and multi-step pipelines — how requests flow through production AI stacks.
- 3
Hosted APIs vs self-hosted inference — throughput, GPU economics, and tail latency at scale.
- 4
RAG ingestion and vector ops at production scale — freshness, cost, and index lifecycle.
- 5
Guardrails, content policy, and graceful degradation when models and dependencies fail.
- 6
Offline evals, online experiments, and AI-specific observability — measuring what users actually get.
- 7
Token economics, budgets, and semantic caching — running AI features without burning the margin.
- 8
Model rollouts, feature flags, and enterprise privacy — operating AI like any other critical service.
- 9
Optional hard-core material for ML platform, inference, and AI infrastructure loops — skip unless your target role owns GPUs or serving stacks.