Observability Reference/SLOs & Error Budgets

SLI & SLO Definition

Availability vs latency SLOs, user-centric SLIs, measurement windows, multi-window targets, and why 99.9% is harder than it sounds.

3/5Overview: 30m

SLI, SLO, SLA — definitions

TermMeaning
SLI (Indicator)Measured metric — "proportion of requests < 300ms"
SLO (Objective)Target for SLI — "99.9% of requests < 300ms per 30 days"
SLA (Agreement)Contract with consequences — often looser than internal SLO

Internal SLO > external SLA — you want to breach the SLA rarely, so alert before the SLA line.

User-centric SLIs

Measure what users experience, not what ops prefers:

Good SLIBad SLI
Successful checkout completion rateCPU < 70%
Search results returned < 500msPod count = 10
Video playback without rebufferJVM heap usage

Google SRE: SLI = good events / valid events in a measurement window.

Availability vs latency SLOs

Availability: 1 - (bad_requests / total_requests)

  • Define "bad": HTTP 5xx, timeout, wrong response
  • Exclude: client errors (4xx) if user-caused

Latency: count(requests < threshold) / count(valid_requests)

  • Use percentiles: "99% of requests complete in < 300ms"
  • Measure server-side + optionally client-side (RUM)

The nines math (interview favorite)

SLOAllowed downtime / 30 days
99%7.2 hours
99.9%43.2 minutes
99.95%21.6 minutes
99.99%4.3 minutes

Each nine is 10× harder. 99.99% on a complex microservice mesh requires error budgets, not heroics.

Error budgets

Error budget = 1 - SLO = allowed bad events.

99.9% SLO → 0.1% budget = ~43 min of errors/month.

When budget remains: ship features, take risks. When budget exhausted: freeze releases, focus on reliability.

This is how SRE negotiates with product — objective, not political.

Measurement windows

  • Rolling 30-day — common for external reporting
  • Calendar month — billing alignment
  • Shorter windows (1h, 6h) — for burn-rate alerting

Multi-window: same SLO evaluated over 1h, 6h, 3d, 30d — fast burns page, slow burns ticket.

Interview answer

"Checkout SLO: 99.95% of requests succeed with latency < 500ms, measured at the load balancer. 4xx from user validation excluded. Error budget drives our release cadence — if we're burning budget, feature freeze until recovery."

Link forward

Burn-Rate Alerting implements the multi-window math that pages on-call before users exhaust the monthly budget.

Further Reading

Hands-On Tasks (Optional)

Low-setup exercises — signal-selection drills, local Prometheus/Grafana, or OTel sandbox. No autograding; the goal is production triage fluency.

  • Define an SLO for a search API

    Write one availability SLI and one latency SLI (with percentile and window). Compute allowed downtime per month at 99.9% and 99.95%. Note what you'd exclude from 'good' events.

    20m