RED, USE & Prometheus Fundamentals

RED for services, USE for resources

RED (Weaveworks) — microservice health:

Signal	Metric	Example
Rate	Requests per second	http_requests_total counter
Errors	Failed requests / total	rate(http_requests_total{status=~"5.."})
Duration	Latency distribution	http_request_duration_seconds histogram

USE (Brendan Gregg) — resource health:

Signal	Meaning
Utilization	% time busy (CPU, disk bandwidth)
Saturation	Queue depth, threads waiting
Errors	Device/controller errors

Use RED on your API; USE on databases, nodes, and thread pools. Google SRE's four golden signals overlap: latency ≈ Duration, traffic ≈ Rate, errors ≈ Errors, saturation ≈ Saturation.

Prometheus metric types

Type	Behavior	Example
Counter	Monotonic, only increases	requests_total, errors_total
Gauge	Up and down	queue_depth, memory_bytes
Histogram	Observations in buckets + _sum, _count	request_duration_seconds
Summary	Client-side quantiles (avoid in most cases)	Legacy — prefer histograms

Counter rule: use rate() or increase() — never graph raw counter values.

PromQL essentials

Interviewers want: correct rate() window, label aggregation with sum() by (...), knowing histogram quantiles are estimated.

Pull vs push

Prometheus pulls from /metrics endpoints on a scrape interval. Benefits: service discovery, up/down detection, no agent buffering logic in app.

Push gateways exist for batch jobs (short-lived processes) — not for long-running services.

Label discipline

Good labels: method, status, service, deployment (bounded enums).

Bad labels: user_id, url (with path params), trace_id.

Cardinality explosion is the #1 Prometheus outage cause at scale.

What OS track covers

iostat, disk await, page faults — resource metrics at the kernel level. This topic instruments your service's request path.

Interview answer

"Every RPC handler exports RED. Node exporters give USE for capacity planning. Alerts fire on SLO burn (see SLOs topic), not raw CPU. Histograms for latency — never averages alone."

Link forward

Histograms, Percentiles & SLIs explains bucket design and why histogram_quantile lies at extreme tails.