Incident phases (NIST lite)
- Prepare — runbooks, contacts, break-glass access
- Detect — alerts, user report, scanner
- Contain — revoke keys, isolate instance, block IP
- Eradicate — patch vuln, remove backdoor
- Recover — restore service, monitor
- Learn — blameless postmortem
App engineer often owns contain + eradicate for their service.
Credential leak playbook
- Revoke credential first (assume active exploitation)
- Audit access logs for abuse window
- Rotate dependent secrets
- Root cause — how it got in git/slack
- Communicate per company policy
Speed beats completeness in first 15 minutes.
vs availability incident
Security incident may require taking service down (isolate malware).
Different comms — legal/privacy team involved if data breach.
Logging for forensics
Retain security-relevant logs (auth failures, admin actions) with tamper-resistant storage.
Balance retention cost vs investigation need.
Behavioral interview
"Tell me about a security issue you handled" — STAR: detection, action, prevention added (scanner, review process).
What not to do
- Hide incident
- Rotate only one of many leaked keys
- Wipe logs before investigation
Cross-reference: Observability → Alerting & Incidents for incident command structure.
Further Reading
Hands-On Tasks (Optional)
Security design drills — threat modeling, auth flows, and incident playbooks. Assumes Networking (TLS) fundamentals.
- Respond to leaked API key in public GitHub15m
Timeline: detect (scanner alert), revoke key, audit usage logs, rotate dependent secrets, notify customers if needed. First 30 minutes checklist.