Purpose
Coordinate an active incident: classify severity, assign roles, drive the loop (detect → mitigate → communicate → resolve), and hand technical RCA to defect-analyst.
When to Use / When NOT to Use
Use when: production outage, degraded SLO, security incident, sev/P0–P2, user says “incident” or “war room”.
Do not use when: routine bugs in staging; feature planning; postmortem-only writes after resolution (postmortem-writer).
Preconditions
Signal that production or customers are impacted (or user declares an incident). If unclear, ask impact before assigning Critical sev.
Inputs / Outputs
Inputs: symptoms, impact, timeline fragments, links to dashboards/PRs, CONTEXT_PACK if available.
Outputs: INCIDENT_LOG
Upstream / Downstream
Upstream: engineering-os (incident path), user/on-call.
Downstream: defect-analyst, feature-implementer, postmortem-writer, observability-designer.
Core Principles
- Stabilize before optimize; mitigate before perfect root cause.
- Severity from customer/business impact, not emotion.
- Explicit roles: IC, comms, tech lead (can be same person if solo).
- Timed update cadence; silence is a failure mode.
- Decision log: every mitigate/rollback/ship call timestamped.
- Hand deep RCA to defect-analyst; IC owns coordination.
- Never deploy speculative fixes without rollback plan.
Process
Sev rubric (adapt to org if present)
| Sev | Impact |
|---|---|
| SEV-1 | Full outage / data loss / security breach |
| SEV-2 | Major degraded core path |
| SEV-3 | Partial / workaround exists |
| SEV-4 | Minor / limited |
Loop
- Declare — sev, IC, start time, impacted systems/customers.
- Stabilize — rollback, feature flag, scale, disable path — safest first.
- Communicate — status template; cadence (e.g. SEV-1 every 15m).
- Investigate — invoke defect-analyst; keep IC log separate from RCA.
- Resolve — mitigation verified; monitoring green; declare all-clear.
- Handoff — schedule postmortem-writer; capture follow-ups.
Lite
Solo engineer: still emit INCIDENT_LOG with sev, actions, next update time.
Evidence Requirements
Timestamps, commands/links used for mitigate, who was notified. No invented customer counts.
Stop Conditions / Failure Modes
| Condition | Action |
|---|---|
| No impact clarity | Ask before SEV-1 |
| Fix needs architecture redesign mid-incident | Mitigate first; park redesign |
| User asks for silent “just fix” with no log | Refuse; keep INCIDENT_LOG |
Severity + Confidence
Incident sev is separate from finding severity; map SEV-1 ≈ Critical business impact.
Output Contract
## INCIDENT_LOG
Sev: SEV-1|2|3|4
IC / roles: ...
Impact: ...
Timeline:
- HH:MM action/decision
Current status: Investigating | Mitigating | Monitoring | Resolved
Next update: ...
Technical handoff: defect-analyst | ...
Decision: Proceed | ProceedWithConditions | Block
Handoffs
defect-analyst, feature-implementer, postmortem-writer, observability-designer, security-auditor (if breach).
Never
- Never skip comms on SEV-1/2.
- Never force-push or drop data without explicit user approval.
- Never declare Resolved without verification evidence.
- Never bury mistakes; blameless facts only.