Incident Postmortem

Incident Postmortem Skill

This skill produces a complete, blameless incident postmortem document following industry-standard format. Output enforces blameless framing throughout — system gaps over individual failures — and drives toward specific, closeable action items rather than vague process commitments.

Proposes Actions

The action items don't have to stay on the page: hand them to action-runner, which previews them (dry-run, risk-rated), runs only what you approve via the connected action MCP, and records what was done back to the brain. Typical: file a follow-up issue per action item (🟡), assigned to its owner with a due date. This skill proposes; action-runner gates and runs — never silently.

Where this sits — turning an incident into fixes

Third in the incident-response spine: /slo-error-budget (frame) → /debugging-log-analyser → incident-postmortem → /oncall-runbook. It receives the root-cause diagnosis from /debugging-log-analyser (read it rather than re-diagnosing) and hands /oncall-runbook the contributing factors and prioritised action items — and the error budget from /slo-error-budget decides how urgent those actions are. Blameless, root cause vs contributing factors, and action item are defined once in docs/craft/incident-response.md; blameless is the load-bearing rule.

The loop

A postmortem fails the moment it assigns blame — the honest data dries up and every future incident is under-reported. Phase 1 sets that frame; everything depends on it.

1. Establish blameless framing first. State up front that this examines the system that let a competent person make the move, never the person. This isn't politeness — it's the precondition for the truthful timeline the rest of the skill needs. Done when: the framing is explicit and no sentence in the document blames an individual; failures are attributed to system gaps.
2. Build the timeline from evidence. Reconstruct start → detection → mitigation → resolution with real timestamps (from the diagnosis and logs, not memory). Detection, mitigation, and resolution are distinct events — track each. Done when: the timeline has real timestamps and separates detection/mitigation/ resolution, and the impact is quantified (users, duration, scope).
3. Find the root cause AND the contributing factors. The root cause is one thing; the contributing factors are what let it reach users and persist (the missing alert, the skipped canary, the unclear runbook). A postmortem with a root cause and no contributing factors hasn't looked hard enough. Done when: at least the load-bearing contributing factors are named, each pointing at a system gap that's fixable.
4. Drive to owned, dated action items — governed by the budget. Convert factors into specific action items, each with an owner and a date; vague "improve monitoring" items decay. Prioritise them against the error budget (spent → now; healthy → soon). Done when: every action item has an owner and a date, and /oncall-runbook could turn the detection/mitigation learnings into an entry without re-analysing the incident.

Required Inputs

Ask the user for these if not provided:

• Incident title / ID
• Severity (P1 / P2 / P3 or SEV1 / SEV2 / SEV3)
• Date and duration of the incident
• What happened (rough notes are fine — the skill will structure them)
• Services or systems affected
• Customer impact (how many users, what was degraded)
• How it was detected
• How it was resolved
• Initial thoughts on root cause
• Action items already identified (optional)
• Responders (who was on-call or responded — names or roles; used for the timeline, not for blame)
• Customer or external communications sent (optional — any status page updates, emails, or support messages with timestamps)

Reads from / Writes to the Brain

If a professional-brain (brain/) exists, use it before asking:

• Read first: the affected system's entities/ file and any related prior decisions/ or past incidents (recurring root causes are the most important thing to surface).
• Write after: log the action items and decisions to decisions/, and the root-cause learning to knowledge/ — tag a measured cause [data] and a suspected one [hunch], never the reverse.

Deeper Materials

• references/root-cause-digging.md — five-whys done properly (stop at a changeable system property, branch into cause/detection/response chains), a contributing-factor taxonomy to sweep, and blame-shaped → systemic language rewrites. Use it while writing the Root Cause section and to reframe any blameful input notes.
• templates/review-meeting-agenda.md — a 45-minute, document-first agenda for the postmortem review meeting, with ground rules and an action-item quality gate. Offer it alongside the finished postmortem.

Output Format

Incident Postmortem: [Incident Title]

Incident ID: [ID] Severity: [P1/P2/P3] Date: [Date] Duration: [Start time → Resolution time — total duration] Status: [Resolved / Monitoring / Ongoing] Author: [Leave blank for user to fill] Last updated: [Date]

Executive Summary

[3–5 sentences. Describe what happened, who was affected, and what was done to resolve it. Written for a non-technical stakeholder. No jargon. No blame.]

Impact

Timeline

List events in chronological order. Each entry: [HH:MM UTC] — [What happened. Who did what. What changed.]

Rules for timeline entries:

• Use passive or system-focused language — avoid "X made a mistake"
• Include: first symptom, detection, escalation, hypothesis tested, fix applied, confirmation of resolution
• Note time between key events (e.g. "22 minutes between detection and escalation")

Timeline, drawn — also render the incident timeline as a Mermaid Gantt so the gaps (e.g. detection → escalation) are visible at a glance (it renders live in the playground and exports as PNG). Use the incident phases as bars; keep it blameless and system-focused:

gantt
    title Incident timeline (UTC)
    dateFormat HH:mm
    axisFormat %H:%M
    section Phases
        Undetected impact   :22:00, 18m
        Detection           :milestone, 22:18, 0m
        Investigation       :22:18, 22m
        Mitigation          :22:40, 15m
        Resolved            :milestone, 22:55, 0m

Root Cause

Primary root cause: [One clear sentence. Technical but plain. "A misconfigured deployment config caused..."]

Contributing factors:

• [Factor 1 — e.g. lack of canary deployment meant change hit 100% of traffic immediately]
• [Factor 2 — e.g. alert threshold was set too high to catch the initial degradation]
• [Factor 3 — add as many as are relevant]

Why did our existing safeguards not prevent this? [Honest paragraph explaining why monitoring, tests, or processes didn't catch this earlier. This is where blameless analysis matters most — focus on system gaps, not individual failures.]

Detection

• How was it first detected? [Customer report / automated alert / internal monitoring / manual observation]
• Time from incident start to detection: [X minutes]
• Should we have detected this faster? [Yes / No — and why]

Resolution

What fixed it? [Clear description of the actual fix — one paragraph] Why did this work? [Brief technical explanation] Was there a temporary mitigation before full resolution? [Yes/No — describe if yes]

Action Items

Rules for action items:

• Each action must be specific enough to close as "done" or "not done" — no vague items like "improve monitoring"
• Distinguish between: Prevent recurrence (fix the root cause), Improve detection (catch it faster next time), Improve response (resolve it faster next time)
• Assign a real owner — not "team" or "TBD" if avoidable
• Flag P1 actions as items that block the incident from being marked fully closed

What Went Well

[3–5 honest observations about the response. Include: fast collaboration, good runbooks used, effective escalation, clear communication. This section builds team confidence and reinforces good habits.]

Lessons Learned

[3–5 key insights from this incident that are worth sharing beyond this team. Write these as transferable lessons — e.g. "Our runbook for database failover didn't account for read-replica lag. All runbooks involving database failover should be reviewed."]

Communication Log

[Optional — list external communications sent: status page updates, customer emails, support responses. Include timestamps.]

Scoring Rubric (0–40)

Score any output of this skill before handing it over; 32+ is ship-quality.

Quality Checks

• Timeline has no blame-focused language
• Root cause is specific (not "human error")
• Root cause answers "why did this happen?" not just "what happened?" — it names a system or process gap, not a symptom
• Contributing factors explain the systemic gaps
• Every action item has an owner and due date
• "What went well" section is genuine, not token
• No action item contains vague language like "improve monitoring", "increase resilience", or "better testing" — each must name a specific change
• Executive summary is readable by non-technical leadership

Anti-Patterns

• Do not assign blame to individuals — postmortems must focus on system and process failures
• Do not write action items with vague language like "improve monitoring" — each must name a specific, ownable change
• Do not skip the contributing factors — root cause alone misses the systemic issues that enable incidents
• Do not omit the detection timeline — how long it took to detect matters as much as how long it took to resolve
• Do not treat the postmortem as closed until all action items have named owners and due dates

Usage Examples

• "Write a postmortem for the [incident name] outage"
• "Help me write a P1 incident report"
• "Generate an RCA document for [service] going down on [date]"
• "Draft a blameless postmortem from these notes: [paste notes]"