# Engineering Incident Postmortem > Runs a blameless incident post-mortem: reconstructs the timeline from evidence, separates trigger from cause from contributing conditions, and produces owned, dated remediation actions. Use when an incident, outage, degradation, or severe bug has been resolved and a written review is due, or when a draft post-mortem needs review before it goes to a review board. Trigger on 'write the post-mortem', 'RCA for the outage', 'incident review', 'five whys', 'what was the root cause', 'corrective actions for INC-'. Not for triaging an incident that is still live — that is engineering-incident-command; not for routine bug write-ups with no customer or availability impact, which belong in the ticket. - Skill: `alihusains/engineering-incident-postmortem` (Agent Skill) - Install (CLI): `npx skillmds@latest add alihusains/engineering-incident-postmortem` - Raw SKILL.md: https://api.skillmd.com/api/skills/alihusains/engineering-incident-postmortem/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Coding & Dev Tools - Author: alihusains (https://skillmd.com/u/alihusains) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/alihusains/engineering-incident-postmortem --- # Incident post-mortem ## Purpose Post-mortems fail in three predictable ways: they stop at the trigger ("a bad deploy went out") and call it a cause, they blame a person instead of the conditions that made the error easy, and they end in actions nobody owns. This skill fixes the timeline to evidence, forces the cause analysis past the first plausible answer, and refuses to close on an action without an owner and a date. ## Prerequisites - **Inputs:** the incident record (ID, severity, declared and resolved times); the alert or report that started detection; deploy/change log for the affected services covering the 48h before impact; chat transcript or timeline of responder actions; the customer-visible impact (what broke, for whom, for how long, measured — not estimated). - **Access:** monitoring/dashboards for the impact window, change management records, ticket system to file actions. If the impact window or the change log is missing, say so and stop. A post-mortem built on recollection produces a plausible story and the wrong actions. ## Procedure 1. **Fix the timeline before writing any narrative.** Build a table of timestamped events with a source for each. Every row is one of: a change, a signal, a human action, or a state transition. If a row has no source, mark it `[unverified]` — never silently promote recollection to fact. | Time (UTC) | Event | Type | Source | | --- | --- | --- | --- | | 09:14 | Config change deployed to payments-api | change | deploy log | | 09:41 | Error rate crosses 2% | signal | dashboard | | 09:58 | On-call paged | signal | alert record | 2. **Derive the four durations.** Time to detect (impact start → first signal), time to page (signal → human engaged), time to diagnose (engaged → cause identified), time to mitigate (diagnose → impact ends). Whichever is longest is where the actions must concentrate. Teams routinely write five actions about the code change and none about the 44 minutes of undetected impact. 3. **Separate trigger, cause, and contributing conditions.** | Layer | Question it answers | Test that it is the right layer | | --- | --- | --- | | Trigger | What set it off this time? | Would removing it prevent *this* recurrence only? | | Cause | What made that trigger able to cause harm? | Would fixing it stop a class of incidents? | | Contributing conditions | What made detection or recovery slow? | Would fixing it shorten the next unrelated incident? | A trigger is never a sufficient answer. "Someone deployed a bad config" is a trigger; "config changes deploy to all regions simultaneously with no validation gate" is the cause. 4. **Push each cause chain until it lands on something you can change.** Stop when the next "why" would be a statement about human nature or organisational budget you cannot action. Stopping early gives you "the engineer made a mistake"; going too far gives you "we are under-resourced". The actionable band sits between: the missing guardrail, the misleading interface, the alert that did not exist. 5. **Test each candidate cause against the counterfactual.** For each, ask: if this had been different, would the impact have been prevented or materially reduced? If no, it is context, not cause — move it to the narrative and drop it from the action list. 6. **Write actions in the required shape.** Every action: a verb, a specific system, an owner (a named individual, never a team), a due date, and a ticket ID. Classify each: | Class | Reduces | Ship within | | --- | --- | --- | | Prevent | Probability of recurrence | The current cycle | | Detect | Time to detect | The current cycle | | Mitigate | Blast radius or time to recover | Next cycle | | Process/knowledge | Repeat confusion in response | Next cycle | If every action is class Prevent, the review is incomplete — detection and mitigation gaps almost always exist and are usually cheaper to close. 7. **Run the blameless check before circulation.** Scan the draft for named individuals. A person's name is acceptable in the timeline as an actor ("on-call acknowledged") and unacceptable in the cause section. Rewrite "X misconfigured the limit" as "the limit could be set to an invalid value without validation". 8. **Set a verification date.** Actions are marked done when the change is in production, not when the ticket closes. Book a check 30 days out that confirms the guardrail exists and fires. ## Failure modes this skill exists to prevent - **Cause laundering.** The trigger is restated in passive voice and presented as the root cause. Caught by step 3's "class of incidents" test. - **Action inflation.** Fourteen actions, none prioritised; six months later two are done and none are the important ones. Cap at five; the rest are backlog. - **Duration blindness.** The whole review addresses the change and ignores that detection took 44 minutes. Caught by step 2. - **Silent unknowns.** Gaps in the timeline get smoothed over by narrative. Write "we do not know why the retry storm began; investigation ticket X" — an explicit unknown is a finding, an implicit one is a lie. ## Data handling Classification: **Internal** by default; **Confidential** if the incident involved a security breach, personal data exposure, or financial loss. Post-mortem text must not carry customer identifiers, account numbers, or log lines containing personal data — reference the affected records by count and by ticket, not by value. If someone pastes raw customer records or credentials into the review, flag it and do not incorporate them. ## Boundaries - Incident still active, or you are deciding mitigations in the moment — use `engineering-incident-command`, not this. - Change was routine and had no customer or availability impact — record it in the ticket; a post-mortem adds process cost and no learning. - Security breach requiring regulatory notification — this skill covers the engineering review only; notification obligations run through `legal-compliance-breach-notification`. - Choosing between architectural options to close an action — that decision belongs in `engineering-decision-record`. ## Hand-offs - **Receives from:** `engineering-incident-command` (timeline, severity, responder log). - **Routes to:** `engineering-decision-record` when a remediation implies an architectural change; `it-change-management` when the fix needs a controlled change window; `engineering-code-review` for the review depth on the fix itself. - **See also:** `data-analytics-report-qa` when impact numbers must be quoted externally and need a verification pass first.