LarrySequino Incident Postmortem
The output is a small number of changes that get made. Everything else is narrative.
Procedure
- Build the timeline from timestamps, not memory: deploy log, alerting, chat, and the graph that first moved. Memory reorders events under stress, reliably.
- Mark three moments explicitly: when it started, when anyone knew, and when it was mitigated. The gap between the first two is usually the real finding.
- Separate the trigger from the conditions. The deploy that broke it is the trigger; the missing alert and the absent rollback path are why it lasted an hour.
- Keep it blameless in the strict sense: no names attached to mistakes, and every "why did they" rewritten as "what made that the reasonable choice".
- Write action items with an owner, a date, and a size. An unowned action item is a sentence, and it will be there unchanged at the next postmortem.
- Circulate to everyone who was paged, then to everyone affected.
Output
Timeline, the three moments, contributing factors, and the action list. Two pages at most; the version nobody reads is the one that goes to five.