Incident Runbooks
Write procedures an on-call engineer can use with the actual service, deployment
environment, permissions, and team escalation policy. Resolve those facts from
existing configuration and runbooks; identify missing operational decisions.
Procedure Requirements
For each consequential step, give the relevant precondition, exact target,
action, expected observation, and what to do if it fails. Keep diagnosis and
state-changing recovery distinguishable.
- Derive commands, identifiers, dashboards, contacts, and thresholds from the
target environment. Do not substitute fictional production defaults.
- Before destructive actions, specify the selected resources, impact, required
authorization, and recovery path. A count threshold alone is not a safe basis
for terminating sessions, deleting data, or failing over a database.
- Rollback instructions must account for data and schema compatibility.
- Verify recovery through affected user operations and relevant health metrics,
not merely a successful command or running process.
- Use team-defined severity, escalation, and update intervals. If absent, mark
proposed policy as a decision rather than inventing an established practice.
- Record ownership and actual validation status. Label unexecuted commands;
documentation work does not authorize testing recovery against production.
A useful layout is: scope and impact, triage, conditional mitigation, verification
and rollback, escalation, and communication. Omit sections that add no useful
decision or action. Use home-postmortem-writing when an incident review is requested.
1---2name: home-incident-runbook-templates3description: Create service-specific incident runbooks with diagnosis, recovery, rollback, and escalation steps. Use for response-procedure authoring; an active incident does not require writing a full runbook first.4---56# Incident Runbooks78Write procedures an on-call engineer can use with the actual service, deployment9environment, permissions, and team escalation policy. Resolve those facts from10existing configuration and runbooks; identify missing operational decisions.1112## Procedure Requirements1314For each consequential step, give the relevant precondition, exact target,15action, expected observation, and what to do if it fails. Keep diagnosis and16state-changing recovery distinguishable.1718- Derive commands, identifiers, dashboards, contacts, and thresholds from the19 target environment. Do not substitute fictional production defaults.20- Before destructive actions, specify the selected resources, impact, required21 authorization, and recovery path. A count threshold alone is not a safe basis22 for terminating sessions, deleting data, or failing over a database.23- Rollback instructions must account for data and schema compatibility.24- Verify recovery through affected user operations and relevant health metrics,25 not merely a successful command or running process.26- Use team-defined severity, escalation, and update intervals. If absent, mark27 proposed policy as a decision rather than inventing an established practice.28- Record ownership and actual validation status. Label unexecuted commands;29 documentation work does not authorize testing recovery against production.3031A useful layout is: scope and impact, triage, conditional mitigation, verification32and rollback, escalation, and communication. Omit sections that add no useful33decision or action. Use `home-postmortem-writing` when an incident review is requested.
nwiizo