Incident Triage
Structured incident triage for alerts from any monitoring source. Five steps, consistent every time.
Pass in the raw alert message, a link to the alert, or a description of what's happening.
Triage Process
When an alert appears:
- Classify — what type and severity?
- Scope — blast radius: who's affected, which environment, since when?
- Correlate — what changed recently? Check deploys, merges, config changes
- Investigate — guided checks based on alert type
- Act — summarize, create ticket, escalate or close
Read references/triage-framework.md for the full framework with checklists and bash snippets for each step.
Alert Parsing
Before starting the triage framework, identify the alert source and extract key fields.
Read references/alert-patterns.md for patterns covering PagerDuty, Datadog, CloudWatch, Sentry, uptime monitors, GitHub Actions, AWS SNS/EventBridge, and custom webhooks.
Escalation
When to page, when to watch, when to close. Severity-based response times and communication templates.
Read references/escalation-guide.md for defaults — customize for your team's on-call structure.
Runbook
During Step 4 (Investigate), load references/runbook-template.md to find service health endpoints, dashboards, log locations, and common fixes.
⚠️ This file is a template — it must be filled in before use. If it still contains <!-- placeholder comments, tell the user to populate it with their actual infrastructure before relying on it during an incident.
Scripts
The scripts/ directory contains helper scripts for the correlation and action steps:
scripts/correlate-recent-deploys.sh — list recent CI/CD runs for a repo (Step 3)scripts/correlate-recent-merges.sh — list recently merged PRs for a repo (Step 3)scripts/create-incident-issue.sh — create a GitHub incident issue (Step 5)
Works Well With
- github (Step 3 — Correlate): check recent deploys, merged PRs, and CI run history for affected repos
- aws-ecs-monitor (Step 4 — Investigate): ECS service health, ALB targets, and CloudWatch logs for downtime and resource alerts
- gh-issues (Step 5 — Act): create incident tickets automatically
References
- references/triage-framework.md — full 5-step triage process with checklists
- references/alert-patterns.md — parsing alerts from common sources
- references/escalation-guide.md — severity levels, response times, escalation policy
- references/runbook-template.md — your infrastructure map (⚠️ fill in before use)
1---2name: incident-triage-43description: Structured incident triage for alerts from any monitoring source. Five-step framework: classify severity, scope blast radius, correlate with recent changes, investigate by alert type, and act (summarize, create ticket, escalate). Parses alerts from PagerDuty, Datadog, CloudWatch, Sentry, uptime monitors, GitHub Actions, AWS SNS/EventBridge, and custom webhooks. Customizable escalation policy and runbook. Uses `gh` CLI for deploy correlation. Use when: an alert fires, something is down, error rates spike, a deployment fails, investigating an incident, triaging a monitoring alert, responding to a page, or an exception tracker reports a new or regressed error.4---56# Incident Triage78Structured incident triage for alerts from any monitoring source. Five steps, consistent every time.910Pass in the raw alert message, a link to the alert, or a description of what's happening.1112## Triage Process1314When an alert appears:15161. **Classify** — what type and severity?172. **Scope** — blast radius: who's affected, which environment, since when?183. **Correlate** — what changed recently? Check deploys, merges, config changes194. **Investigate** — guided checks based on alert type205. **Act** — summarize, create ticket, escalate or close2122Read [references/triage-framework.md](references/triage-framework.md) for the full framework with checklists and bash snippets for each step.2324## Alert Parsing2526Before starting the triage framework, identify the alert source and extract key fields.2728Read [references/alert-patterns.md](references/alert-patterns.md) for patterns covering PagerDuty, Datadog, CloudWatch, Sentry, uptime monitors, GitHub Actions, AWS SNS/EventBridge, and custom webhooks.2930## Escalation3132When to page, when to watch, when to close. Severity-based response times and communication templates.3334Read [references/escalation-guide.md](references/escalation-guide.md) for defaults — customize for your team's on-call structure.3536## Runbook3738During Step 4 (Investigate), load [references/runbook-template.md](references/runbook-template.md) to find service health endpoints, dashboards, log locations, and common fixes.3940> ⚠️ **This file is a template — it must be filled in before use.** If it still contains `<!--` placeholder comments, tell the user to populate it with their actual infrastructure before relying on it during an incident.4142## Scripts4344The `scripts/` directory contains helper scripts for the correlation and action steps:4546- `scripts/correlate-recent-deploys.sh` — list recent CI/CD runs for a repo (Step 3)47- `scripts/correlate-recent-merges.sh` — list recently merged PRs for a repo (Step 3)48- `scripts/create-incident-issue.sh` — create a GitHub incident issue (Step 5)4950## Works Well With5152- **github** (Step 3 — Correlate): check recent deploys, merged PRs, and CI run history for affected repos53- **aws-ecs-monitor** (Step 4 — Investigate): ECS service health, ALB targets, and CloudWatch logs for downtime and resource alerts54- **gh-issues** (Step 5 — Act): create incident tickets automatically5556## References5758- [references/triage-framework.md](references/triage-framework.md) — full 5-step triage process with checklists59- [references/alert-patterns.md](references/alert-patterns.md) — parsing alerts from common sources60- [references/escalation-guide.md](references/escalation-guide.md) — severity levels, response times, escalation policy61- [references/runbook-template.md](references/runbook-template.md) — your infrastructure map (⚠️ fill in before use)
knownasnaffy