Incident response team of agents
An outage worked by one agent is five hypotheses chased in series with no record
of what was ruled out. A real incident team splits the work: one agent commands,
several investigate in parallel, one narrates to stakeholders, one writes the
timeline. The commander coordinates and never touches the fix, so investigators
stay heads-down and the decisions survive the night.
Team
- Commander (
incident-commander-role): sets severity, assigns, holds
go/no-go.
- Investigators (
site-reliability-engineer): chase separate hypotheses in
parallel.
- Comms (
technical-writer-role): drafts status updates on a cadence.
- Scribe: timestamps every action and finding.
Shape: a coordinating hub with parallel investigation and a cadence loop; see
war-room-protocol.
Method
- Spawn the commander first, exactly one. It scores severity from
user-visible impact (SEV1 full outage, SEV2 major degraded path, SEV3
contained), opens
incident.md, and assigns surfaces. One named commander
ends the "someone else owns it" gap.
- Fan investigators out on disjoint hypotheses. No two chase the same
graph. Each returns a hypothesis card: symptom, hypothesis, test run,
result. The commander reassigns as cards come back.
- Run the scribe as a passive logger. It subscribes to every agent's output
and appends a timestamped line to
timeline.md: "14:32 rolled back deploy
4471, error rate flat." This is the postmortem's raw material.
- Hold a fixed comms cadence. Comms drafts a status every 15 to 30 minutes
by severity, even when it reads "still investigating, next update 14:50." A
human approves before anything posts to a public status page.
- Drive to mitigation before root cause. The commander picks the fastest
safe stop: roll back, fail over, flip the flag, shed load. Forensics wait;
the customer's minutes do not.
- Declare resolved against written criteria, then hand off. Metrics normal
for a set window with no manual mitigation holding them up. The commander
names a postmortem owner and date and closes the channel.
Run it
In Claude Code, launch investigators as parallel subagents in one orchestrator
turn, each with the incident brief and a distinct hypothesis; the commander
agent (or you as orchestrator) reads their cards and the scribe's timeline to
pick the next move, and comms output routes to a human gate before external
posting. Port it to CrewAI as a hierarchical crew with the commander as manager,
to AutoGen as a GroupChat with a manager agent, or to LangGraph as a supervisor
routing to investigator nodes over a shared state object.
Signals it works
- Any agent's output names the current commander and severity without scrolling.
- The timeline lets someone joining at hour two catch up in two minutes.
- Mitigation is the fastest safe option, not the most satisfying root-cause fix.
Boundaries
This coordinates the live response, not the retrospective, which a postmortem
skill owns, nor the code fix, which stays with the owning engineers. Severity
ladders and paging policy are company convention: match your on-call runbook. A
security breach or data-disclosure event pulls in human security and legal, who
make those calls.
1---2name: agent-incident-response-team3description: Replicate a live incident team as agents with a commander, parallel investigators, comms, and a scribe, coordinated on a fixed cadence. Use when you want an outage worked by a coordinated agent team instead of one agent debugging alone.4---56# Incident response team of agents78An outage worked by one agent is five hypotheses chased in series with no record9of what was ruled out. A real incident team splits the work: one agent commands,10several investigate in parallel, one narrates to stakeholders, one writes the11timeline. The commander coordinates and never touches the fix, so investigators12stay heads-down and the decisions survive the night.1314## Team1516- **Commander** (`incident-commander-role`): sets severity, assigns, holds17 go/no-go.18- **Investigators** (`site-reliability-engineer`): chase separate hypotheses in19 parallel.20- **Comms** (`technical-writer-role`): drafts status updates on a cadence.21- **Scribe**: timestamps every action and finding.2223Shape: a coordinating hub with parallel investigation and a cadence loop; see24`war-room-protocol`.2526## Method27281. **Spawn the commander first, exactly one.** It scores severity from29 user-visible impact (SEV1 full outage, SEV2 major degraded path, SEV330 contained), opens `incident.md`, and assigns surfaces. One named commander31 ends the "someone else owns it" gap.322. **Fan investigators out on disjoint hypotheses.** No two chase the same33 graph. Each returns a hypothesis card: symptom, hypothesis, test run,34 result. The commander reassigns as cards come back.353. **Run the scribe as a passive logger.** It subscribes to every agent's output36 and appends a timestamped line to `timeline.md`: "14:32 rolled back deploy37 4471, error rate flat." This is the postmortem's raw material.384. **Hold a fixed comms cadence.** Comms drafts a status every 15 to 30 minutes39 by severity, even when it reads "still investigating, next update 14:50." A40 human approves before anything posts to a public status page.415. **Drive to mitigation before root cause.** The commander picks the fastest42 safe stop: roll back, fail over, flip the flag, shed load. Forensics wait;43 the customer's minutes do not.446. **Declare resolved against written criteria, then hand off.** Metrics normal45 for a set window with no manual mitigation holding them up. The commander46 names a postmortem owner and date and closes the channel.4748## Run it4950In Claude Code, launch investigators as parallel subagents in one orchestrator51turn, each with the incident brief and a distinct hypothesis; the commander52agent (or you as orchestrator) reads their cards and the scribe's timeline to53pick the next move, and comms output routes to a human gate before external54posting. Port it to CrewAI as a hierarchical crew with the commander as manager,55to AutoGen as a GroupChat with a manager agent, or to LangGraph as a supervisor56routing to investigator nodes over a shared state object.5758## Signals it works5960- Any agent's output names the current commander and severity without scrolling.61- The timeline lets someone joining at hour two catch up in two minutes.62- Mitigation is the fastest safe option, not the most satisfying root-cause fix.6364## Boundaries6566This coordinates the live response, not the retrospective, which a postmortem67skill owns, nor the code fix, which stays with the owning engineers. Severity68ladders and paging policy are company convention: match your on-call runbook. A69security breach or data-disclosure event pulls in human security and legal, who70make those calls.