# On Call Design

> On-Call Design

- Skill: `sairam0424/on-call-design` (Agent Skill)
- Install (CLI): `npx skillmds@latest add sairam0424/on-call-design`
- Raw SKILL.md: https://api.skillmd.com/api/skills/sairam0424/on-call-design/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: sairam0424 (https://skillmd.com/u/sairam0424)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/sairam0424/on-call-design

---


# On-Call Design

## When this skill activates

This skill activates when designing, evaluating, or improving on-call rotations,
escalation policies, alerting strategies, and the operational sustainability of
incident response. It addresses rotation fairness, alert fatigue prevention,
runbook coverage, and burnout prevention to create a humane and effective on-call
system.

## Mandatory actions when this skill is active

### Before

1. **Audit current state** — How many pages per week per person? What percentage have
   runbooks? What is the false-positive rate? How is on-call currently compensated?
2. **Identify pain points** — Survey on-call participants: What is most frustrating?
   What causes the most stress? What pages feel unnecessary?
3. **Define service level objectives** — What response times does the business actually
   need? Not everything requires a 2 AM page.

### During

4. **Rotation design principles:**
   - Minimum 2 people in rotation at all times (primary + secondary)
   - Maximum 1 week per rotation (longer causes burnout)
   - Follow-the-sun for global teams (no one gets paged at 3 AM regularly)
   - Handoff includes: active incidents, known issues, upcoming risky changes
   - New team members shadow for 2 rotations before going primary
   - Easy swap mechanism for personal conflicts

5. **Fairness and compensation:**
   - Equal distribution of on-call burden across all eligible engineers
   - Track after-hours pages per person — redistribute if imbalanced
   - Compensate on-call time (stipend for carrying the pager, additional compensation
     for after-hours pages)
   - Swap-friendly policy — no guilt for requesting swaps
   - On-call load counts toward sprint capacity (reduce story points by 20-30%)
   - Never mandate on-call for people who did not agree to it at hiring

6. **Escalation SLAs:**
   - **Acknowledge** — 5 minutes (confirm human is aware)
   - **Triage** — 15 minutes (assess severity, determine if action needed now)
   - **Resolve or escalate** — 60 minutes (either fix it or pull in help)
   - Clear escalation path: primary → secondary → team lead → engineering manager
   - Automatic escalation if acknowledgment SLA is missed
   - No shame in escalating — better to involve help early than struggle alone

7. **Runbook coverage requirements:**
   - Every alert MUST have a corresponding runbook
   - Runbook contains: what the alert means, impact if ignored, diagnostic steps,
     resolution steps, escalation criteria
   - If no runbook exists for an alert, the alert should not page — it is toil
   - Runbooks must be tested quarterly (can a new engineer follow them successfully?)
   - Runbook updates are mandatory after any incident where the runbook was insufficient

8. **Toil budget management:**
   - Define toil: repetitive, automatable, reactive work with no lasting value
   - If >30% of on-call time is toil, invest in automation
   - Track toil hours separately from incident response hours
   - Dedicate engineering time each sprint to reducing top toil sources
   - Toil that persists for >3 months without investment is a leadership failure

9. **Alert fatigue prevention:**
   - If an alert does not require human action within 5 minutes, delete it
   - Target: fewer than 5 pages per week per person
   - Review all alerts quarterly: keep, tune, or delete
   - Silence non-actionable alerts aggressively (log them, don't page for them)
   - Group related alerts to prevent page storms (one page for a cascading failure,
     not 50)
   - Every page that did not need human intervention is a bug in your alerting

10. **Burnout prevention:**
    - Monitor page frequency trends — rising trends require immediate action
    - After a high-severity incident with extended resolution, give the responder
      compensatory time off
    - Regular check-ins on on-call stress (not just "how many pages" but "how do you
      feel about on-call?")
    - Option to temporarily step out of rotation for personal reasons without stigma
    - Celebrate good on-call: quiet weeks are victories, not luck

### After

11. **Publish on-call handbook** — Document all policies, rotations, escalation paths,
    compensation, and expectations in one discoverable location.
12. **Review quarterly** — Analyze page frequency, response times, toil percentage,
    fairness distribution, and team satisfaction.
13. **Iterate on alerts** — After every on-call rotation, the outgoing person reviews
    alerts received and recommends: keep as-is, tune threshold, add runbook, or delete.

## Self-check before task completion

- [ ] Rotation has minimum 2 people with maximum 1-week shifts
- [ ] Compensation model defined for on-call time and after-hours pages
- [ ] Escalation SLAs documented (acknowledge: 5min, triage: 15min, resolve: 60min)
- [ ] Every alert has a corresponding runbook (or is deleted/silenced)
- [ ] Page frequency target set (<5/week/person) with current baseline measured
- [ ] Toil budget defined (<30%) with automation investment plan for top sources
- [ ] Alert review cadence established (quarterly minimum)
- [ ] Burnout prevention measures in place (comp time, swap policy, check-ins)
- [ ] On-call handbook published and discoverable
- [ ] Fairness tracking mechanism active (pages per person distribution)