# Sre

> Use when working on system reliability, SLOs/SLIs/SLAs, incident management, postmortems, capacity planning, chaos engineering, or production readiness. Trigger phrases: "SRE", "site reliability", "SLO", "SLI", "error budget", "incident response", "on-call", "postmortem", "toil reduction", "chaos engineering", "production readiness", "capacity planning", "runbook", "alerting strategy", "uptime", "MTTR", "MTTD".

- Skill: `barastrong/sre` (Agent Skill, multi-file: 5 files)
- Install (CLI): `npx skillmds@latest add barastrong/sre`
- Raw SKILL.md: https://api.skillmd.com/api/skills/barastrong/sre/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Finance & Business
- Author: barastrong (https://skillmd.com/u/barastrong)
- Updated: 2026-09-22
- Page: https://skillmd.com/skills/barastrong/sre

---


# Site Reliability Engineer (SRE)

## Role Summary

A Site Reliability Engineer applies software engineering principles to
operations — ensuring systems are reliable, scalable, and efficient. SRE
balances reliability with velocity using SLOs, error budgets, automation,
and incident management.

## Core Responsibilities

- Define and measure SLOs/SLIs aligned with user experience
- Manage error budgets and reliability trade-offs
- Build and improve monitoring, alerting, and observability
- Incident management: detection, response, mitigation, postmortem
- Automate toil (repetitive operational tasks)
- Capacity planning and load testing
- Chaos engineering and failure injection
- Production readiness reviews for new services
- On-call rotation management and escalation policies

## Standard Workflow

1. **SLO Definition** — identify critical user journeys, define SLIs
   (latency p50/p99, error rate, throughput), set SLO targets.
2. **Monitoring** — instrument services for SLIs; build dashboards;
   actionable alerts (not noise).
3. **Alerting** — symptom-based alerts (user impact), not cause-based.
   Multi-window, multi-burn-rate SLO alerts.
4. **Incident Management** — detect → triage → mitigate → resolve →
   communicate → postmortem.
5. **Postmortem** — blameless, timeline-based, action items with owners
   and deadlines.
6. **Toil Reduction** — identify repetitive manual work; automate or
   eliminate (target <50% toil).
7. **Chaos Engineering** — steady state hypothesis → inject failure →
   observe → fix weaknesses.

## Technology Stack

| Layer | Tools |
|-------|-------|
| Monitoring | Prometheus, Grafana, Datadog, New Relic, Honeycomb |
| Alerting | PagerDuty, Opsgenie, Grafana Alerting, AlertManager |
| Logging | ELK, Loki, Splunk, CloudWatch Logs |
| Tracing | Jaeger, Zipkin, Tempo, OpenTelemetry |
| Incident Mgmt | PagerDuty, Incident.io, FireHydrant, Rootly |
| Chaos | Gremlin, Chaos Monkey, LitmusChaos, Toxiproxy |
| Load Testing | k6, Locust, Gatling, Artillery |
| SLO Tracking | Nobl9, Sloth, custom Prometheus rules |

## Best Practices

- SLOs based on user-facing behavior, not internal metrics.
- Error budget policy: when budget is exhausted, freeze features, fix reliability.
- Symptom-based alerting: page on "users can't check out", not "CPU > 80%".
- Every alert must be actionable — if no action needed, delete the alert.
- Blameless postmortems — focus on systemic improvements, not individuals.
- Production readiness checklist before any new service goes live.
- Target toil <50% — if operations dominate, the system needs engineering work.
- Runbooks for every alert with step-by-step mitigation guide.
- On-call load: max 2 pages/shift; more = alert hygiene problem.

## Anti-Patterns to Avoid

- Alert fatigue: too many non-actionable alerts → pages get ignored.
- SLOs that nobody checks or enforces.
- Postmortems without action items, or action items without owners.
- Heroics culture: relying on specific people instead of system improvements.
- No error budget policy: reliability never gates feature work.
- Manual scaling instead of autoscaling with proper metrics.

## References

- `references/slo-design-guide.md` — SLO definition and measurement
- `references/incident-response-playbook.md` — incident response process
- `references/postmortem-template.md` — blameless postmortem template

## Expected Output Format

1. SLO definition (SLI, target, measurement window, error budget policy)
2. Alert rules (PromQL/Datadog query + runbook link)
3. Postmortem document (timeline, root cause, action items)
4. Dashboard link or configuration