Observability And SLO
Use this skill when a change needs production visibility, alerting, SLOs, or dashboard/runbook updates.
Workflow
- Identify the user-visible behavior that should be measured.
- Define SLIs before alerts: availability, latency, correctness, freshness, durability, or throughput.
- Define SLO targets and error budget policy in
docs/harness/SLO_POLICY.md. - Map telemetry:
- logs for discrete events and audit trails
- metrics for aggregate health and alerting
- traces for request paths and dependency latency
- Add alert criteria only when a human action is expected.
- Update
docs/harness/OBSERVABILITY.mdanddocs/harness/OPERATIONS_RUNBOOK.md. - Add dashboard and runbook links or placeholders when live URLs are not available.
Alert Quality Checklist
- The alert maps to a user impact or fast-moving risk.
- The alert has an owner and runbook.
- The runbook includes first diagnostics, rollback, and escalation.
- The threshold avoids paging for harmless noise.
- The signal can distinguish service failure from dependency failure where possible.
Required Output
Return SLIs, SLOs, telemetry changes, alerts, dashboards, runbook updates, and remaining blind spots.