# Observability

> Use for production observability: logs, metrics, traces, health, dashboards, alerts, SLOs; not debug prints.

- Skill: `kreek/observability` (Agent Skill, multi-file: 3 files)
- Install (CLI): `npx skillmds@latest add kreek/observability`
- Raw SKILL.md: https://api.skillmd.com/api/skills/kreek/observability/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: kreek (https://skillmd.com/u/kreek)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/kreek/observability

---


# Observability

## Iron Law

`NO USER-REACHABLE SERVICE PATH SHIPS BLIND.`

## When to Use

- Logs, metrics, traces, health checks, dashboards, SLOs, alerts, dependency
  health, incident diagnosis, OpenTelemetry, cardinality, burn-rate alerts.

## When NOT to Use

- Local-only scripts or libraries with no operational surface.
- Error type design; use `error-handling`. Release sequencing; use `release`.

## Rules

1. Every new user-reachable path emits request, error, and duration (RED)
   signals, and traces cover its inbound and outbound boundaries. Critical
   dependencies expose latency, error, timeout, retry, circuit state, and
   saturation.
2. Logs are structured events with stable names, typed fields, severity,
   outcome, and trace or correlation ID, using OpenTelemetry semantic
   conventions where they exist. Levels distinguish expected client failures
   from operator-actionable errors. Load `references/logging.md` for any
   logging change.
3. Metric labels are bounded. User IDs, request IDs, and paths go in logs or
   traces, because cardinality is a production cost.
4. Sensitive data is redacted at the source with a field allowlist (see
   `security`); collector filtering is defense in depth only.
5. Liveness never depends on external systems. Readiness does.
6. Alerts fire on user impact, SLO burn, or actionable dependency failure,
   and link to a runbook with action and escalation. Tune noisy alerts by
   signal, owner, or threshold; never silence them.
7. Dashboards answer current health and likely fault location.
8. A prototype may defer observability only if the deferral is recorded and
   the path is promoted before real users reach it.

## Tripwires

| Trigger | Do this instead | False alarm |
|---|---|---|
| "We logged the error, so it is observable" | Add stable event name, outcome, severity, and correlation ID. | The project logger adds those fields automatically and tests prove it. |
| "Put user ID / request ID / path in a metric label" | Bound the labels; put high-cardinality values in logs or traces. | The label set is proven bounded. |
| "Alert on every error" | Alert on user impact, SLO burn, or actionable dependency failure. | Low-volume critical security or data-loss event. |
| "Health check should test the database" | Keep liveness local; put dependencies in readiness. | The endpoint is explicitly readiness. |
| "The collector will redact it" | Redact at the source. | Source redaction is impossible and the risk is documented. |
| "Dashboard later" | Add the view needed to find current health and fault location. | Local-only prototype with the deferral recorded. |

## Handoffs

- `documentation`: runbook shape.
- `async-systems`: stream lag, fanout, replay, delivery semantics.
- `release`: rollout gates and production verification.
- `error-handling`: timeout, retry, and circuit-breaker behavior.
- `security`: redaction allowlists and security-event alerts.

## References

- `references/logging.md`: load for any change to log events or fields.