# Observability Coverage Audit

> Observability Coverage Audit

- Skill: `liyown/observability-coverage-audit` (Agent Skill, multi-file: 6 files)
- Install (CLI): `npx skillmds@latest add liyown/observability-coverage-audit`
- Raw SKILL.md: https://api.skillmd.com/api/skills/liyown/observability-coverage-audit/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Security
- Author: liyown (https://skillmd.com/u/liyown)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/liyown/observability-coverage-audit

---


# Observability Coverage Audit

Find which code paths, error states, or failure modes are exercised by tests, monitors, alerts, or runbooks — and which are silent. The output is a per-site coverage matrix with gap rows for the uncovered sites.

## Required Loading

Always load:

- `prompts/coverage-audit.md` — the per-site coverage framework; the per-control matrix.

## When To Run

- After an incident, before closing the postmortem: "could this have been detected earlier?".
- Before launching a new code path: "is this path observable end to end?".
- Quarterly as part of the resilience-engineering review.
- When a SLO breach keeps recurring despite the obvious fix: "is the test that 'fixes' it actually testing the production path?".

## Discovery Order

1. Identify the in-scope code path or failure class. Examples: the `payments.charge` path; every `throw` in the service; every external HTTP call.
2. For each site, find the existing coverage: test files (`*_test.go`, `*.spec.ts`, etc.), alert rules (in the metrics backend), runbook entries (in the runbook repo), dashboards.
3. For each site, ask: does the existing coverage actually exercise the production path? A test that mocks the dependency is not coverage of the dependency.
4. For each gap, classify: missing test, missing alert, missing runbook, missing all three. The classification is the suggested fix.

## Output Contract

The coverage matrix is grouped by site. For each site, name:

- the site (path + line)
- the test coverage (existing test name, or `MISSING`)
- the alert coverage (existing alert name, or `MISSING`)
- the runbook coverage (existing runbook path, or `MISSING`)
- the suggested fix (which coverage to add)

A site missing all three is a `COV-SILENT` finding. A site missing only the runbook is a `COV-NO-RUNBOOK` finding. The classification is the suggested-fix priority.

## Tools

- **`codegraph_explore`** — one-shot fanout when the in-scope path is a feature.
- **`codegraph_search`** — locate the error / exception / throw site by name.
- **`codegraph_callers`** — every site that can hit the error.
- **`codegraph_files`** — test file filter.
- **Fallback** — per-feature source inspection + manual test/alert/runbook search.

## Fallback

If CodeGraph is unavailable, the fallback is per-feature source inspection. The final report must include the line:

```text
CodeGraph unavailable; coverage gathered by rg/file inspection.
```

## Related Skills

- **`spec-doc-linter`** — keeps `DevAgent.md` / `CONTEXT.md` honest.
- **`incident-call-trace`** — walks the failing path during an incident; this skill is the post-incident 'could this have been detected?' audit.
- **`data-lineage-trace`** — the upstream input: a coverage gap without a data-lineage report is missing the sink list.
- **`goal-driven-development`** — orchestrates the spec-to-code flow; this skill is the post-incident / pre-launch counterpart.

## Examples

Each `bad-*` doc has a matching `good-*` in `examples/` showing the minimum matrix. Read them side by side to calibrate depth. `examples/coverage-output.md` is the canonical "what the agent should emit" sample.