Observability And Operations Audit
Definition
Confirm the team can see, diagnose, and operate the system in production. A release is not ready if failures will be invisible or ownerless.
Questions To Ask
- What logs, metrics, traces, and health checks prove the system is healthy?
- Which dashboards and alerts cover production-critical paths?
- Who owns each service, job, queue, and dependency?
- What runbooks explain common incidents and recovery steps?
- What operational gaps would slow response?
Existing Project Comparison
- Inspect logging calls, metric names, tracing, health endpoints, dashboards, alert config, runbooks, ownership docs, and error reporting.
- Compare observability coverage against must-have journeys, dependencies, and failure modes.
- Check that sensitive data is not exposed in logs or telemetry.
Suggestive Plan
- Map critical journeys and operational signals.
- Inspect logs, metrics, traces, health checks, dashboards, and alerts.
- Verify runbooks and ownership are discoverable.
- Identify invisible failures and noisy or missing alerts.
- Update
PRODUCTION_READINESS.md.
Example
For invoice sending, confirm metrics for send success and failure, logs with invoice IDs but no private data, alert on queue failure, and a resend runbook.
Vocabulary
- Observability: ability to understand system state from outputs.
- Health check: endpoint or probe indicating service readiness or liveness.
- Runbook: operational guide for diagnosing and fixing incidents.
- Ownership: accountable person or team for a production area.
Expected Outcome
Produce an operations audit with observability evidence, alert and dashboard coverage, runbook status, ownership, blockers, and accepted risks.
1---2name: observability-operations-audit3description: Audit observability and operations readiness before production. Use when checking logs, metrics, traces, dashboards, alerts, health checks, runbooks, ownership, on-call readiness, and operational evidence.4---56# Observability And Operations Audit78## Definition910Confirm the team can see, diagnose, and operate the system in production. A release is not ready if failures will be invisible or ownerless.1112## Questions To Ask1314- What logs, metrics, traces, and health checks prove the system is healthy?15- Which dashboards and alerts cover production-critical paths?16- Who owns each service, job, queue, and dependency?17- What runbooks explain common incidents and recovery steps?18- What operational gaps would slow response?1920## Existing Project Comparison2122- Inspect logging calls, metric names, tracing, health endpoints, dashboards, alert config, runbooks, ownership docs, and error reporting.23- Compare observability coverage against must-have journeys, dependencies, and failure modes.24- Check that sensitive data is not exposed in logs or telemetry.2526## Suggestive Plan27281. Map critical journeys and operational signals.292. Inspect logs, metrics, traces, health checks, dashboards, and alerts.303. Verify runbooks and ownership are discoverable.314. Identify invisible failures and noisy or missing alerts.325. Update `PRODUCTION_READINESS.md`.3334## Example3536For invoice sending, confirm metrics for send success and failure, logs with invoice IDs but no private data, alert on queue failure, and a resend runbook.3738## Vocabulary3940- Observability: ability to understand system state from outputs.41- Health check: endpoint or probe indicating service readiness or liveness.42- Runbook: operational guide for diagnosing and fixing incidents.43- Ownership: accountable person or team for a production area.4445## Expected Outcome4647Produce an operations audit with observability evidence, alert and dashboard coverage, runbook status, ownership, blockers, and accepted risks.
harsh817