Observability Maturity Check

Use when performing observability maturity check — evaluates an organization's observability maturity across the three pillars (metrics, logs, traces) plus alerting, dashboards, and AIOps capabilities. Identifies gaps in visibility, assesses signal quality, and produces a roadmap for achieving full-stack observability.

cloudthinker-ai Updated Sep 8, 2026 7 repo stars

Product & Planning

Install

Press C to copy

$ npx skillmds@latest add cloudthinker-ai/observability-maturity-check

Installs into ~/.claude/skills (or .claude/skills inside a project). Restart Claude Code to pick it up.

Download SKILL.md

＋ Add to plugin

README badge

[![SkillMD](https://skillmd.com/badge/cloudthinker-ai/observability-maturity-check.svg)](https://skillmd.com/skills/cloudthinker-ai/observability-maturity-check)

Trust & safety

2 of 3 audits passed

Last audited Sep 24, 2026 · commit b921d6b

Permissions this skill uses

Documentation only
No scripts, network access, or credentials are involved.

Works with: Claude Code Claude.ai OpenAI Codex +

File contents

Observability Maturity Check

Phase 1: Metrics Assessment

Evaluate metrics collection and usage
- Infrastructure metrics (CPU, memory, disk, network)
- Application metrics (request rate, error rate, latency - RED)
- Business metrics (transactions, revenue, user activity)
- Custom metrics for domain-specific KPIs
- Metrics retention and resolution appropriate
- Metrics naming conventions standardized
- Service-level indicators (SLIs) derived from metrics
Score: 1 (Basic infra only) to 5 (Full-stack with business metrics)

Metrics Coverage

Layer	Coverage	Gaps	Quality
Infrastructure	% of hosts/containers		High/Med/Low
Application (RED)	% of services
Database	% of instances
External dependencies	% of integrations
Business KPIs	% defined

Phase 2: Logging Assessment

Evaluate logging practices
- Centralized log aggregation
- Structured logging (JSON) across all services
- Consistent log levels (DEBUG, INFO, WARN, ERROR)
- Request/correlation IDs in all log entries
- PII/sensitive data redaction in logs
- Log retention policies defined
- Log search performance adequate
- Log-based alerting configured
Score: 1 (Scattered file logs) to 5 (Centralized, structured, searchable)

Phase 3: Distributed Tracing Assessment

Evaluate tracing implementation
- Tracing instrumented across services
- Trace propagation across service boundaries
- Span attributes include relevant context
- Sampling strategy defined (head, tail, adaptive)
- Trace-to-log and trace-to-metric correlation
- Service dependency map generated from traces
- Trace data used in incident investigation
Score: 1 (No tracing) to 5 (Full distributed tracing with correlation)

Three Pillars Coverage

Pillar	Coverage (%)	Quality (1-5)	Tool	Key Gap
Metrics	%
Logs	%
Traces	%
Combined	%	/5

Phase 4: Alerting Quality Assessment

Evaluate alerting effectiveness
- Alerts are actionable (every alert requires human action)
- Alert severity levels match impact
- On-call rotation and escalation configured
- Alert noise ratio acceptable (signal vs. noise)
- Symptom-based alerts (not cause-based)
- Runbooks linked to alerts
- Alert fatigue measured and managed
- SLO-based alerting (burn rate alerts)
Measure alert quality metrics

Alert Quality Metrics

Metric	Current	Target	Status
Alerts/week		<
Actionable alerts %	%	> 80%
Alerts with runbooks %	%	100%
MTTA (time to acknowledge)	min	< 5 min
False positive rate	%	< 10%
Duplicate/correlated alerts	%	< 5%

Phase 5: Dashboards & Visualization

Evaluate dashboard practices
- Service-level dashboards for each team
- On-call dashboard (single pane of glass)
- SLO status dashboards
- Infrastructure overview dashboards
- Business metrics dashboards
- Dashboard naming and organization standards
- Dashboard-as-code (version controlled)
Assess dashboard discoverability and usefulness

Phase 6: Advanced Capabilities

Evaluate advanced observability features
- Anomaly detection (automated baseline comparison)
- Root cause analysis automation
- Service dependency visualization
- Change correlation (deploy/config changes vs. incidents)
- Continuous profiling (CPU, memory, lock)
- Real User Monitoring (RUM) / frontend observability
- Synthetic monitoring for critical paths
- OpenTelemetry adoption for vendor neutrality

Maturity Scorecard

Dimension	Score (1-5)	Current State	Target State
Metrics
Logging
Tracing
Alerting
Dashboards
Advanced (AI/ML)
Overall	/5

Counter-Rationalizations

Shortcut	Counter	Why
"We can skip some steps for this case"	Adapt the workflow steps, don't skip them	Skipped steps are where incidents and oversights originate
"The user seems to already know what to do"	Complete all workflow phases with the user	The workflow catches blind spots that experience alone misses
"This is a minor case, full process is overkill"	Scale the process down, don't turn it off	Minor cases become major when unstructured; the process scales, not disappears
"I'll fill in the details later"	Complete each section before moving on	Deferred details are forgotten; real-time capture is more accurate
"The template output isn't necessary"	Always produce the structured output format	Structured output enables comparison, audit trails, and handoff to other teams

Output Format

Pillar Assessment: Metrics, logs, traces coverage and quality
Alert Quality Report: Noise analysis and improvement recommendations
Coverage Gaps: Services and layers lacking observability
Tool Assessment: Current stack evaluation and recommendations
Improvement Roadmap: Phased plan to advance observability maturity

Action Items

Assess coverage across all three pillars
Audit alert quality and reduce noise
Implement structured logging across all services
Roll out distributed tracing to uninstrumented services
Standardize dashboards and make discoverable
Evaluate and pilot advanced capabilities
Schedule quarterly observability review

---name: observability-maturity-checkdescription: Use when performing observability maturity check — evaluates an organization's observability maturity across the three pillars (metrics, logs, traces) plus alerting, dashboards, and AIOps capabilities. Identifies gaps in visibility, assesses signal quality, and produces a roadmap for achieving full-stack observability.---# Observability Maturity Check## Phase 1: Metrics Assessment1. Evaluate metrics collection and usage - [ ] Infrastructure metrics (CPU, memory, disk, network) - [ ] Application metrics (request rate, error rate, latency - RED) - [ ] Business metrics (transactions, revenue, user activity) - [ ] Custom metrics for domain-specific KPIs - [ ] Metrics retention and resolution appropriate - [ ] Metrics naming conventions standardized - [ ] Service-level indicators (SLIs) derived from metrics2. Score: 1 (Basic infra only) to 5 (Full-stack with business metrics)### Metrics Coverage| Layer | Coverage | Gaps | Quality ||-------|----------|------|---------|| Infrastructure | % of hosts/containers | | High/Med/Low || Application (RED) | % of services | | || Database | % of instances | | || External dependencies | % of integrations | | || Business KPIs | % defined | | |## Phase 2: Logging Assessment1. Evaluate logging practices - [ ] Centralized log aggregation - [ ] Structured logging (JSON) across all services - [ ] Consistent log levels (DEBUG, INFO, WARN, ERROR) - [ ] Request/correlation IDs in all log entries - [ ] PII/sensitive data redaction in logs - [ ] Log retention policies defined - [ ] Log search performance adequate - [ ] Log-based alerting configured2. Score: 1 (Scattered file logs) to 5 (Centralized, structured, searchable)## Phase 3: Distributed Tracing Assessment1. Evaluate tracing implementation - [ ] Tracing instrumented across services - [ ] Trace propagation across service boundaries - [ ] Span attributes include relevant context - [ ] Sampling strategy defined (head, tail, adaptive) - [ ] Trace-to-log and trace-to-metric correlation - [ ] Service dependency map generated from traces - [ ] Trace data used in incident investigation2. Score: 1 (No tracing) to 5 (Full distributed tracing with correlation)### Three Pillars Coverage| Pillar | Coverage (%) | Quality (1-5) | Tool | Key Gap ||--------|-------------|--------------|------|---------|| Metrics | % | | | || Logs | % | | | || Traces | % | | | || **Combined** | **%** | **/5** | | |## Phase 4: Alerting Quality Assessment1. Evaluate alerting effectiveness - [ ] Alerts are actionable (every alert requires human action) - [ ] Alert severity levels match impact - [ ] On-call rotation and escalation configured - [ ] Alert noise ratio acceptable (signal vs. noise) - [ ] Symptom-based alerts (not cause-based) - [ ] Runbooks linked to alerts - [ ] Alert fatigue measured and managed - [ ] SLO-based alerting (burn rate alerts)2. Measure alert quality metrics### Alert Quality Metrics| Metric | Current | Target | Status ||--------|---------|--------|--------|| Alerts/week | | < | || Actionable alerts % | % | > 80% | || Alerts with runbooks % | % | 100% | || MTTA (time to acknowledge) | min | < 5 min | || False positive rate | % | < 10% | || Duplicate/correlated alerts | % | < 5% | |## Phase 5: Dashboards & Visualization1. Evaluate dashboard practices - [ ] Service-level dashboards for each team - [ ] On-call dashboard (single pane of glass) - [ ] SLO status dashboards - [ ] Infrastructure overview dashboards - [ ] Business metrics dashboards - [ ] Dashboard naming and organization standards - [ ] Dashboard-as-code (version controlled)2. Assess dashboard discoverability and usefulness## Phase 6: Advanced Capabilities1. Evaluate advanced observability features - [ ] Anomaly detection (automated baseline comparison) - [ ] Root cause analysis automation - [ ] Service dependency visualization - [ ] Change correlation (deploy/config changes vs. incidents) - [ ] Continuous profiling (CPU, memory, lock) - [ ] Real User Monitoring (RUM) / frontend observability - [ ] Synthetic monitoring for critical paths - [ ] OpenTelemetry adoption for vendor neutrality### Maturity Scorecard| Dimension | Score (1-5) | Current State | Target State ||-----------|-----------|---------------|-------------|| Metrics | | | || Logging | | | || Tracing | | | || Alerting | | | || Dashboards | | | || Advanced (AI/ML) | | | || **Overall** | **/5** | | |## Counter-Rationalizations| Shortcut | Counter | Why ||----------|---------|-----|| "We can skip some steps for this case" | Adapt the workflow steps, don't skip them | Skipped steps are where incidents and oversights originate || "The user seems to already know what to do" | Complete all workflow phases with the user | The workflow catches blind spots that experience alone misses || "This is a minor case, full process is overkill" | Scale the process down, don't turn it off | Minor cases become major when unstructured; the process scales, not disappears || "I'll fill in the details later" | Complete each section before moving on | Deferred details are forgotten; real-time capture is more accurate || "The template output isn't necessary" | Always produce the structured output format | Structured output enables comparison, audit trails, and handoff to other teams |## Output Format- **Pillar Assessment**: Metrics, logs, traces coverage and quality- **Alert Quality Report**: Noise analysis and improvement recommendations- **Coverage Gaps**: Services and layers lacking observability- **Tool Assessment**: Current stack evaluation and recommendations- **Improvement Roadmap**: Phased plan to advance observability maturity## Action Items- [ ] Assess coverage across all three pillars- [ ] Audit alert quality and reduce noise- [ ] Implement structured logging across all services- [ ] Roll out distributed tracing to uninstrumented services- [ ] Standardize dashboards and make discoverable- [ ] Evaluate and pilot advanced capabilities- [ ] Schedule quarterly observability review

cloudthinker-ai/CloudSkills/tree/main/skills/templates/observability-maturity-check commit b921d6ba7a

Frequently asked questions

npx skillmds@latest add cloudthinker-ai/observability-maturity-check