STANDARD OPERATING PROCEDURE
Purpose
Design and implement OpenTelemetry collection, instrumentation, and validation so traces, metrics, and logs are actionable.
Trigger Conditions
- Positive: Add or improve OpenTelemetry instrumentation; Standardize telemetry schema across services; Harden collectors/pipelines with SLOs
- Negative: Performance debugging without instrumentation changes (route to performance-analysis); Pure infrastructure build (route to infrastructure); Vendor-specific dashboard asks without pipeline changes (route to production-readiness)
Guardrails
- Structure-first: keep SKILL.md aligned with examples/, tests/, and any resources/references so downstream agents always have scaffolding.
- Adversarial validation is mandatory: cover boundary cases, failure paths, and rollback drills before declaring the SOP complete.
- Prompt hygiene: separate hard vs. soft vs. inferred constraints and confirm inferred constraints before acting.
- Explicit confidence ceilings: format as 'Confidence: X.XX (ceiling: TYPE Y.YY)' and never exceed the ceiling for the claim type.
- MCP traceability: tag sessions WHO=operations-{name}-{session_id}, WHY=skill-execution, and capture evidence links in outputs.
- Avoid anti-patterns: undocumented changes, missing rollback paths, skipped tests, or unbounded automation without approvals.
Required Artifacts
- SKILL.md (this SOP)
- examples/ for instrumentation
- tests/ for telemetry validation
- resources/ for collectors
Execution Phases
Baseline signals
- Inventory services, runtimes, and existing telemetry
- Identify critical user journeys and SLOs
- Capture schema and attribute standards
Design OTEL pipeline
- Plan exporters, collectors, sampling, and resource attributes
- Define security, retention, and cost controls
- Prepare rollout stages and fallback options
Instrument and deploy
- Add or refine instrumentation with context propagation
- Deploy collectors/agents with configs per environment
- Enable logging/metrics/traces validation and alerts
Validate and tune
- Run data-quality checks (cardinality, loss, latency)
- Validate SLOs/SLIs and alert thresholds
- Document runbooks and continuous improvement loops
Output Format
- Instrumentation plan mapped to key journeys
- Collector topology and configuration references
- Schema/attribute standards and governance notes
- Validation report on data quality and SLOs
- Runbook for operations, tuning, and fallbacks
Validation Checklist
- Sampling/resource impact reviewed and acceptable
- Data quality verified for completeness and latency
- SLOs/SLIs defined with alert thresholds
- Security and retention controls documented
- Confidence ceiling stated for telemetry readiness
Confidence: 0.70 (ceiling: inference 0.70) - OpenTelemetry SOP applies structured rollout and verification
1---2name: opentelemetry-observability-23description: Instrument services with OpenTelemetry and reliable signal pipelines4---5
6
7
8
9## STANDARD OPERATING PROCEDURE
10
11### Purpose
12Design and implement OpenTelemetry collection, instrumentation, and validation so traces, metrics, and logs are actionable.
13
14### Trigger Conditions
15- **Positive:** Add or improve OpenTelemetry instrumentation; Standardize telemetry schema across services; Harden collectors/pipelines with SLOs
16- **Negative:** Performance debugging without instrumentation changes (route to performance-analysis); Pure infrastructure build (route to infrastructure); Vendor-specific dashboard asks without pipeline changes (route to production-readiness)
17
18### Guardrails
19- Structure-first: keep SKILL.md aligned with examples/, tests/, and any resources/references so downstream agents always have scaffolding.
20- Adversarial validation is mandatory: cover boundary cases, failure paths, and rollback drills before declaring the SOP complete.
21- Prompt hygiene: separate hard vs. soft vs. inferred constraints and confirm inferred constraints before acting.
22- Explicit confidence ceilings: format as 'Confidence: X.XX (ceiling: TYPE Y.YY)' and never exceed the ceiling for the claim type.
23- MCP traceability: tag sessions WHO=operations-{name}-{session_id}, WHY=skill-execution, and capture evidence links in outputs.
24- Avoid anti-patterns: undocumented changes, missing rollback paths, skipped tests, or unbounded automation without approvals.
25
26### Required Artifacts
27- SKILL.md (this SOP)
28- examples/ for instrumentation
29- tests/ for telemetry validation
30- resources/ for collectors
31
32### Execution Phases
331. **Baseline signals**
34 - Inventory services, runtimes, and existing telemetry
35 - Identify critical user journeys and SLOs
36 - Capture schema and attribute standards
37
382. **Design OTEL pipeline**
39 - Plan exporters, collectors, sampling, and resource attributes
40 - Define security, retention, and cost controls
41 - Prepare rollout stages and fallback options
42
433. **Instrument and deploy**
44 - Add or refine instrumentation with context propagation
45 - Deploy collectors/agents with configs per environment
46 - Enable logging/metrics/traces validation and alerts
47
484. **Validate and tune**
49 - Run data-quality checks (cardinality, loss, latency)
50 - Validate SLOs/SLIs and alert thresholds
51 - Document runbooks and continuous improvement loops
52
53### Output Format
54- Instrumentation plan mapped to key journeys
55- Collector topology and configuration references
56- Schema/attribute standards and governance notes
57- Validation report on data quality and SLOs
58- Runbook for operations, tuning, and fallbacks
59
60### Validation Checklist
61- Sampling/resource impact reviewed and acceptable
62- Data quality verified for completeness and latency
63- SLOs/SLIs defined with alert thresholds
64- Security and retention controls documented
65- Confidence ceiling stated for telemetry readiness
66
67Confidence: 0.70 (ceiling: inference 0.70) - OpenTelemetry SOP applies structured rollout and verification
majiayu000