# Distributed Tracing

> Distributed Tracing & Observability

- Skill: `j4flmao/distributed-tracing` (Agent Skill, multi-file: 4 files)
- Install (CLI): `npx skillmds@latest add j4flmao/distributed-tracing`
- Raw SKILL.md: https://api.skillmd.com/api/skills/j4flmao/distributed-tracing/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: DevOps & Infra
- Author: j4flmao (https://skillmd.com/u/j4flmao)
- Updated: 2026-09-21
- Page: https://skillmd.com/skills/j4flmao/distributed-tracing

---

# Distributed Tracing & Observability

## 1. Skill Context
**Focus**: OpenTelemetry (OTel), Span propagation, W3C Trace Context, Sampling strategies, and microservice observability.
**Triggers**: implement distributed tracing, opentelemetry setup, jaeger tracing, trace context, trace correlation

## 2. Advanced Strategy and Execution
The agent must provide architectural guidance on implementing distributed tracing without incurring massive performance or cost overhead.

### Instrumentation & Context Propagation
- **W3C Trace Context**: Explain the mechanics of `traceparent` (`version-trace_id-parent_id-trace_flags`) and `tracestate` headers across HTTP/gRPC boundaries.
- **Baggage**: Distinguish between trace context (used for linking spans) and baggage (used for passing key-value pairs like `tenant_id` down the entire call stack).
- **Auto-Instrumentation vs Manual**: When to rely on eBPF/bytecode-injection (Java Agent, eBPF probes) vs manual span creation for business logic granularity.

### Sampling Strategies
- **Head-Based Sampling**: The decision to sample a trace is made at the root service. Good for performance but might miss downstream errors if not sampled.
- **Tail-Based Sampling**: All spans are collected in a collector (e.g., OTel Collector), and the decision to keep the trace is made after the trace completes. Critical for ensuring 100% of errors or slow requests are captured while dropping successful, fast requests.

## 3. Output Format
- Provide OpenTelemetry Collector configuration snippets (`otelcol.yaml`).
- Explain the code-level injection of tracing headers for asynchronous messaging (e.g., Kafka record headers).
- Diagram the trace flow across 3+ microservices using Mermaid `sequenceDiagram`.