# Observability and Scaling

> Prometheus TSDB compaction and OpenTelemetry trace context propagation.

- Skill: `j4flmao/observability-and-scaling` (Agent Skill)
- Install (CLI): `npx skillmds@latest add j4flmao/observability-and-scaling`
- Raw SKILL.md: https://api.skillmd.com/api/skills/j4flmao/observability-and-scaling/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: j4flmao (https://skillmd.com/u/j4flmao)
- Updated: 2026-09-21
- Page: https://skillmd.com/skills/j4flmao/observability-and-scaling

---

# Observability & Scaling: Core Mechanics

## Prometheus TSDB Compaction
Prometheus utilizes a custom Time Series Database (TSDB) optimized for fast ingestion and querying.
- **Head Block**: Active data is stored in memory and a Write-Ahead Log (WAL).
- **Compaction**: Persistent blocks (default 2h) are compacted logarithmically into larger blocks (e.g., 2h -> 8h -> 32h) to reduce index overhead and improve query performance.
- **Index Structure**: Inverted index mapping label matchers to series IDs.

```mermaid
flowchart TD
%%{init: {"theme": "default", "flowchart": {"useMaxWidth": true}}}%%
    subgraph TSDBPrometheusTSDB ["Prometheus TSDB<br><br><br>"]
        Mem[In-Memory Head] -->|"Flush(2h)"| Block1[2h Block]
        Block1 -->|"Compact(Level 1)"| Block2[8h Block]
    end
    subgraph TracingOpenTelemetry ["OpenTelemetry<br><br><br>"]
        Req[Incoming Request] -->|"Extract(TraceContext)"| Span1[Root Span]
        Span1 -->|"Inject(W3C Headers)"| Downstream[Downstream RPC]
    end
```

## OpenTelemetry Trace Context Propagation
Distributed tracing relies on propagating metadata across network boundaries.
- **W3C Trace Context Specification**: Standardizes HTTP headers `traceparent` (Trace ID, Span ID, Flags) and `tracestate` (vendor-specific data).
- **Context Implantation**: Language-specific agents dynamically instrument RPC libraries (e.g., gRPC interceptors) to inject/extract context automatically.