telemetry-inspect
One-shot health check for OrchestKit's telemetry pipeline. Reports writer activity, file sizes, schema lock coverage, orphan files, and growth warnings. Use when verifying the pipeline is flowing correctly or debugging a missing writer.
When to use
- Before or after a risky hook refactor, to prove telemetry still writes as expected
- Weekly health check on a long-running project
- When
analytics output looks suspicious — inspect the underlying data first
- When adding a new telemetry file and wanting to confirm it's picked up
- Auditing which files are schema-locked vs. drift-vulnerable
What it checks
- Writer activity — for each registered telemetry file, recent write count (from mtime scan) and last-write delta
- File health — size (warn at 256 KB, critical at 1 MB), line count, mtime
- Schema lock status — which files have validators in
lib/telemetry-schemas.ts
- Orphan detection — files on disk under
.claude/{telemetry,logs,state,feedback}/ that aren't in the registry (possible stale writer or new file needing schema), plus inventory rows whose writer hook no longer exists in src/hooks/src/ (dead writer → orphan)
- Growth trend — bytes per hour since session start (fire alert if > 100 KB/hr)
- Coordination layer (M168) — live counts from
sessions.db (running sessions, held locks, pending worktree links, skill invocations) plus write throughput from coordination-metrics.jsonl
- Runtime fired-census — which hooks actually fired, from the per-invocation records
run-hook.mjs writes to ~/.claude/analytics/hook-timing.jsonl. Checks 1–4 answer "is the file being written?"; this answers "is the writer running at all?" — the upstream question. A hook can be wired and reachable (the closure gate proves that statically) and still never fire: #2886 shipped exactly that with 12 green tests.
Usage
telemetry-inspect
telemetry-inspect --session sess-abc123
telemetry-inspect --json
Default mode: terminal-friendly ASCII report. --json emits a structured result suitable for piping into another tool or uploading.
Output shape (ASCII mode)
Telemetry Health — 2026-07-09 11:50
────────────────────────────────────
Schema-locked files (7)
.claude/telemetry/pre-compact-decisions.jsonl ◆ 3 lines 1.1 KB ✓ healthy
.claude/telemetry/image-responses.jsonl ◆ 0 lines — ✗ no writes
.claude/logs/decisions.jsonl ◆ 0 lines — ✗ no writes
.claude/logs/subagent-spawns.jsonl ◆ 6 lines 3 KB ✓ healthy
.claude/state/edit-history.jsonl ◆ 94 lines 412 KB ⚠ rotate
.claude/state/ork-metrics-*.json ◆ (N/A) 2.1 KB ✓ healthy
.claude/logs/skill-channels.jsonl ◆ 12 lines 4 KB ✓ healthy
Unlocked telemetry files (14)
.claude/feedback/changelog-decisions.json ○ 4 KB ✗ no schema
.claude/feedback/learned-patterns.json ○ 8 KB ✗ no schema
(...14 more...)
Orphan files (1)
.claude/feedback/skill-usage.json — delisted, writer unwired since #959
Runtime fired-census (7d window)
ALIVE 59 IDLE 3 NEVER 83 UNOBSERVABLE 54
tree 8.75.0 · installed 8.73.0/8.74.0 <-- SKEW: NEVER is not a verdict
never-fired writers (the ones that matter here):
posttool/dirty-file-tracker [PostToolUse] 0 fires, ever
Summary
Pipeline health: GREEN (21/21 expected writers active)
Schema coverage: 7/21 (33%)
Largest file: edit-history.jsonl (412 KB)
Hotspot: edit-history.jsonl +40 KB/hr
Implementation plan (for an agent/LLM running this skill)
- List known files — read
lib/telemetry-schemas.ts's SCHEMA_LOCKED inventory for the 7 locked paths. Extend with the unlocked paths listed in the skill-local references/telemetry-inventory.md.
- Cross-check inventory writers against the registry closure — never trust the inventory blindly:
- Run
node src/hooks/scripts/validate-registry.mjs (from the OrchestKit repo root; skip gracefully if not in the OrchestKit repo) and confirm it passes.
- For every writer named in the inventory (e.g.
posttool/metrics-bridge), verify the source file exists: test -f src/hooks/src/<writer>.ts.
- Any inventory writer that is NOT a live
src/hooks/src/ file is ORPHANED — report 🔴, never "expected-but-empty". Its file(s) on disk are orphans regardless of freshness, and the inventory row is stale (flag it for removal).
- For each file:
- Use
Glob to resolve .claude/state/ork-metrics-*.json pattern → may be multiple
- Use
Read with limit: 10 to see shape and Bash wc -l for line count
- Use
Bash stat for mtime + size
- Classify health:
- size > 1 MB → critical
- size > 256 KB → warn
- mtime > 7 days → "no recent writes"
- line count 0 → "no writes"
- Orphan scan —
Bash find .claude/{telemetry,logs,state,feedback} -type f cross-check against registered paths. Any on-disk files not in inventory → orphan. Merge in dead-writer orphans from step 2.
- Runtime fired-census — steps 1–5 ask "is the file being written?". This asks the upstream question: "does the writer hook run at all?"
- Run
node src/hooks/scripts/fired-census.mjs --json (OrchestKit repo only; skip gracefully elsewhere — it exits 0 and prints an absent-census notice if ~/.claude/analytics/hook-timing.jsonl doesn't exist yet).
- Report the class counts (
totals), and the versions.skewed line verbatim when true — the census is written by the INSTALLED plugin across ALL projects, while the closure comes from this tree, so NEVER on a hook newer than the installed build is expected, not a finding.
- Cross-reference: for every inventory writer from step 2 that is a live file, look up its census row. A writer that is
ALIVE but whose file has 0 lines is a real defect (it runs and produces nothing). A writer that is NEVER is the #2886 class — surface it, but as advisory.
- Never call a hook dead from
NEVER alone. UNOBSERVABLE rows (dispatcher children) cannot appear by construction, and rare events (WorktreeRemove, PreCompact) look identical to a broken guard from here. Read the guard before concluding.
- Render report — ASCII table by default, JSON if
--json argument passed.
Core logic is deterministic + read-only. Do NOT write to any telemetry file — this skill is an observer.
Coordination layer (M168 #1915)
The SQLite coordination layer lives outside .claude/, at ~/.local/state/orchestkit/:
| Source |
What it tells you |
sessions.db |
live session / lock / worktree state (SQLite) |
events.jsonl |
coordination event stream (goal_converged, chain_stale, …) |
coordination-metrics.jsonl |
sessions.db write throughput counters (#1915) |
Live counts — the DB file is a standard SQLite database; read it with sqlite3 (read-only SELECTs only):
DB="$HOME/.local/state/orchestkit/sessions.db"
[ -f "$DB" ] || echo "coordination layer idle (no multi-session activity yet)"
sqlite3 "$DB" "SELECT COUNT(*) FROM sessions WHERE status='running'" # live sessions
sqlite3 "$DB" "SELECT COUNT(*) FROM skill_invocation" # skill invocations
Write throughput — coordination-metrics.jsonl is append-only {ts, metric, count} lines emitted async by lib/metrics-emitter.ts on every sessions.db write. Event rate ≈ recent sessions_db_write lines:
M="$HOME/.local/state/orchestkit/coordination-metrics.jsonl"
[ -f "$M" ] && tail -200 "$M" | grep -c '"sessions_db_write"'
Degrade gracefully: if sqlite3 is absent or the DB / metrics file doesn't exist, report "coordination layer idle" — never error. Like the rest of this skill, these are read-only observations.
Upstream OTel metric notes
When inspecting Claude Code's own OTel metrics (downstream of this skill — claude_code.* in your collector):
- CC 2.1.129+:
claude_code.pull_request.count now also counts PRs/MRs filed via MCP tools (e.g., GitHub MCP create_pull_request), not just shell commands run through the Bash tool. Dashboards built before 2.1.129 will see a step-function increase at the cutover, annotate, don't alert. See ../monitoring-observability/references/metrics-collection.md for the join pattern that distinguishes MCP- from shell-filed PRs.
- CC 2.1.161+:
OTEL_RESOURCE_ATTRIBUTES values are now attached as labels on all metric datapoints, enabling dimensional slicing (team, repo, environment). Existing dashboards keep working; new dashboards should use label selectors to segment usage.
- CC 2.1.145+:
claude_code.tool OTEL spans carry agent_id + parent_agent_id, and background subagent spans nest under the dispatching Agent tool span. Build the trace tree by querying on parent_agent_id — enables per-skill fan-out timing and cost attribution for multi-agent skills (brainstorm, explore, implement); no schema change needed.
- CC 2.1.174+:
/usage exposes CC-native per-component attribution — cache misses, long context, subagents, and per-skill/agent/plugin/MCP cost breakdowns over 24h/7d (surfaced first in the VSCode Account & usage dialog). Treat it as a cross-check source in the health report: if ork telemetry shows a skill/agent active but CC attribution shows zero usage for it (or vice versa), flag the divergence as a possible missing writer or stale install rather than trusting either side alone.
- CC 2.1.202+: telemetry from workflow-spawned agents (a
/workflows run) carries workflow.run_id + workflow.name attributes, so a whole workflow run is reconstructable from OTel — group all agent events on workflow.run_id to rebuild one run's fan-out, and slice by workflow.name. See ../analytics/references/otel-fields.md ("From 2.1.202") for the field table; filter select(.["workflow.run_id"] != null) first, since ordinary session events lack these attributes.
Related
lib/telemetry-schemas.ts — source of truth for schema-locked paths
analytics — aggregates data across sessions (different use case)
- M121 "Observability Consolidation" milestone
1---2name: telemetry-inspect3license: MIT4compatibility: "Claude Code 2.1.277+."5description: "Read-only check of this project's OrchestKit telemetry files: write counts, sizes, schema locks, growth and orphans. Use when verifying the observability pipeline or hunting a missing writer."6argument-hint: "[--session <id>] [--json]"7context: inherit8user-invocable: true9allowed-tools: "Bash Read Grep Glob"10disallowed-tools: [Write, Edit, MultiEdit, NotebookEdit]11effort: low12model: haiku13metadata:14 source: https://skillmd.com/skills/yonatangross/telemetry-inspect15 category: diagnostic16 version: "1.0.0"17 author: "OrchestKit"18 complexity: "low"19 tags: "telemetry, observability, diagnostics, metrics, session, health, schema, inspection"20---2122# telemetry-inspect2324One-shot health check for OrchestKit's telemetry pipeline. Reports writer activity, file sizes, schema lock coverage, orphan files, and growth warnings. Use when verifying the pipeline is flowing correctly or debugging a missing writer.2526## When to use2728- Before or after a risky hook refactor, to prove telemetry still writes as expected29- Weekly health check on a long-running project30- When `analytics` output looks suspicious — inspect the underlying data first31- When adding a new telemetry file and wanting to confirm it's picked up32- Auditing which files are schema-locked vs. drift-vulnerable3334## What it checks35361. **Writer activity** — for each registered telemetry file, recent write count (from mtime scan) and last-write delta372. **File health** — size (warn at 256 KB, critical at 1 MB), line count, mtime383. **Schema lock status** — which files have validators in `lib/telemetry-schemas.ts`394. **Orphan detection** — files on disk under `.claude/{telemetry,logs,state,feedback}/` that aren't in the registry (possible stale writer or new file needing schema), plus inventory rows whose writer hook no longer exists in `src/hooks/src/` (dead writer → orphan)405. **Growth trend** — bytes per hour since session start (fire alert if > 100 KB/hr)416. **Coordination layer (M168)** — live counts from `sessions.db` (running sessions, held locks, pending worktree links, skill invocations) plus write throughput from `coordination-metrics.jsonl`427. **Runtime fired-census** — which hooks actually *fired*, from the per-invocation records `run-hook.mjs` writes to `~/.claude/analytics/hook-timing.jsonl`. Checks 1–4 answer "is the file being written?"; this answers "is the *writer* running at all?" — the upstream question. A hook can be wired and reachable (the closure gate proves that statically) and still never fire: #2886 shipped exactly that with 12 green tests.4344## Usage4546```bash47telemetry-inspect48telemetry-inspect --session sess-abc12349telemetry-inspect --json50```5152Default mode: terminal-friendly ASCII report. `--json` emits a structured result suitable for piping into another tool or uploading.5354## Output shape (ASCII mode)5556```57Telemetry Health — 2026-07-09 11:5058────────────────────────────────────5960Schema-locked files (7)61 .claude/telemetry/pre-compact-decisions.jsonl ◆ 3 lines 1.1 KB ✓ healthy62 .claude/telemetry/image-responses.jsonl ◆ 0 lines — ✗ no writes63 .claude/logs/decisions.jsonl ◆ 0 lines — ✗ no writes64 .claude/logs/subagent-spawns.jsonl ◆ 6 lines 3 KB ✓ healthy65 .claude/state/edit-history.jsonl ◆ 94 lines 412 KB ⚠ rotate66 .claude/state/ork-metrics-*.json ◆ (N/A) 2.1 KB ✓ healthy67 .claude/logs/skill-channels.jsonl ◆ 12 lines 4 KB ✓ healthy6869Unlocked telemetry files (14)70 .claude/feedback/changelog-decisions.json ○ 4 KB ✗ no schema71 .claude/feedback/learned-patterns.json ○ 8 KB ✗ no schema72 (...14 more...)7374Orphan files (1)75 .claude/feedback/skill-usage.json — delisted, writer unwired since #9597677Runtime fired-census (7d window)78 ALIVE 59 IDLE 3 NEVER 83 UNOBSERVABLE 5479 tree 8.75.0 · installed 8.73.0/8.74.0 <-- SKEW: NEVER is not a verdict80 never-fired writers (the ones that matter here):81 posttool/dirty-file-tracker [PostToolUse] 0 fires, ever8283Summary84 Pipeline health: GREEN (21/21 expected writers active)85 Schema coverage: 7/21 (33%)86 Largest file: edit-history.jsonl (412 KB)87 Hotspot: edit-history.jsonl +40 KB/hr88```8990## Implementation plan (for an agent/LLM running this skill)91921. **List known files** — read `lib/telemetry-schemas.ts`'s `SCHEMA_LOCKED` inventory for the 7 locked paths. Extend with the unlocked paths listed in the skill-local `references/telemetry-inventory.md`.932. **Cross-check inventory writers against the registry closure** — never trust the inventory blindly:94 - Run `node src/hooks/scripts/validate-registry.mjs` (from the OrchestKit repo root; skip gracefully if not in the OrchestKit repo) and confirm it passes.95 - For every writer named in the inventory (e.g. `posttool/metrics-bridge`), verify the source file exists: `test -f src/hooks/src/<writer>.ts`.96 - Any inventory writer that is NOT a live `src/hooks/src/` file is **ORPHANED — report 🔴**, never "expected-but-empty". Its file(s) on disk are orphans regardless of freshness, and the inventory row is stale (flag it for removal).973. **For each file**:98 - Use `Glob` to resolve `.claude/state/ork-metrics-*.json` pattern → may be multiple99 - Use `Read` with `limit: 10` to see shape and `Bash wc -l` for line count100 - Use `Bash stat` for mtime + size1014. **Classify health**:102 - size > 1 MB → critical103 - size > 256 KB → warn104 - mtime > 7 days → "no recent writes"105 - line count 0 → "no writes"1065. **Orphan scan** — `Bash find .claude/{telemetry,logs,state,feedback} -type f` cross-check against registered paths. Any on-disk files not in inventory → orphan. Merge in dead-writer orphans from step 2.1076. **Runtime fired-census** — steps 1–5 ask "is the file being written?". This asks the upstream question: "does the writer hook run at all?"108 - Run `node src/hooks/scripts/fired-census.mjs --json` (OrchestKit repo only; skip gracefully elsewhere — it exits 0 and prints an absent-census notice if `~/.claude/analytics/hook-timing.jsonl` doesn't exist yet).109 - Report the class counts (`totals`), and the `versions.skewed` line **verbatim when true** — the census is written by the INSTALLED plugin across ALL projects, while the closure comes from this tree, so `NEVER` on a hook newer than the installed build is expected, not a finding.110 - Cross-reference: for every inventory writer from step 2 that is a live file, look up its census row. A writer that is `ALIVE` but whose file has 0 lines is a **real defect** (it runs and produces nothing). A writer that is `NEVER` is the #2886 class — surface it, but as advisory.111 - **Never** call a hook dead from `NEVER` alone. `UNOBSERVABLE` rows (dispatcher children) *cannot* appear by construction, and rare events (`WorktreeRemove`, `PreCompact`) look identical to a broken guard from here. Read the guard before concluding.1127. **Render report** — ASCII table by default, JSON if `--json` argument passed.113114Core logic is deterministic + read-only. Do NOT write to any telemetry file — this skill is an observer.115116## Coordination layer (M168 #1915)117118The SQLite coordination layer lives **outside `.claude/`**, at `~/.local/state/orchestkit/`:119120| Source | What it tells you |121|--------|-------------------|122| `sessions.db` | live session / lock / worktree state (SQLite) |123| `events.jsonl` | coordination event stream (goal_converged, chain_stale, …) |124| `coordination-metrics.jsonl` | `sessions.db` write throughput counters (#1915) |125126**Live counts** — the DB file is a standard SQLite database; read it with `sqlite3` (read-only SELECTs only):127128```bash129DB="$HOME/.local/state/orchestkit/sessions.db"130[ -f "$DB" ] || echo "coordination layer idle (no multi-session activity yet)"131sqlite3 "$DB" "SELECT COUNT(*) FROM sessions WHERE status='running'" # live sessions132sqlite3 "$DB" "SELECT COUNT(*) FROM skill_invocation" # skill invocations133```134135**Write throughput** — `coordination-metrics.jsonl` is append-only `{ts, metric, count}` lines emitted async by `lib/metrics-emitter.ts` on every `sessions.db` write. Event rate ≈ recent `sessions_db_write` lines:136137```bash138M="$HOME/.local/state/orchestkit/coordination-metrics.jsonl"139[ -f "$M" ] && tail -200 "$M" | grep -c '"sessions_db_write"'140```141142Degrade gracefully: if `sqlite3` is absent or the DB / metrics file doesn't exist, report **"coordination layer idle"** — never error. Like the rest of this skill, these are **read-only** observations.143144## Upstream OTel metric notes145146When inspecting Claude Code's own OTel metrics (downstream of this skill — `claude_code.*` in your collector):147148- **CC 2.1.129+**: `claude_code.pull_request.count` now also counts PRs/MRs filed via MCP tools (e.g., GitHub MCP `create_pull_request`), not just shell commands run through the Bash tool. Dashboards built before 2.1.129 will see a step-function increase at the cutover, annotate, don't alert. See `../monitoring-observability/references/metrics-collection.md` for the join pattern that distinguishes MCP- from shell-filed PRs.149- **CC 2.1.161+**: `OTEL_RESOURCE_ATTRIBUTES` values are now attached as labels on all metric datapoints, enabling dimensional slicing (team, repo, environment). Existing dashboards keep working; new dashboards should use label selectors to segment usage.150- **CC 2.1.145+**: `claude_code.tool` OTEL spans carry `agent_id` + `parent_agent_id`, and background subagent spans nest under the dispatching Agent tool span. Build the trace tree by querying on `parent_agent_id` — enables per-skill fan-out timing and cost attribution for multi-agent skills (`brainstorm`, `explore`, `implement`); no schema change needed.151- **CC 2.1.174+**: `/usage` exposes CC-native per-component attribution — cache misses, long context, subagents, and per-skill/agent/plugin/MCP cost breakdowns over 24h/7d (surfaced first in the VSCode Account & usage dialog). Treat it as a cross-check source in the health report: if ork telemetry shows a skill/agent active but CC attribution shows zero usage for it (or vice versa), flag the divergence as a possible missing writer or stale install rather than trusting either side alone.152- **CC 2.1.202+**: telemetry from workflow-spawned agents (a `/workflows` run) carries `workflow.run_id` + `workflow.name` attributes, so a whole workflow run is reconstructable from OTel — group all agent events on `workflow.run_id` to rebuild one run's fan-out, and slice by `workflow.name`. See `../analytics/references/otel-fields.md` ("From 2.1.202") for the field table; filter `select(.["workflow.run_id"] != null)` first, since ordinary session events lack these attributes.153154## Related155156- `lib/telemetry-schemas.ts` — source of truth for schema-locked paths157- `analytics` — aggregates data across sessions (different use case)158- M121 "Observability Consolidation" milestone