# Telemetry Event Analyst

> Analyze Bench My Harness normalized events, raw hook references, metric observations, token usage, context window usage, tool calls, command execution, outputs, benchmark run artifacts, and harness comparison reports.

- Skill: `vinilana/telemetry-event-analyst` (Agent Skill, multi-file: 3 files)
- Install (CLI): `npx skillmds@latest add vinilana/telemetry-event-analyst`
- Raw SKILL.md: https://api.skillmd.com/api/skills/vinilana/telemetry-event-analyst/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: vinilana (https://skillmd.com/u/vinilana)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/vinilana/telemetry-event-analyst

---


# Telemetry Event Analyst

## Workflow

1. Identify baseline, candidate, benchmark version, harness profiles, and trial counts.
2. Read `references/metrics-policy.md` before interpreting tokens, context, or cost.
3. Verify comparability: same scenario, initial state, model policy, permissions, environment, and adapter capabilities.
4. Separate functional failure from performance regression and telemetry gaps.
5. Report absolute values, deltas, source/confidence, variance, and outliers.
6. Link conclusions to event IDs, artifact paths, or metric observation IDs.

## Rules

- Do not compare native token usage with estimated token usage as if they were equivalent.
- Do not hide inconclusive, failed, or missing-data trials.
- Mark runs as `limited` or `not_comparable` when capabilities differ materially.
- Prefer median and distribution over single-run anecdotes.
- Keep raw evidence available for audit.

## Analysis Outputs

Produce concise findings with:

- result summary;
- comparability status;
- biggest deltas;
- data quality gaps;
- likely causes;
- recommended next measurement or fix.