Measure Telemetry Span

Measure Telemetry Span

Pattern: from repo root, run one command with a span name and platform → stdout is a small summary table (avg / min / max + sample ms list). The script measures whatever Git checkout is currently active: it never runs git checkout or otherwise switches branches. To compare this branch with main (or any other revision), check out each commit/branch in turn—or use two worktrees/clones—and run measure.sh separately, then compare the two printed summaries.

Command

.claude/skills/measure-telemetry-span/measure.sh <span-name> [runs] [platform] [--boot]

To pick a specific Android AVD or iOS simulator, use the same global flags agent-device already supports (for example --device "Pixel_7_API_34") on boot and on later commands — either run agent-device boot --platform android --device "…" yourself before measure.sh, or rely on agent-device config (~/.agent-device/config.json). --boot inside this script only passes --platform through to boot.

Environment: APP_ID overrides app bundle (default com.expensify.chat.dev). If the flow declares # @param KEY …, set AD_KEY to pass -e KEY=VALUE to replay.

Output: table + Samples: …ms line; stderr has progress (Using flow:, runs, optional reset).

Before you run

If you see no Android device (adb devices empty): append --boot to the measure command, or run manually first:

agent-device boot --platform android (optional --device "<AVD name>"). For iOS, agent-device boot --platform ios or agent-device ensure-simulator --boot when you need a created simulator instance.

Contract

• App logs: [Sentry][<SpanName>] Ending span (<N>ms) via console.debug.
• Flow file includes # @tag sentry-<SpanName> (same name, case-sensitive).
• Optional flow headers: @reset <path.ad> (run by the script after warmup and each measured replay; if absent, the script relaunches the app instead so each run starts from @pre); @param keys overridable via AD_* (passed as -e KEY=VALUE to replay).
• Parsing: stats take the last RUNS matching log lines from the capture. That matches one sample per measured replay only if each replay emits one such line for this span name. Extra matches (duplicate logs, nested/sub-spans with the same message pattern, noisy startup logging) can shift which samples are included—fix the app logging or tighten the grep if that happens.

@reset and loop stability

measure.sh replays the same tagged flow every iteration. Treat @reset as “return to a known anchor,” not a second copy of the whole scenario:

• Prefer a short reset flow (tabs to Inbox, dismiss sheet, etc.). When agent-device splits macros vs tests, point @reset at a macro path so one file stays the source of truth.
• If runs are flaky locally but fine for others, walk the bring-up checklist in .claude/skills/agent-device/SKILL.md (Metro, dev build, device boot, iOS + DevTools for console.debug) before blaming selectors.

Optional: keep a tiny markdown table in your team notes mapping SpanName → one-line intent + @pre anchor; the span name still drives which .ad is picked — no need to repeat long repro prose in every chat.

If something fails

For another app bundle, export APP_ID before the command.