# Pilot Analysis

> Use this skill to compute and inspect ACP metrics, telemetry, and export outputs for intervention and baseline cycles without overclaiming.

- Skill: `jasontang-ai/pilot-analysis` (Agent Skill, multi-file: 7 files)
- Install (CLI): `npx skillmds@latest add jasontang-ai/pilot-analysis`
- Raw SKILL.md: https://api.skillmd.com/api/skills/jasontang-ai/pilot-analysis/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: jasontang-ai (https://skillmd.com/u/jasontang-ai)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/jasontang-ai/pilot-analysis

---


# What this skill is for

Use this skill when a pilot, benchmark, or rehearsal needs condition-comparable analysis rather than raw cycle inspection.

# When to use it

- Summarizing one or more cycles.
- Comparing intervention and baseline conditions.
- Preparing pilot evidence for operator or reviewer handoff.

# When not to use it

- Do not use it to manage lifecycle transitions.
- Do not use it to author participant content.
- Do not convert prototype metrics into field-efficacy claims.

# Inputs expected

- One or more cycle ids or cycle JSON files
- Condition labels
- Optional export mode
- Claim boundary for the analysis

# Preflight / prerequisites

1. Confirm cycles are comparable.
2. Confirm exports and metrics are generated by Relay.
3. Separate benchmark evidence, pilot evidence, and field-efficacy claims.

# Workflow

1. Run one-cycle metric summaries.
2. Run condition comparison when at least two cycles exist.
3. Inspect missing metrics and telemetry gaps.
4. Summarize only what the evidence supports.
5. Call out non-claims explicitly.

# Decision rules / judgment criteria

- Comparable denominators matter more than impressive numbers.
- Missing telemetry is a gap.
- Synthetic benchmark evidence supports operational feasibility, not field efficacy.
- Baseline and intervention condition counts should remain aligned where the scenario requires it.

# Escalation rules

Escalate when intervention and baseline differ in participant or contribution count unexpectedly, when key metrics are missing, or when a requested claim exceeds the evidence class.

# Available scripts

- `scripts/metrics-summary.mjs <cycle-id>`
- `scripts/compare-cycles.mjs <cycle-id> <cycle-id> [...]`

# Outputs

- Metrics summary
- Condition comparison
- Explicit evidence gaps
- Claim-safe analysis notes

# Failure handling

If telemetry is incomplete, report incompleteness and stop short of numeric inference.

# Trust / safety notes

Pilot analysis is vulnerable to overclaiming. Treat every output as a bounded evidence artifact.

# Composition notes

Use inside `pilot-cycle-review` and `foresight-submission-prep`.

# Examples to inspect next

Read `examples/casebook.md` for condition comparison and overclaiming anti-patterns.

# Evaluation hooks

Run `scripts/metrics-summary.mjs`, `scripts/compare-cycles.mjs`, and `npm run skills:audit`.