jasontang-ai What this skill is for
Use this skill when a pilot, benchmark, or rehearsal needs condition-comparable analysis rather than raw cycle inspection.
When to use it
- Summarizing one or more cycles.
- Comparing intervention and baseline conditions.
- Preparing pilot evidence for operator or reviewer handoff.
When not to use it
- Do not use it to manage lifecycle transitions.
- Do not use it to author participant content.
- Do not convert prototype metrics into field-efficacy claims.
Inputs expected
- One or more cycle ids or cycle JSON files
- Condition labels
- Optional export mode
- Claim boundary for the analysis
Preflight / prerequisites
- Confirm cycles are comparable.
- Confirm exports and metrics are generated by Relay.
- Separate benchmark evidence, pilot evidence, and field-efficacy claims.
Workflow
- Run one-cycle metric summaries.
- Run condition comparison when at least two cycles exist.
- Inspect missing metrics and telemetry gaps.
- Summarize only what the evidence supports.
- Call out non-claims explicitly.
Decision rules / judgment criteria
- Comparable denominators matter more than impressive numbers.
- Missing telemetry is a gap.
- Synthetic benchmark evidence supports operational feasibility, not field efficacy.
- Baseline and intervention condition counts should remain aligned where the scenario requires it.
Escalation rules
Escalate when intervention and baseline differ in participant or contribution count unexpectedly, when key metrics are missing, or when a requested claim exceeds the evidence class.
Available scripts
scripts/metrics-summary.mjs <cycle-id>scripts/compare-cycles.mjs <cycle-id> <cycle-id> [...]
Outputs
- Metrics summary
- Condition comparison
- Explicit evidence gaps
- Claim-safe analysis notes
Failure handling
If telemetry is incomplete, report incompleteness and stop short of numeric inference.
Trust / safety notes
Pilot analysis is vulnerable to overclaiming. Treat every output as a bounded evidence artifact.
Composition notes
Use inside pilot-cycle-review and foresight-submission-prep.
Examples to inspect next
Read examples/casebook.md for condition comparison and overclaiming anti-patterns.
Evaluation hooks
Run scripts/metrics-summary.mjs, scripts/compare-cycles.mjs, and npm run skills:audit.