# Trial Run

> Plan a scored ERC-26 trial run (procedure, pass criteria, pre-run checklist) or log a completed one with the real point breakdown. Use before running a scored trial, or right after one.

- Skill: `faaz17/trial-run` (Agent Skill)
- Install (CLI): `npx skillmds@latest add faaz17/trial-run`
- Raw SKILL.md: https://api.skillmd.com/api/skills/faaz17/trial-run/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: Faaz17 (https://skillmd.com/u/faaz17)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/faaz17/trial-run

---


# /trial-run, trial planner and logger

Two modes. Ask which applies if it isn't obvious from the conversation.

A trial run is the only thing that produces a real score. Impressions of how a run looked are not data, and the report's results section needs the numbers. See the test rungs in `.claude/rules/workflow.md`.

## The discipline this skill exists to enforce

**One run proves almost nothing.** The marker digits, the colour-to-row mapping, the book's horizontal position and the robot's starting orientation all re-randomise on every world load. A single successful run may only mean the seed was favourable.

So every plan states a reload count, and every log states which seed conditions applied.

## Mode 1: plan, before the run

1. **Ask what's being tested and what would count as passing.** A test without a pre-declared numeric criterion is a demo, not a test.

2. Write the procedure:
   - Which subsystem or which full-run stage
   - **How many world reloads.** 20 for anything perception-related, 10 for grasp attempts. Not 3
   - Which requested column and colour on each run, varied across the set
   - What gets measured and how it's recorded
   - What gets changed between runs, one variable at a time

3. State the pass criterion numerically. Starting points, from `.claude/agents/qa.md`:
   - Column detection: *correct digit on 20 of 20 world reloads*
   - Book detection: *correct colour and row on 20 of 20 reloads, all four colours and all four rows represented*
   - Annotated images: *present, timestamped, correctly boxed, in 20 of 20 runs*
   - Navigation: *reaches the standoff pose 20 of 20, zero collisions*
   - Grasp: *book lifted clear in 8 of 10 attempts, broken out by row*
   - Gentle place: *released under control in 8 of 10*
   - Recovery: *each path deliberately triggered and observed at least once*

4. Pre-run checklist, mandatory, from `.claude/rules/workflow.md`:
   - [ ] `colcon build` clean, workspace sourced
   - [ ] `simulation.launch.py` running first, world fully loaded
   - [ ] Launch with the **exact organiser command**, both arguments passed
   - [ ] `erc_images/` writable and **cleared of stale images**
   - [ ] `results/` writable, `trial_logger` running
   - [ ] Both scored topics confirmed live with `ros2 topic echo`
   - [ ] **World reloaded since the last run**, so this is a new seed
   - [ ] Requested column and colour recorded before the run

5. Note the two traps:
   - **Stale images.** Images produced outside the trial run are explicitly not accepted, and a leftover file from yesterday is indistinguishable from one. Clear the folder.
   - **Repeating a seed.** Running twice without reloading is one test, not two.

## Mode 2: log, after the run

1. **Ask for the real point breakdown**, line by line against the official criteria, not an impression. "13 of 19: both IDs scored, grasp on the second attempt, dropped rather than gently placed, one collision on retreat" is data. "The run went well" is not.

2. Score it explicitly:

   | Item | Points | Scored? |
   |---|---|---|
   | Column publish | +1 | |
   | Column image | +2 | |
   | Navigate to column (needs the grasp) | +3 | |
   | Row publish | +1 | |
   | Row image | +2 | |
   | Grasp | +3 | |
   | Navigate to bin (needs delivery) | +3 | |
   | Place: dropped +2 or gentle +4 | | |
   | Collisions | -0.5 each | |

   Remember the two contingencies: navigation to the column scores nothing unless the grasp succeeded, and navigation to the bin scores nothing unless the book was delivered. A run that drove perfectly and failed the grasp scores 6, not 9.

3. Append the row to `results/trials.csv`. Schema in `docs/BLUEPRINT.md` §7. **Never skip a bad run.** Failed trials are the useful data and section 6 of the report is graded on them.

4. Append to `docs/BUILD_LOG.md` using the standard format, with the rung reached and the actual score.

5. Diagnose failures against the table in `.claude/agents/qa.md`, and record which hypothesis you're acting on.

6. Record parameter changes: old value, new value, observed effect. If `config/params.yaml` changed, note which keys and why.

7. Update `.claude/PROJECT_NOTES.md` only if a build-order gate was genuinely met. A 17 of 20 pass rate does not close a gate that requires 20 of 20.

## The five report trials are a special case

Section 5 of the report requires scores across 5 simulation trials. That set has rules:

- Five consecutive trials, each on a fresh world load
- **No parameter changes between them.** Changing anything mid-set invalidates it as a dataset
- All five logged in full, including bad ones
- Requested column and colour varied across the five

A set with one characterised failure is honest data with a story. A set where the bad runs were quietly re-run is a fabrication, and the limitations section is graded, so the bad run earns more than it costs.

## Rules

- Never round up the test rung. "Ran once" is not "works". See `.claude/rules/workflow.md`.
- One variable at a time. Changing three parameters between runs tells you nothing about which mattered.
- Record failures in full. The failed runs are the useful data; the successful ones just confirm it.
- Break grasp results out by row. Row-dependent failure points at torso positioning; uniform failure points at the grasp pose.
- Don't chase the grasp before the two perception nodes are at 20 of 20. The cheap points come first, per `docs/SCORING.md`.
- Write plainly: no em dashes, no emoji in headings, no title case.