# Moos Ivp Eval Mission Builder > Build or repair one self-evaluating MOOS-IvP mission by adding an evaluation layer to an ordinary mission: headless startup, pMissionEval grading, results.txt, uMayFinish/xlaunch completion, zlaunch automation, scoped teardown, and single-scenario validation. - Skill: `cbenjamin23/moos-ivp-eval-mission-builder` (Agent Skill, multi-file: 21 files) - Install (CLI): `npx skillmds@latest add cbenjamin23/moos-ivp-eval-mission-builder` - Raw SKILL.md: https://api.skillmd.com/api/skills/cbenjamin23/moos-ivp-eval-mission-builder/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Coding & Dev Tools - Author: cbenjamin23 (https://skillmd.com/u/cbenjamin23) - Updated: 2026-09-22 - Page: https://skillmd.com/skills/cbenjamin23/moos-ivp-eval-mission-builder --- # MOOS-IvP Eval Mission Builder ## Overview Use this skill for one self-evaluating mission folder: a normal MOOS-IvP mission with an added single-run grading contract. The mission should still be readable and runnable by a person, but it must also run headlessly, decide pass/fail inside the mission, write `results.txt`, and finish through the shared `xlaunch.sh` / `uMayFinish` path. For ordinary mission layout, use `moos-ivp-mission-builder` first. For multi-case matrices, patch sweeps, parallel runs, or expected-vs-actual aggregation, use `moos-ivp-harness-builder`. For post-run `.alog` evidence, use `moos-alog-analysis`. ## Core Rules - Start from an ordinary mission that already launches cleanly. Prefer the `moos-ivp-mission-builder` baselines or an existing nearby mission family. - Add only the evaluation plumbing needed for one scenario: optional `pAutoPoke`, optional `uTimerScript`, `pMissionEval`, `results.txt`, and a thin `zlaunch.sh`. - Keep `launch.sh` human-facing. It may accept `--xlaunched`, `--nogui`, and port overrides, but it should not contain case loops or result aggregation. - Keep `zlaunch.sh` thin: parse automation arguments, truncate `results.txt`, call shared `xlaunch.sh`, validate that `results.txt` contains `grade=`, then apply project-local scoped cleanup. - Let `xlaunch.sh --max_time=` own `uMayFinish` and the timed wait/stop contract. Do not duplicate that lifecycle in mission-local wrappers. - Do not synthesize `grade=` or write the final result row from `zlaunch.sh`, `launch.sh`, or target-file parsing. `pMissionEval` must own the verdict and write `results.txt`; wrappers may only truncate, launch, wait, validate presence of `grade=`, and clean up. - For cleanup backstops, copy `assets/moos_scoped_teardown.sh` into the target project as `/scripts/moos_scoped_teardown.sh` if it does not already exist. Reuse an existing project-root helper unless it is clearly stale or incompatible. - Prefer `pAutoPoke` to seed deploy and evaluation variables in moving missions. Unit-style evals may use `uTimerScript` or the app under test for readiness when there is no vehicle/deploy lifecycle. Do not put pass/fail logic in `pAutoPoke`. - Use `pMissionEval` as the primary verdict owner. Prefer mission-level booleans or simple scalar checks over harness-side parsing of raw MOOS traffic. - Prefer event-driven `pMissionEval` leads: evaluate when the mission-owned completion event occurs. Use `uMayFinish` through `xlaunch.sh --max_time` as the outer infrastructure ceiling. Use a time-driven evaluation-window lead only when non-completion is an expected mission outcome that should produce mission-owned `grade=fail`. - Multiple `lead_condition` lines in the same aspect are allowed, but they are ANDed: all must be true before pass/fail conditions are evaluated. A `lead_condition` after pass/fail conditions starts the next ordered aspect. A single `lead_condition` may use textual `or` when each operand is parenthesized, for example `(EVENT_A = true) or (EVENT_B = true)`. Do not use `||`; it is not a supported `LogicCondition` operator. - Treat `BHV_ERROR_SEEN=false` as a normal safety/integrity pass condition. Treat `BHV_WARNING` as advisory development evidence by default: inspect and investigate it with appcasts or `.alog` tools, but do not add a sticky `BHV_WARNING_SEEN` mailflag, result column, or pass condition unless the scenario is explicitly warning-intolerant and the warning signal is known to be stable rather than transient/retracted. - Keep `results.txt` scalar and parseable. The only hard schema requirement is `grade=`; fields such as `form=`, `eval=`, `timeout=`, domain facts, and `mhash=` are recommended evidence, not a mandatory metric set. - `mission_mod` is optional mission-owned provenance. Use it only when one mission folder intentionally supports multiple named standalone modes. Omit it from single-scenario eval missions, and do not use it to represent harness cases. - If a vehicle-local variable is graded shoreside, bridge it explicitly through the vehicle broker and shoreside broker. - For GUI-capable eval missions, keep normal operator buttons available. Do not force appcast/realmcast viewer modes unless the evaluation scenario needs it. - Do not add `--case`, `--jobs`, temp mission copies, per-case port blocks, or expected-vs-actual aggregation here. Those belong to the harness builder. ## Workflow 1. Confirm the base mission launches and generates targets. 2. Identify the smallest mission-owned pass/fail signal. - unit-style app variable - behavior end flag - arrival/collision/encounter outcome - load/process/host info signal 3. Add evaluation state to the relevant `.bhv` or app config. - When adapting an ordinary waypoint mission, make the graded behavior finite, such as `repeat = 0`, or add an explicit completion flag. A repeating operator survey is usually not a valid eval completion signal. 4. Bridge graded vehicle-local variables to shoreside when needed. 5. Add `pAutoPoke` or an equivalent explicit initializer for deploy and evaluation variables. 6. Add `pMissionEval` with simple lead condition(s), clear pass conditions, `result_flag = MISSION_EVALUATED = true`, and `report_file = results.txt`. 7. Add or update `zlaunch.sh` to set a mission-appropriate `MAX_TIME` default, accept `--max_time=` as an override, and forward the final value to `xlaunch.sh --max_time=`. 8. Add or update `README.md` with scenario, grading signal, and run commands. 9. Validate target generation, then run the headless cycle and inspect `results.txt`. ## Reference Use - Read `references/eval-mission-style.md` for boundaries and file layout. - Read `references/evaluator-apps.md` before wiring `pAutoPoke` or `pMissionEval`. - Read `references/scenario-and-grading.md` before grading obstacles, contacts, moving/integration outcomes, or structured payloads. - Read `references/zlaunch-xlaunch.md` before editing automation wrappers. - Read `references/validation.md` before reporting an eval mission as done. - Copy `assets/eval-single-vehicle/` when a concrete minimal moving example is useful. - Copy `assets/moos_scoped_teardown.sh` into the target project as `/scripts/moos_scoped_teardown.sh` when the project does not already have an equivalent root-scoped helper. - Run `scripts/static_check_eval_mission.sh ` for a quick structural check. - Run `scripts/live_check_eval_mission.sh --port_base=` for bundled-example or high-trust validation when MOOS-IvP runtime tools are available. - Treat live-check teardown failure as a test failure, show the teardown error, and preserve the temporary workdir for diagnosis. ## Validation Checklist - `./launch.sh --just_make --nogui ` succeeds. - Generated targets contain `pMissionEval`, explicit initialization (`pAutoPoke`, `uTimerScript`, or an app-owned producer), and any evaluator apps needed for reported columns such as `pMissionHash`. - If `pMissionHash` is used for `mhash=` evidence, keep it headless-only by default; GUI targets should not launch both `pMissionHash` and `pMarineViewer` unless the overlapping pMarineViewer hash feature is deliberately disabled. - Generated targets include bridged graded variables if the verdict depends on vehicle-local posts. - `./zlaunch.sh --just_make ` succeeds when `xlaunch.sh` is on `PATH`. - Headless `./zlaunch.sh --max_time= ` exits cleanly. - `results.txt` contains one parseable result line with `grade=`. - Runtime warnings are investigated during validation; only stable, scenario-relevant warning metrics are surfaced in `results.txt`. - High-trust checks use `scripts/live_check_eval_mission.sh` or equivalent to verify result rows, surface warning evidence, and detect leftover listeners on scoped ports. - No mission wrapper uses global `ktm`, `pkill`, or unrelated cleanup. - Eval wrappers use `/scripts/moos_scoped_teardown.sh` as a scoped backstop after `xlaunch.sh`; they do not use global `ktm`, `pkill`, or broad process discovery. - GUI runs retain normal operator controls unless the user requested a headless-only mission.