runtime-smoke
Lifecycle stage: VERIFY
Trigger
Use when commands, skills, adapters, harness prompts, runtime setup, or command routing changed and the claim depends on a real agent runtime rather than only fixture validation.
When not to use
Do not use when this trigger is absent; choose the command or skill that owns the requested state, artifact, and verification gate. Do not use it for destructive, credentialed, production, paid, or public side-effect runs without explicit approval evidence.
Inputs
- Target runtime or adapter name.
- Clean checkout or disposable workspace path.
- Sandbox or write-mode constraints.
- Baseline validation command and expected scope.
- Exact task prompt or command to run in the runtime.
- Approval state for any write-capable or side-effecting command.
Procedure
- Start from a clean checkout or disposable copy and preserve user changes before any runtime task.
- Record runtime name, version, Python executable, writable temp-dir status, git freshness result, exact command, sandbox/write mode, whether
/brain-* entries are native commands or markdown specs, and blocked commands.
- Run a read-only smoke first unless explicit approval allows full validation.
- Ask the runtime to load Agent Brain, choose the matching
/brain-* command, load only the command-listed skills, and produce the expected artifact.
- Capture evidence in
templates/runtime-smoke.md and, when JSON evidence is produced, validate it against schemas/runtime-smoke.schema.json. Use the checked-in helper path (python scripts/runtime_smoke.py ...) for JSON artifacts so the exact command is helper-validated instead of reconstructed from a transcript. For full_validation, capture a durable transcript path; stdout-only markers are only acceptable for read-only smoke.
- Route any setup, command-boundary, sandbox, or artifact-contract gap to the smallest follow-up hardening slice.
- Stop before claiming full validation if the runtime could only run read-only smoke checks.
Anti-Rationalization
| Shortcut |
Rebuttal |
| "The validator passed, so runtime usability is proven." |
Fixture validation is not real-runtime evidence. Capture a runtime smoke artifact. |
| "The runtime probably supports slash commands." |
Record whether /brain-* entries are native commands or markdown specs. |
| "Read-only output is enough for full validation." |
Mark read-only smoke separately and list blocked commands. |
| "The setup failed, so summarize the intent." |
Capture the blocker, command, sandbox mode, and follow-up fix. |
Verification
- Runtime, version, Python executable, writable temp-dir status, git freshness result, exact command, sandbox/write mode, command mode, blocked commands, run scope, and evidence are recorded.
- The run scope distinguishes read-only smoke from full validation.
- The runtime followed command routing and skill-loading boundaries: any
pass artifact names a selected command, at least one loaded skill, and only loaded skills declared by selected command.
- Any generated JSON evidence validates against
schemas/runtime-smoke.schema.json.
- Public copy remains neutral and does not name private dogfood coordinators or source-specific branding.
Output Artifact
Runtime Smoke
Use templates/runtime-smoke.md for narrative evidence and schemas/runtime-smoke.schema.json for machine-checkable JSON evidence. The artifact must name blockers and the next action when the run is read-only, blocked, stale, or not full validation.
Failure Modes
- Claiming adapter readiness from fixture tests only.
- Omitting sandbox/write mode or blocked commands.
- Treating markdown command specs as native runtime commands without checking.
- Running write-capable or side-effecting tasks without approval evidence.
- Hiding setup failures, stale git state, missing Python dependencies, or unavailable temp-dir writes.
Example
Trigger: adapter README or harness prompt changes. Action: run read-only smoke first, capture command mode, sandbox mode, blocked commands, and transcript evidence. Output artifact: templates/runtime-smoke.md or schema-valid JSON with blockers and next action. Verification: cite runtime, version, Python executable, selected command, loaded skills, adapter path, and transcript redaction status.
A maintainer changes an adapter README. Before shipping, the verifier runs a read-only CLI-runtime smoke against a disposable checkout, records the runtime version, sandbox mode, command mode, blocked commands, and evidence, then files a follow-up hardening slice for any command-routing gap instead of claiming full validation from prose.
1---2name: runtime-smoke3description: Use when checking Agent Brain inside a real agent runtime or adapter before trusting harness usability.4---56# runtime-smoke78Lifecycle stage: VERIFY910## Trigger1112Use when commands, skills, adapters, harness prompts, runtime setup, or command routing changed and the claim depends on a real agent runtime rather than only fixture validation.1314## When not to use1516Do not use when this trigger is absent; choose the command or skill that owns the requested state, artifact, and verification gate. Do not use it for destructive, credentialed, production, paid, or public side-effect runs without explicit approval evidence.1718## Inputs1920- Target runtime or adapter name.21- Clean checkout or disposable workspace path.22- Sandbox or write-mode constraints.23- Baseline validation command and expected scope.24- Exact task prompt or command to run in the runtime.25- Approval state for any write-capable or side-effecting command.2627## Procedure28291. Start from a clean checkout or disposable copy and preserve user changes before any runtime task.302. Record runtime name, version, Python executable, writable temp-dir status, git freshness result, exact command, sandbox/write mode, whether `/brain-*` entries are native commands or markdown specs, and blocked commands.313. Run a read-only smoke first unless explicit approval allows full validation.324. Ask the runtime to load Agent Brain, choose the matching `/brain-*` command, load only the command-listed skills, and produce the expected artifact.335. Capture evidence in `templates/runtime-smoke.md` and, when JSON evidence is produced, validate it against `schemas/runtime-smoke.schema.json`. Use the checked-in helper path (`python scripts/runtime_smoke.py ...`) for JSON artifacts so the exact command is helper-validated instead of reconstructed from a transcript. For `full_validation`, capture a durable transcript path; stdout-only markers are only acceptable for read-only smoke.346. Route any setup, command-boundary, sandbox, or artifact-contract gap to the smallest follow-up hardening slice.357. Stop before claiming full validation if the runtime could only run read-only smoke checks.3637## Anti-Rationalization3839| Shortcut | Rebuttal |40| --- | --- |41| "The validator passed, so runtime usability is proven." | Fixture validation is not real-runtime evidence. Capture a runtime smoke artifact. |42| "The runtime probably supports slash commands." | Record whether `/brain-*` entries are native commands or markdown specs. |43| "Read-only output is enough for full validation." | Mark read-only smoke separately and list blocked commands. |44| "The setup failed, so summarize the intent." | Capture the blocker, command, sandbox mode, and follow-up fix. |4546## Verification4748- Runtime, version, Python executable, writable temp-dir status, git freshness result, exact command, sandbox/write mode, command mode, blocked commands, run scope, and evidence are recorded.49- The run scope distinguishes read-only smoke from full validation.50- The runtime followed command routing and skill-loading boundaries: any `pass` artifact names a selected command, at least one loaded skill, and only loaded skills declared by selected command.51- Any generated JSON evidence validates against `schemas/runtime-smoke.schema.json`.52- Public copy remains neutral and does not name private dogfood coordinators or source-specific branding.5354## Output Artifact5556Runtime Smoke5758Use `templates/runtime-smoke.md` for narrative evidence and `schemas/runtime-smoke.schema.json` for machine-checkable JSON evidence. The artifact must name blockers and the next action when the run is read-only, blocked, stale, or not full validation.5960## Failure Modes6162- Claiming adapter readiness from fixture tests only.63- Omitting sandbox/write mode or blocked commands.64- Treating markdown command specs as native runtime commands without checking.65- Running write-capable or side-effecting tasks without approval evidence.66- Hiding setup failures, stale git state, missing Python dependencies, or unavailable temp-dir writes.6768## Example6970Trigger: adapter README or harness prompt changes. Action: run read-only smoke first, capture command mode, sandbox mode, blocked commands, and transcript evidence. Output artifact: `templates/runtime-smoke.md` or schema-valid JSON with blockers and next action. Verification: cite runtime, version, Python executable, selected command, loaded skills, adapter path, and transcript redaction status.7172A maintainer changes an adapter README. Before shipping, the verifier runs a read-only CLI-runtime smoke against a disposable checkout, records the runtime version, sandbox mode, command mode, blocked commands, and evidence, then files a follow-up hardening slice for any command-routing gap instead of claiming full validation from prose.
rohitg00