OpenSpec Test
Generic verification executor for OpenSpec gates. Follows test.md strategy literally — no improvisation.
flowchart LR
A["Read test.md"] --> B["Validate quality"]
B --> C{"Lazy patterns?"}
C -->|Yes| X["BLOCKED"]
C -->|No| D["Execute verification"]
D --> E["Write logs"]
E --> F["Report result"]
classDef action fill:#C8E6C9,stroke:#388E3C,color:#000
classDef decision fill:#FFF9C4,stroke:#FBC02D,color:#000
classDef fail fill:#FFCDD2,stroke:#D32F2F,color:#000
class A,B,D,E,F action
class C decision
class X fail
⚠️ AskUserQuestion Guard
CRITICAL: After EVERY AskUserQuestion call, check if answers are empty/blank. Known Claude Code bug: outside Plan Mode, AskUserQuestion silently returns empty answers without showing UI.
If answers are empty: DO NOT proceed with assumptions. Instead:
- Output: "⚠️ Questions didn't display (known Claude Code bug outside Plan Mode)."
- Present the options as a numbered text list and ask user to reply with their choice number.
- WAIT for user reply before continuing.
checkpoint
Input: $ARGUMENTS = change-id section-number (e.g., add-feature 3)
Workflow:
- Read
openspec/changes/{change-id}/tasks.md→ find## {section-number}. ... - Read
openspec/changes/{change-id}/test.md→ verification strategy. If missing: warn + offer/openspec-plan tasks {change-id}or skip (garage only). - If test.md has a
**Design**:header → readopenspec/changes/{change-id}/design.mdfor invariants, key flows, and ADRs (informs gate evaluation context) - Validate: Scan for lazy patterns (
grep "keyword",[ -f file ],wc -l,echo). If found → BLOCKED with suggestion for functional verification. - Init logs: Create
test-logs/gate-{n}-{yyyyMMddHHmm}.yaml(raw, append-per-step) +test-logs/gate-{n}-{yyyyMMddHHmm}.md(summary). Timestamp = run start time. See reference.md §Log Formats. - Execute by type tag (default
[auto]):
| Tag | Action | Evaluator |
|---|---|---|
[auto] |
Run command, assert exit+output | Agent |
[smoke] |
Run command, display output | Agent + human glance |
[manual] |
Display instructions, AskUserQuestion (PASS/FAIL/SKIP) | Human |
- Order:
[auto]+[smoke]first, group[manual]at end - Per step: Append YAML step block to raw log immediately after execution (not batched)
- Write summary to
gate-{n}-{ts}.md— rendered from raw YAML, not memory. See reference.md §Test Log Format. - Display gate summary inline to user.
- Evaluate:
- PASS → mark
### GATE {n}: desc [PASS]in tasks.md - PARTIAL → report failures with per-step diagnostics
- BLOCKED → report blockers, suggest options
- PASS → mark
Log Strategy
Two artifacts per gate run in openspec/changes/{change-id}/test-logs/:
| File | Format | Purpose |
|---|---|---|
gate-{n}-{yyyyMMddHHmm}.yaml |
YAML | Source of truth — append-per-step, full stdout/stderr (mask secrets) |
gate-{n}-{yyyyMMddHHmm}.md |
Markdown | Human summary — rendered from raw YAML after all steps |
- Timestamped filenames: Each run = new file pair. No overwriting, no array append gymnastics.
- Append-per-step: Raw YAML grows incrementally — each step appended immediately after execution.
- Markdown truncation: stdout >50 lines → first 30 + last 10; stderr >20 lines → first 15
See reference.md §Raw Log Format for YAML schema, §Test Log Format for markdown template.
Mode Behavior
Read openspec/project.md → mode:
- garage: Warn if test.md missing, allow skip
- scale/maintenance: Block until test.md exists + passes
References
See reference.md for: log format templates, YAML schema, test progression strategy, framework-specific patterns, execution tracing, failure diagnostics.
Converted and distributed by TomeVault — claim your Tome and manage your conversions.