Validate Spec — Implementation Audit
Audit an implemented /spec-format spec against the codebase. Walk every tagged
constraint, ground each one in file:line evidence, execute the acceptance
commands, and emit a single audit document. The spec itself is never modified —
the audit doc is the only artifact this skill writes.
This skill is the closing-the-loop partner for /spec. /spec builds the
specification; validate-spec proves whether it actually got built.
Arguments
$0— spec id (the directory name underdocs/specs/). If not provided, list candidate spec ids found underdocs/specs/and ask the user.--no-run— skip Phase 4 (acceptance command execution). Useful for fast structural+evidence audits when tests are slow or require external services.
Workflow Overview
Five phases, each gated. Do not skip ahead. If a phase fails fatally (e.g. spec not found, wrong format), stop with a clear message — partial audits mislead.
The reasoning behind the structure: a spec is a set of claims (constraints, acceptance criteria, key decisions). An audit checks each claim against evidence (code, tests, runtime behavior). If you skip phases, you're guessing, not auditing.
Phase 1 — Locate & Validate Format
Resolve spec id. Use
$0if given. Otherwise list directories underdocs/specs/(search the project root walking up from cwd) and ask the user which one to validate.Locate the spec directory. Look for
docs/specs/<spec-id>/starting from cwd, walking up to find the project root (the directory containingdocs/specs/).Report whether the spec declares
acceptance_criteria. Ifspec.jsonhas noacceptance_criteriaarray, record an INFO finding: the field is optional in the schema, so a spec written before it existed is not malformed — but its claims are prose only, and nothing machine-checkable backs them. Say so in the audit rather than letting the absence pass unremarked. Never escalate this to a failure.Confirm the multi-file
/speclayout. Required files:spec.jsonREADME.mdspec/1-problem-motivation.mdspec/2-scope.mdspec/3-high-level-architecture.mdspec/4-data-flow-components.mdspec/5-interfaces-apis.mdspec/6-implementation-plan.mdspec/7-non-functional-requirements.mdspec/8-risks-alternatives.mdresearch/sources.md
If any are missing, abort with:
This skill validates only the structured
/specformat (8 sections + spec.json). The directory<path>is missing:<list>. For ad-hoc or single-file specs, run/create-auditwith the spec file as input instead.Read everything into context.
spec.json,README.md, all 8 section files,research/sources.md, andcurrent-state/analysis.mdif present (brownfield).
Phase 2 — Structural & Metadata Audit
Verify the spec itself is well-formed before going code-hunting. Findings here are INFO unless they break downstream phases.
spec.jsonschema:- Required fields present and non-empty:
id,title,status,owners,linked_paths,acceptance_commands,depends_on_specs,active_prs. idmatches the directory name.statusis one of:drafting,review,approved,implementing,done,superseded.linked_paths,acceptance_commands,depends_on_specs,active_prsare arrays (may be empty for the latter two).
- Required fields present and non-empty:
linked_pathsresolution. Each entry is a glob. For each, expand viagit ls-files(or filesystem walk if not a git repo) and confirm at least one match exists. List unresolved globs as WARNING.depends_on_specsresolution. Each entry must point to anotherdocs/specs/<dep>/spec.jsonthat exists. Missing dependencies are WARNING.README consistency. Parse the status table in
README.md. For each row, compare the marker ([ ] empty,[~] in-progress,[x] done) against the matchingspec/<n>-*.mdfile. A[x] donerow whose section file is empty or scaffold-only is INFO (doc drift, not blocking).DOC-N round-trip. Grep
research/sources.mdfor\bDOC-\d+\bto build the catalog. Then grep all section files for\bDOC-\d+\breferences. Flag both directions:DOC-Nreferenced in a section but not defined insources.md→ WARNING.DOC-Ndefined insources.mdbut never cited → INFO.
Phase 3 — Constraint Evidence Audit
This is the audit's substance. The spec makes claims via tagged constraints; this
phase checks whether the codebase actually delivers each one. The discipline is
the same as /ground-first: every verdict cites file:line, never "probably"
or "should be".
Build the constraint inventory. Grep all section files for the pattern
\b(ARCH|IMPL|TEST|PERF|REL|OPS|SEC|KD|R|A)-\d+\b. For each match, record:- the constraint id
- the section it's defined in (where the row introduces it, not just mentions it)
- the constraint text (the row content for table-defined constraints, or the surrounding paragraph for prose-defined constraints)
Search for evidence per constraint. For each constraint, search the
linked_pathsfiles first (highest signal), then the broader repo if nothing found. Look for:- Direct traceability: the literal constraint id (
PERF-2,KD-7) in a code comment, test name, or commit message. This is the strongest evidence. - Behavioral evidence: function names, log fields, metric names, error codes, env vars, config keys, or test descriptions that the constraint text mentions explicitly.
- Test evidence: test files that name the constraint or its behavior.
- Direct traceability: the literal constraint id (
Classify each constraint:
- Implemented — at least one direct or behavioral citation, with
file:line. - Partial — some evidence but the constraint has multiple parts and only some are covered (cite what was found, name what's missing).
- Unverified — no evidence found in
linked_pathsor the wider repo. Do not guess. Unverified is a legitimate verdict — that's what the audit is for. - N/A — the spec explicitly defers it (e.g. "Out of scope for v1" in the constraint row, or
R-Nrisks classified as "accepted, no mitigation").
- Implemented — at least one direct or behavioral citation, with
§6 Implementation Plan extra checks. Per the
/specconvention (seereferences/cc-prompt-templates.mdin the spec skill), prompt blocks under §6.3 should each contain:- A
<read-first>block listing source files, - Test names (TDD-first),
- A
<verify>block.
Flag prompt blocks missing any of the three as INFO (process drift, not implementation failure).
- A
§8 Risks audit. For each
R-Nrow, check whether theMitigationcolumn points to anything in the codebase (a guard, a test, a runbook). Risks with empty mitigations are INFO unlessLikelihood+Impactare both High → WARNING.
Phase 4 — Acceptance Command Execution
The acceptance_commands array in spec.json is the spec's executable
ground truth. Run them.
Skip this phase entirely if --no-run was passed. Record the commands as "not executed in this run" in the audit and proceed to Phase 5.
For each command in
spec.json.acceptance_commands:- Run it from the project root.
- Capture exit code, last 30 lines of stdout, last 30 lines of stderr.
- Record duration.
Verify the acceptance criteria, when the spec declares any:
dotbabel criteria verify --spec <spec-id> --json<spec-id>is the directory name resolved in Phase 1 — it must match an existing entry underdocs/specs/. If it does not, stop rather than pass an unverified string to a command line.--jsonis not optional here. When every criterion is stillplannedthe command short-circuits, prints no per-criterion lines on stdout, and exits 0 — which is exactly the shape of a freshly scaffolded spec, so without--jsonthere is nothing to record.This belongs here, beside the acceptance commands, because both are the spec's executable ground truth. An audit that ran only the commands could report a spec as implemented while the criteria naming its tests fail.
Record each criterion's verdict in the audit by id. The tool emits one of five, and
plannedis not among them — a criterion declaredplannedinspec.jsonis reported aspending:Verdict Meaning Audit passran, and every named test was confirmed OK pendingdeclared planned; its tests have not landed yetINFO, not a failure failran and a named test failed CRITICAL — the spec asserts what its own tests contradict unconfirmedran, but a declared test name was not found in the report CRITICAL — usually a paraphrased name; the lookup is exact errorcould not run: spawn failure, timeout, unreadable report INFO, blocked — an environment problem, not a spec problem Exit 2 is the environment, not the spec. Criteria plans require trust, and trust is granted per realpath, so a spec audited from a fresh worktree exits 2 with no verdicts at all. Record that as blocked, label no criterion failed, and re-run after trusting the path.
Never resolve a failing criterion by editing the criterion. This skill audits a spec against the code; rewriting the claim to match broken code inverts that, and the audit would then certify exactly the drift it exists to catch.
On any failure, prove pre-existing vs. introduced. This is non-negotiable per the user's CLAUDE.md test discipline — never assert "pre-existing" without proof. The required check:
git stash --include-untracked <failing-command> STASHED_EXIT=$? git stash pop- If
STASHED_EXITis also non-zero → label the failure pre-existing (recorded as WARNING, not CRITICAL, since not the spec's fault but worth flagging). - If
STASHED_EXITis zero → label the failure introduced (recorded as CRITICAL — the spec implementation broke this command). - If there is nothing to stash (clean working tree), the failure is in committed code — label pre-existing.
- If
Always restore working state.
git stash popafter every stash. Ifgit stash poperrors (e.g. conflicts), stop the entire skill, surface the situation to the user, and do not write the audit. Losing user work is worse than missing the audit.
Phase 5 — Write the Audit Doc
Compose the audit using the exact template at references/audit-template.md. Fill every section. If a section has no findings, write "No issues found." rather than omitting it — readers need to know it was checked.
Severity convention (matches
/create-audit):- CRITICAL — Constraint marked "Implemented" or status
done/implementinginspec.jsonbut acceptance command fails as introduced. Or: a CRITICAL/HIGH risk has no mitigation evidence and the corresponding behavior fails. - WARNING —
PartialorUnverifiedconstraints; pre-existing acceptance failures; unresolvedlinked_paths; missing dependency specs; missing risk mitigations on high-impact risks. - INFO — README/section status drift; orphan or undefined
DOC-N; §6 prompt blocks missing<read-first>/<verify>/test names; documentation nits.
- CRITICAL — Constraint marked "Implemented" or status
Filename:
spec-<spec-id>-validation-<YYYY-MM-DD>.md, lowercase kebab-case, written to<project-root>/docs/audits/. Createdocs/audits/if it doesn't exist. If a file with that name already exists from earlier today, append-2,-3, etc. — never overwrite a prior audit.Do not modify the spec. No edits to
spec.json,README.md, or any section file. No staging, no committing of any file. Leave the audit as untracked. The user decides whether to commit it.Report to the user, in this exact tight form (no document dump in chat):
Wrote: docs/audits/spec-<id>-validation-<date>.md Constraints: <implemented>/<partial>/<unverified>/<n/a> (total <N>) Acceptance: <pass>/<fail-introduced>/<fail-pre-existing>/<not-run> Issues: <critical> CRITICAL · <warning> WARNING · <info> INFOThat's it. The audit doc speaks for itself.
Key Principles
Evidence before claims. Every "Implemented" verdict cites
file:line. "Unverified" is honest; guessing is forbidden. The audit's value is in being trustworthy — a single hallucinated citation poisons the whole document.Audit, don't fix. Findings only. The user decides what to act on. If you spot a bug while auditing, log it as an issue with severity and location — do not patch it.
Read-only against the spec. Never edit section files,
README.md, orspec.json. Status drift between README and section files is a finding, not something to silently correct. The spec is an artifact of human + AI collaboration; "fixing" it without the user changes the historical record.Honor the test discipline. Pre-existing failures must be proven via
git stash, never asserted. The user's CLAUDE.md is explicit on this and past sessions have been wrong about it. Always restore working state before moving on.Tight output. Tables over prose. No filler, no narrative throat-clearing. The audit is a working document the user opens to decide what to do next, not a report to read end-to-end. Match
/create-audit's discipline.One spec at a time. Don't try to validate multiple specs in one invocation. Each spec gets its own focused audit. Multi-spec runs encourage shallow checks and cross-contamination of evidence.