Lisa Spec Conformance

Spec Conformance: $ARGUMENTS

Compare shipped work against its spec section-by-section. This is the "accountant lens" — did the work ship exactly what was written, nothing more, nothing less? It is NOT UX review (that's product-specialist) and it is NOT empirical system verification (that's verification-specialist). Run it alongside those, not instead of them.

Phase 1 — Resolve Spec Source

Determine the source of truth for this work. Check in this order:

1. Explicit spec argument — plan file path (e.g. .claude/plans/<name>.md), JIRA key (e.g. PROJ-123), Linear key, GitHub issue URL, or PRD path passed as $ARGUMENTS.
2. Linked JIRA ticket — if the current branch or PR body references a JIRA key, use it.
3. PR body — if the PR description contains a "Spec" or "Ticket" link, follow it.
4. Plan file on disk — check .claude/plans/ for an active plan matching the branch name.

If none of the above resolves, stop. Do not guess what the spec was. Report: "No spec source found — pass a plan file, ticket key, or PR URL."

Based on the source, load the full spec:

Phase 2 — Extract Requirements

Parse the spec into a structured requirement list. Do NOT skip sections — every requirement becomes a row in the coverage matrix.

Sections to extract:

If an acceptance criterion is not in Gherkin, still extract it as a requirement — but flag it as LOW_SPECIFICITY so the verdict downgrades. Downgrade rule: if any LOW_SPECIFICITY requirement exists, the maximum possible verdict is PARTIAL unless the spec is tightened and re-evaluated.

Skip narrative prose (Context / Business Value) — it isn't directly verifiable. Reference it only when explaining a miss.

Phase 3 — Inspect Shipped Work

Gather evidence of what was actually shipped:

1. Diff scope — the commits on the current branch vs. the default branch:

   BASE_BRANCH="$(git symbolic-ref refs/remotes/origin/HEAD | sed 's@^refs/remotes/origin/@@')"
   git log "${BASE_BRANCH}"..HEAD --oneline
   git diff "${BASE_BRANCH}"...HEAD --stat
   

2. File-level changes — per-file diff for each changed file:

   git diff "${BASE_BRANCH}"...HEAD -- <file>
   

3. Test coverage — which tests were added/changed for the requirements:

   git diff "${BASE_BRANCH}"...HEAD -- '**/*.test.*' '**/*.spec.*'
   

4. Empirical evidence — output of verification-specialist if available (proof artifacts, API captures, UI screenshots, DB queries). If that report isn't in context, ask the caller for it before proceeding — do not substitute reading code for running the system. 4a. The machine-readable verdict — Read ${CLAUDE_PROJECT_DIR:-.}/.lisa/verification-status.json. Under schema v2 it is the structured form of the evidence above, and it is what lets you check a claim's reach instead of taking "verified" at its word. Load artifact (repository, head_sha, environment), claims[] (claim_id, statement, boundary, required_evidence_kinds, status, evidence_refs, not_established), evidence[] (evidence_id, kind, locator, sha256, captured_at, artifact_head_sha), and the not_established_reviewed flag.
5. PR description — gh pr view --json title,body,files if a PR exists.
6. Deployed state — if the verification phase already hit a deployed environment, use those captures.

Do NOT run the system yourself — that's the verification-specialist's job. Your job is to map their evidence to the spec.

Phase 3b — Cross-Check Claims Against Their Boundaries

The claim-evidence-mapping rule is the contract: every claim declares a boundary, and a claim is established only by evidence of a kind that reaches that boundary. Conformance is not just "was it built" — it is also "does the proof offered actually reach the thing the requirement asserts." A unit test-run-log cited for a requirement about browser-visible behavior is a conformance defect even when the code is perfect.

For every v2 claim loaded in Phase 3 step 4a, run three checks:

Then bind the verdict back to the spec: map each claim_id to the requirement row it discharges. A requirement whose only supporting claim fails a check is not MATCH, no matter what the verification report's prose said. A requirement with no claim at all is MISSING, not PARTIAL.

Degrade, never block. If .lisa/verification-status.json is absent, or carries v1 (no schema_version, or schema_version: 1 — only plan / status / criteria[] / updated_at), the boundary cross-check is not available. Say so explicitly in the report ("v2 verdict not present — boundary reach unverified"), fall back to the prose evidence from Phase 3, and cap the verdict at PARTIAL for any requirement whose boundary you cannot confirm. Do not invent a mismatch you could not check, and do not silently upgrade an unchecked claim to MATCH.

Phase 4 — Build Coverage Matrix

For every requirement extracted in Phase 2, produce one row:

Status definitions

• MATCH — requirement is implemented AND there is empirical evidence it works (test + verification report).
• PARTIAL — implementation exists but evidence is incomplete (e.g. code present, no test; or test present, no run-time verification).
• MISSING — requirement has no corresponding implementation OR no evidence at all.
• BOUNDARY_MISMATCH — the requirement was implemented and evidence was cited, but the evidence does not reach the claim's boundary (a unit test-run-log offered for a browser claim), or its artifact_head_sha does not match artifact.head_sha, or the verdict omits the required Not-established review. This is a distinct failure from a miss: the work may be right and the proof still does not establish it. A BOUNDARY_MISMATCH row forces the verdict to DIVERGES — it can never render as CONFORMS or PARTIAL. Name the boundary, the kind cited, and the kind(s) required, citing the claim-evidence-mapping taxonomy.
• SCOPE_CREEP_VIOLATION — used for excluded classification only. An Out-of-Scope item appears to have been shipped anyway. This is a different failure than a miss — it means the agent exceeded the spec.

Behavior-contract rows (frontend work)

For behavior rows, evidence is the contract itself, not prose: the scenario exists in the project's behavior contract with its stable ID, and each required scenario-platform obligation resolves to a mapping in the coverage map (or a dated waiver). Read the generated coverage matrix and run the project's coverage gate rather than inferring from the diff.

• Shipped frontend behavior with no scenario → MISSING. It is a spec-conformance failure, not a documentation nit.
• A scenario satisfies this row only when its provenance tag (tracker item reference) names the current work item, or the PRD atom this item implements. A same-ID or coincidentally-matching scenario with no provenance link to this item does not satisfy the row — treat it as MISSING and flag the traceability gap; a passing gate elsewhere in the contract proves nothing about this requirement.
• Scenario present but a required scenario-platform obligation has neither a mapping nor a waiver, or the coverage gate fails (stale mapping, invalid waiver, floor regression) → MISSING.
• Scenario, mapping, and a passing coverage gate present, but the mapped test's execution has not been observed to pass (no verification-specialist evidence, no named passing CI run) → PARTIAL. The gate only proves the mapping is statically valid — the file exists and the evidence string is still present — not that the test currently passes; that is a separate signal per the bdd-e2e-coverage rule, and MATCH requires both.
• Scenario and mappings present but the gate was not re-run and the matrix not regenerated → PARTIAL.
• Scenario, mapping, passing gate, AND observed execution evidence that the mapped test passes → MATCH.

Per the bdd-e2e-coverage rule, a waiver is a dated IOU and never counts as coverage — a row backed only by a waiver is reported as MISSING, never MATCH, with the waiver and its recordedAt date named in the Notes column so the IOU stays visible in the matrix instead of disappearing into a passing status. (There is no separate WAIVED status; the existing enum is reused deliberately so a waiver is never mistaken for coverage.) A waiver satisfies the bdd-e2e-coverage rule's definition of done — it does not block the work item from shipping — but spec-conformance is a stricter accounting lens than that gate: reporting the row as MISSING caps this item below CONFORMS until the waiver is cleared. That is intentional, not a false failure — it is the mechanism that keeps a waiver from quietly becoming permanent.

Scope creep detection

Separately from the matrix, scan the diff for work NOT traceable to any requirement. For each such change:

• Identify the file/module
• Summarize the change in one line
• Classify as UNTRACEABLE_CHANGE (not necessarily wrong — refactors often land here — but MUST be surfaced)

Untraceable changes are not automatic failures. They become findings the human reviews.

Phase 5 — Verdict

Produce exactly one verdict:

• CONFORMS — every requirement is MATCH. No SCOPE_CREEP_VIOLATION, no BOUNDARY_MISMATCH. Untraceable changes, if any, are clearly refactors or test support.
• PARTIAL — some requirements are PARTIAL but none are MISSING, BOUNDARY_MISMATCH, or SCOPE_CREEP_VIOLATION. Work is mostly there but evidence is thin.
• DIVERGES — at least one requirement is MISSING, OR at least one BOUNDARY_MISMATCH exists, OR at least one SCOPE_CREEP_VIOLATION exists, OR there are substantive untraceable changes that materially alter behavior.

A verdict of PARTIAL or DIVERGES blocks task completion. The caller must resolve the gaps (implement the miss, remove the creep, add the missing evidence) before re-running.

Phase 6 — Output

Structure the report so it can be pasted into a PR comment or JIRA ticket:

## Spec Conformance Report

**Spec source:** <plan file / JIRA key / Linear / GitHub issue / PRD>
**Shipped scope:** <N commits, M files, K tests on branch <branch> vs <default-branch>>
**Verdict artifact:** <.lisa/verification-status.json schema v2, artifact.head_sha <sha> — or "v2 verdict not present — boundary reach unverified">

### Coverage Matrix

| ID | Class | Requirement | Evidence | Boundary | Evidence kind | Status | Notes |
|----|-------|-------------|----------|----------|---------------|--------|-------|
| AC-1 | acceptance | [text] | [pointer] (AC-1 / EV-1) | browser | screenshot | MATCH | |
| AC-2 | acceptance | [text] | — | — | — | MISSING | No corresponding code or test |
| AC-3 | acceptance | [text] | EV-4 | browser | test-run-log | BOUNDARY_MISMATCH | Unit log cannot establish a browser claim — needs screenshot or recording |
| OOS-1 | excluded | [text] | src/foo.ts:42 | — | — | SCOPE_CREEP_VIOLATION | Added anyway |
| ASSERT-1 | assertion | [text] | verification-report §2 | http-api | — | PARTIAL | Asserted in code, not run in verification |

### Not Established

Reproduce the verdict's `not_established` entries verbatim, grouped by claim, plus anything the matrix could not confirm. This section is **never omitted and never blank**: with nothing outstanding it renders `None outstanding — reviewed`. State whether `not_established_reviewed` was `true`.

- AC-1 — not exercised on mobile viewports; Safari not tested
- AC-4 — offline behavior consciously out of scope for this ticket

### Untraceable Changes
- src/utils/helpers.ts — extracted shared regex constant (refactor, no behavior change)
- src/auth/session.ts — added retry logic (NOT IN SPEC — verify intentional)

### Verdict: CONFORMS | PARTIAL | DIVERGES

**Matches:** N/Total
**Partial:** N
**Missing:** N
**Boundary mismatches:** N
**Scope creep violations:** N
**Untraceable changes flagged for review:** N

### Required Actions (if PARTIAL or DIVERGES)
1. [specific action — implement X, remove Y, add test for Z, capture evidence for W]
2. ...

Rules

• Never substitute "I read the code and it looks right" for empirical evidence. If verification-specialist hasn't run yet, request its report before producing a verdict.
• Never mark a requirement MATCH based on the presence of code alone — evidence means test + runtime observation.
• Never mark a requirement MATCH on evidence that does not reach its claim's boundary. Per the claim-evidence-mapping contract, a unit test-run-log establishes only code-unit behavior; cited for a browser, http-api, deploy-health, or standards-compat claim it is a BOUNDARY_MISMATCH, not a match.
• Always report the Not-established section, even when empty. A report that lists only what passed is unreadable at a gate.
• Always surface scope creep separately from misses. They are distinct failures.
• Always surface untraceable changes — even benign refactors — so the human can confirm intent.
• The Out of Scope section is load-bearing. If the spec has one, every item must appear in the matrix as excluded.
• If the spec has no acceptance criteria, flag the spec itself as inadequate before running the matrix. The verdict is DIVERGES for spec inadequacy until criteria are added.
• Do not invent requirements the spec didn't state. If the shipped work does something reasonable but unspecified, it becomes an untraceable change, not a match.