[BLOCKING] Execute skill steps in declared order. NEVER skip, reorder, or merge steps without explicit user approval. [BLOCKING] Before each step or sub-skill call, update task tracking: set
in_progresswhen step starts, setcompletedwhen step ends. [BLOCKING] Every completed/skipped step MUST include brief evidence or explicit skip reason. [BLOCKING] If Task tools are unavailable, create and maintain an equivalent step-by-step plan tracker with the same status transitions.
Quick Summary
Goal: Ensure the review target (changed production code) is covered by tests that protect real business behavior with correct data assertions, infinite repeatability, and spec alignment — verifying every behavior change has test coverage (integration-first, unit fallback) so that specs ↔ tests ↔ code stay aligned (spec-driven development).
Summary:
- Purpose: Review target is the CHANGE (collect BOTH changed production code AND changed test files), never just the test files — Gates 1-6 and 8 judge test quality, Gate 7 maps every behavior-changing production file to a covering test (integration-first; unit only with recorded justification) + spec TC. Uncovered changed behavior = HIGH finding minimum.
- The 8 Gates (main review steps): G1 Assertion Value — mutation-score, record the Mutation Probe Ledger (no ledger = FAIL); G2 Data State — assert specific DB fields with async polling; G3 Repeatability — unique IDs, additive-only, 2 consecutive green runs; G4 Domain Logic — read handler, assert ONLY fields it writes; G5 Spec Traceability —
TestSpecannotation → TC in spec docs (1 TC → many tests is correct); G6 Three-Way Sync — feature-docs > test-spec docs > code > test, escalate conflicts; G7 Change Coverage — every behavior-changing file → covering test + non-stale §8 TC; G8 Scenario Fidelity — the setup's sequence, pacing, and data must be reachable in production; settle barriers in ARRANGE, never widened assertion timeouts. - The phase pipeline (run ALL,
TaskCreateeach): P0 Scope-detect → P1 Collect (split prod vs test files) → P2 Gate Review (Gates 1-6 + 8 per file, Gate 7 across set) → P3 Spec Cross-Check (both directions) → P4 Initial Report → P5 Fix ALL Crit/High + WRITE missing tests → P6 Validated-fix + full fresh re-review until 0 Crit/0 High → P7 Build & run ALL tests → P8 Failure Investigation → P9 Why-Review self-validation. - Read handler/service source (and feature docs) BEFORE judging any assertion. FAIL smoke-only, existence-only (not-null), dead (always-true), copy-paste, and DI-resolution-only tests — why: assertion quality is unknowable without knowing what the handler actually writes.
- Don't just report gaps — fix them. Gate 6: NEVER fix a test to match broken code, NEVER self-resolve a three-way conflict (escalate via
AskUserQuestion). Phase 5 WRITES the missing test (runs/spec [mode=tests]for SPEC-GAPs); a full fresh re-review runs after every validated fix cycle until a clean pass returns 0 CRITICAL/0 HIGH. - MANDATORY feature-area-wide TC audit (Phase 1 task + Phase 3 addendum): Gate 7 alone is diff-scoped — it can't see a pre-existing §8 TC that lost its covering test outside the diff. Every review non-skippably enumerates the FULL Section-8 TC list of the implicated feature doc(s), not only diff-touched TCs, into the SAME Coverage Mapping Table; zero GAP rows required, whole table, before PASS.
Scope: The FULL change set — changed production code AND changed test files — from uncommitted changes (default), user-specified files, or a user-specified diff (branch/PR). The review target is never "just the test files".
Workflow: Phase 0 Detect → Collect → Coverage Map (Gate 7) → 8-Gate Review → Spec Cross-Check → Report → validate findings → fix validated issues (including writing missing tests) → full re-review after fixes → Build & verify → If fail: investigate + fix plan
Non-negotiable rules:
MUST collect BOTH changed production code AND changed test files — coverage of the change is part of the review, not an optional extra
MUST verify every behavior-changing production change maps to a covering test — integration test FIRST; unit test ONLY with recorded justification (Gate 7)
MUST treat an uncovered changed behavior as a HIGH finding minimum — fix by writing the missing test in Phase 5, not just reporting
MUST verify spec↔test↔code alignment for changed code, not only for existing tests — a changed behavior with no TC in spec docs is a spec gap finding
MUST read handler/service source BEFORE judging any test assertions
MUST flag smoke-only tests (no-exception-only checks) as FAIL
MUST flag DI-resolution-only tests (resolve + not-null) as FAIL — NOT integration tests
MUST verify tests use unique IDs per run (infinitely repeatable)
MUST use async polling/retry for ALL DB assertions — async delays are norm
MUST flag repository-created or repository-mutated test data that bypasses real use cases and can leave invalid state
MUST treat an unrealistic setup as a review finding (Gate 8) — compressed pacing between actor steps, a fixed sleep standing in for a real observable, a widened assertion timeout replacing an ARRANGE barrier, or a retry wrapped around a failing assertion
MUST require 2 consecutive successful suite/project runs before declaring integration tests verified/idempotent
NEVER accept assertions that always pass regardless of handler correctness
NO smoke/fake/useless tests — every test MUST execute actual operations and verify data state
docs/project-reference/integration-test-reference.md— Integration test patterns, fixture setup, seeder conventions, lessons learned (MUST READ before reviewing)
First Principle — Easy to Change
The success metric of every coding decision is future change cost. DRY, SRP, abstraction, design patterns, naming, layering, tests — every technique exists to serve one goal: making the next change cheaper.
When evaluating code, refactor, test, or abstraction, ask: does this make next change cheaper or more expensive?
- Reject "best practices" raising change cost (premature abstraction, speculative generality, leaky indirection, ceremony without payoff).
- Name real enemies in findings: coupling, hidden state, duplicated knowledge, unclear intent, irreversible decisions exposed too early.
- Simpler design easy to change beats sophisticated design that isn't.
Apply this lens before invoking any specific rule, pattern, or checklist below — if downstream rule would raise change cost, this principle wins.
Phase 0: Scope Detection
Classify BEFORE any gate review. Route wrong → waste all effort.
| Signal | Classification | Action |
|---|---|---|
| No user-specified files | Uncommitted changes | Run git diff --name-only (staged + unstaged) to collect scope — BOTH production code AND test files |
| User specifies files/diff (branch, PR, etc.) | Explicit scope | Use provided list/diff directly — still split into production vs test files |
| 10+ test files | Large scope | Parallel sub-agents grouped by module |
| 1-9 test files | Normal scope | Single review pass |
| 0 test files BUT production code changed | Coverage-gap review | Gate 7 IS the review — map every changed behavior to existing tests; uncovered behavior = finding. Do NOT exit |
| 0 changes at all | Empty target | Ask user for explicit scope via AskUserQuestion |
The review target is the CHANGE, not the test files. Changed test files are reviewed for quality (Gates 1-6 and 8); changed production files are checked for coverage and spec alignment (Gate 7). Both halves are mandatory.
Search for test reference docs — NEVER hardcode paths. Grep for integration-test-reference, test-patterns, integration-test-guide near changed test files to discover project-specific conventions before starting gate review.
The 8 Quality Gates
Gates 1-6 and Gate 8 apply per changed/target TEST file. Gate 7 applies to the CHANGE SET — every behavior-changing production file must map to a covering test and a spec TC.
Gate 1: Assertion Value — "Would this catch the bug?" (MUTATION-SCORE gate)
Think: If a single line of the handler's core logic were changed (a
>flipped to>=, a field assignment removed, a boolean negated), would at least one assertion FAIL? If NONE → FAIL. This is the mutation-testing question, made automatic.
#1 AI failure: hallucination assertions — look real, verify nothing.
Operationalized — run the project's mutation tool (PRIMARY). "Would this catch the bug?" is exactly the mutation-score question. Mechanize it instead of eyeballing it:
- Discover the configured mutation tool from
docs/project-config.json, dependency manifests, and CI config. Common per stack: Stryker (JS/TS, .NET — StrykerNet), PITest (Java/JVM), mutmut or cosmic-ray (Python). Cite the local config or command if one exists. - Run it scoped to the CHANGED handler/service (mutate only the production files in the review target — never the whole repo) against the covering tests from Gate 7.
- Read the surviving-mutant report. Each surviving mutant = a missing invariant = an assertion gap → a HIGH finding minimum (CRITICAL when the mutated line touches authorization, money, or data integrity).
- Fix in Phase 5 by WRITING the killing test — the assertion or property that fails on that mutant. Re-run until the changed code's mutants are killed (or each survivor has a recorded justification, e.g. equivalent mutant). Raising the line-coverage number is NOT a fix — coverage that executes a line without asserting its effect leaves the mutant alive.
Manual fallback (when NO mutation tool is configured or addable). Apply the single-mutation thought experiment by hand: read the handler source, then for each core-logic line ask "if I deleted or inverted this, which assertion fails?" If the answer is NONE for any business-critical line → FAIL. Prefer recommending the stack-appropriate mutation tool as a harness add so the gate becomes automatic next time.
PASS: Every changed core-logic line is killed by ≥1 assertion — no surviving mutant on the changed code (mutation tool), or the manual single-mutation check finds a failing assertion for each (fallback) — AND the Mutation Probe Ledger (below) is recorded in the report with a KILLED/SURVIVOR verdict per changed core-logic line. No ledger → no PASS, regardless of how clean the eyeball check felt.
FAIL:
- A surviving mutant on a changed core-logic line with no killing test (or no recorded equivalent-mutant justification)
- No-exception as ONLY assertion
- Not-null without content check
- Assertions on fields handler doesn't modify
- Dead assertions:
x >= 0where x always >= 0,count >= 0, string not-empty on required fields
Verify: Run the mutation tool on the changed handler → list surviving mutants → each survivor is a missing assertion to write. When no tool is available: read handler source → list fields/branches it changes → check at least one assertion would fail if each were mutated.
Recorded artifact — Mutation Probe Ledger (REQUIRED, non-skippable, BOTH paths). "I checked it mentally" is not evidence. Gate 1 cannot be marked PASS without this ledger written into the review report — it is the proof the probe ran, identical in obligation whether a tool ran or the manual fallback did:
Changed core-logic line (file:line, abstract) |
Mutation applied (>→>=, assignment dropped, boolean negated, branch removed) |
Killing assertion / test (TC-… + file:line) |
Verdict |
|---|---|---|---|
| {the line} | {the mutant} | {the assertion that fails on it} | KILLED |
| {the line} | {the mutant} | — none — | SURVIVOR → finding (or recorded equivalent-mutant justification) |
Rules: (1) every changed core-logic line gets a row — no sampling, no "representative subset". (2) Tool path: rows come from the surviving-mutant report; manual fallback: rows come from the line-by-line thought experiment — same table, same columns. (3) A SURVIVOR row with no killing assertion is a HIGH finding minimum (CRITICAL on auth/money/data-integrity lines) UNLESS it carries a written equivalent-mutant justification. (4) An empty or absent ledger = Gate 1 FAIL (not "skipped") — the gate is unproven, so it cannot pass.
Gate 2: Data State — "Does it check the database?"
Think: Does this test prove the database changed, or just that no exception occurred?
PASS: After command, test queries DB and asserts specific entity field values.
FAIL:
- Only checks return value, never verifies DB state
- Checks existence (not-null) without field values
- Missing async polling on side-effect assertions
Exception: Smoke-only ONLY when side effect truly unobservable. MUST include explicit justification comment.
ALWAYS use async polling/retry for data assertions. Event handlers, bus consumers, background jobs run async — data may not be immediately available.
Gate 3: Repeatability — "Can I run this 100 times?"
Think: If this test runs N times in a shared database, does it get noisier each run? Would run #2 fail?
FAIL: Hardcoded IDs, hardcoded business keys without unique suffix, teardown/cleanup, ordering dependency, seeders without existence check, or direct repository setup that creates state users could not create through real use cases.
FAIL (not parallel-safe — see SYNC:test-data-isolation): Assertions hung off a shared mutable entity another test can change, OR off a parent a bulk re-sync/recompute/rebuild/cascade consumer can wipe — even when THIS test never mutates it. Each test MUST own fresh per-test data; only immutable lookup data may be shared. Verify by grepping every other test on that shared data AND every cross-cutting consumer over it.
Verify: Repeatability is only proven when the relevant suite/project passes 2 consecutive runs without resetting data. One green run is not enough.
Gate 4: Domain Logic — "Does test match handler?"
Think: Did I read the handler source? Do I know which exact fields it writes? Do assertions check those fields — and ONLY those fields?
PASS: Assertions match what handler ACTUALLY does (verified by reading source). Covers primary business rule. Validation paths tested.
FAIL: Assertions on untouched fields (copy-paste), missing primary side-effect assertion, event handler tests that never trigger the event.
Verify: Grep handler class → read it → list what it does → compare with assertions.
Also check:
- Authorization: test verifies both authorized AND unauthorized access paths?
- Coverage: happy path + validation failure + DB state check (3 tests minimum)
Gate 5: Spec Traceability — "Is this tracked?"
Think: Can I trace TC-XXX-NNN from test annotation → spec docs → feature docs in one unbroken chain?
PASS: Business test has a TestSpec annotation linking to a TC ID that exists in spec docs. Technical-only test has a TechnicalSpec annotation and does not claim §8 business coverage. The test method name need NOT match the TC, and many test methods may legitimately carry the same TC (one business TC → many tests across components/services — the join key is the test-spec annotation, not the method name; see tc-format.md → TC ↔ Test Code Cardinality).
FAIL (WARN, not BLOCK): Missing annotation, orphaned TC ID (business TestSpec points to a TC absent from spec docs), technical-only test still carrying a business TestSpec, or spec says "Planned" but test exists. NOT a finding: several tests sharing one TC, a method name that differs from the TC, or a technical-only test carrying TechnicalSpec instead of TestSpec.
Gate 6: Three-Way Sync — "Do test, code, and docs agree?"
Think: Have I read ALL 3 sources? Where exactly do they disagree? Does evidence support a verdict, or must I escalate?
Hardest gate. Identify discrepancy, classify using source-of-truth hierarchy — NEVER silently pick winner. Always state the resolved source with file:line evidence — why: a winner picked without evidence hides bugs.
Source of Truth Hierarchy (highest → lowest)
| Priority | Source | Why |
|---|---|---|
| 1 (Highest) | Feature docs (docs/specs/…/Section 8 TCs) |
Business intent — defines WHAT must happen |
| 2 | Test-spec docs (docs/specs/) |
TC scenarios derived from feature docs — defines HOW to verify |
| 3 | Implementation code (handler/entity/service) | What WAS built — may reflect intentional evolution not yet in docs |
| 4 (Lowest) | Integration test code | What IS being tested — most likely to be wrong or stale |
Rule: Docs win over code. Code wins over tests. Feature docs win over test-spec docs.
Conflict Classification
| Pattern | Feature Doc | Impl Code | Test Code | Verdict | Action |
|---|---|---|---|---|---|
| All agree | ✓ | ✓ | ✓ | PASS | None |
| Stale docs | — | ✓ | ✓ | Docs lag code | Flag docs for /docs-update; test is correct |
| Wrong test | ✓ | ✓ | ✗ | Test wrong | Fix test assertions to match code + docs |
| Code bug | ✓ | ✗ | ✓ | Code has bug | Report as BUG — do NOT fix test to match code |
| Test + code diverge from docs | ✓ | ✗ | ✗ | Code bug + wrong test | Fix test to match docs; report code bug |
| Three-way conflict | ✗ | ✗ | ✗ | ESCALATE | Cannot self-resolve — AskUserQuestion |
CRITICAL rules:
- NEVER fix a test to match broken code — that hides bugs
- NEVER assume docs are wrong without evidence they were intentionally superseded
- NEVER self-resolve a three-way conflict — always escalate via
AskUserQuestion - "Stale docs" verdict requires BOTH code AND test to agree — one source never enough
- When escalating, include: TC ID, what each source says, evidence found
Verify Each Source
- Feature doc: Read Section 8 — scenario title, preconditions, steps, expected results
- Test-spec doc: Find same TC — Planned/Implemented status and described scenario
- Implementation code: Read handler/entity/service — fields written, events fired, validation rules
- Test code: Read test method — arrange, execute, assert
Compare each pair with file:line evidence for each source.
PASS: All three agree. WARN: Minor wording, same semantic. FAIL: Semantic disagreement on field/rule/outcome. ESCALATE: All three differ and evidence cannot resolve.
Gate 7: Change Coverage — "Is every changed behavior tested AND specced?"
Think: For each behavior-changing production file in the review target, which test would FAIL if this change were broken? If NONE → coverage gap. Which spec TC describes this behavior? If NONE → spec gap.
This gate makes the skill verify the REVIEW TARGET has coverage — not merely review tests that happen to exist.
Scope note: protocol below diff-scoped (changed production files → TC). Does NOT by itself prove 100% feature-area coverage — a pre-existing TC that lost its covering test, or was never covered, sits outside this diff and passes silently. Phase 3 addendum — Feature-Area-Wide TC Audit (below) closes that gap: audits every Section-8 TC in implicated feature doc(s), not only ones this diff touches; both feed the SAME Coverage Mapping Table, SAME zero-GAP exit bar.
Protocol:
- Collect changed production files from the review target (Phase 0 scope): commands, queries, handlers, entities, services, event handlers, consumers, controllers, frontend services/stores with business logic.
- Filter to behavior-changing files. Exclude: migrations (one-time execution paths), generated code, pure renames/formatting, config-only, DI registration-only changes. Record each exclusion with reason.
- Find covering tests per changed behavior — use graph (
query tests_for <fn>,trace <file> --direction both) plus grep for the handler/class name under test directories. A test COVERS a change only if it exercises the changed path and asserts the changed outcome — read the test; name match alone is NOT coverage. - Apply test-type priority: integration test FIRST (subcutaneous CQRS through real DI, data-state assertions). Unit test is an acceptable fallback ONLY when integration coverage is infeasible (pure function/calculation logic, no observable data state, no DI path) — record the justification per fallback.
- Check spec alignment for the change — existence AND correctness. Each changed behavior must map to a TC in spec docs (feature doc Section 8 / test-spec docs). Finding a TC is NOT enough: READ the mapped TC and confirm it describes the CURRENT behavior. New behavior with no TC, or a TC that exists but still describes the OLD/superseded behavior → spec gap (spec-driven development violation). A behavior is only fully covered when a covering test exercises it AND a non-stale §8 TC documents it — so this correctness re-check applies to COVERED rows too, never just to GAP rows.
Coverage Mapping Table (MANDATORY output):
Rows are keyed by changed production behavior, not by TC. A behavior is COVERED when ≥1 covering test exercises it — list ALL covering tests in the column when several apply. One
Spec TCmay legitimately appear across multiple rows and be covered by many tests (one TC → many tests, 1:N). Do NOT expect or require one test per TC, and do NOT flag a TC reused across rows as a duplicate (seetc-format.md→ TC ↔ Test Code Cardinality).
| Changed File / Behavior | Spec TC | Covering Test(s) | Test Type | Verdict |
|---|---|---|---|---|
| {file:line — behavior} | TC-X-NNN / MISSING | {test file:method}[, …one or more] / NONE | integration / unit (justified) / — | COVERED / COVERED-UNIT / GAP / SPEC-GAP |
Verdicts:
- COVERED — integration test exercises the changed path with data-state assertions AND the mapped §8 TC describes the CURRENT behavior. A covering test whose mapped TC is stale is NOT COVERED — record it as SPEC-GAP.
- COVERED-UNIT — unit test covers it, integration infeasible, justification recorded, and the mapped §8 TC is current (same stale-TC rule applies).
- GAP (FAIL) — no test would fail if the change broke. Severity: HIGH minimum; CRITICAL when the change touches authorization, money, or data integrity. Fix in Phase 5 by WRITING the missing test (integration-first) — reporting alone does not clear this gate
- SPEC-GAP (FAIL) — behavior has no TC, OR a covering test exists but its mapped TC still describes OLD/superseded behavior (stale TC ≠ covered). Both the missing-TC and the stale-but-covered case are SPEC-GAP. Fix via
/spec [mode=tests]UPDATE (and/specwhen business rules changed)
FAIL:
- Changed handler/command/entity rule with zero covering test
- Unit test substituted where an integration test is feasible, with no justification
- Test exists but does not assert the changed outcome (stale coverage counted as coverage)
- New/changed behavior absent from spec docs, or TC describing superseded behavior
- A COVERED row marked COVERED without reading its mapped §8 TC — TC existence assumed instead of its CURRENT-behavior correctness verified (a stale-but-covered TC silently passes as covered)
Explicit user waiver (recorded verbatim in the report with the user's reason) is the ONLY alternative to closing a GAP.
Gate 8: Scenario Fidelity — "Could production ever reach this setup?"
Think: Read the ARRANGE block as a production trace. Could this exact sequence, timing, and data actually occur in the running system? If no, the test is mis-specified — the finding lands on the SCENARIO, never on the assertion.
An unrealistic setup proves nothing when it passes and burns hours when it fails. Judge fidelity BEFORE judging assertion strength — a strong assertion over an impossible scenario is still a defective test. Applies per changed/target TEST file, alongside Gates 1-6.
Finding triggers — mechanically checkable; each fires as a finding with severity + file:line evidence:
| Trigger (what to look for in the test) | Severity | Fix direction |
|---|---|---|
| Chained actor actions with no settle barrier — two or more distinct actor actions issued back-to-back that production separates by seconds, minutes, or hours | HIGH (CRITICAL when an in-flight async message can land between them and clobber state) | Add an ARRANGE barrier that polls a real observable proving the prior action finished |
| Fixed sleep standing in for a real observable — a hardcoded delay where a persisted state change, version/audit stamp, queue/worker idle marker, or completion event exists | MEDIUM (HIGH when the delay is what keeps a race from firing) | Replace with poll-until-settled on that observable; a fixed delay is acceptable ONLY when no observable exists AND a comment says so |
| Widened assertion timeout instead of an added precondition — the ASSERT-side wait grew, or a retry/poll wrapper appeared there, rather than an ARRANGE barrier being added | HIGH | Move the wait into ARRANGE and restore the original assertion window |
| Unreachable setup state with no explanation — ARRANGE constructs a state and nothing records how production could reach it | MEDIUM (HIGH when the asserted outcome depends on that state) | Reach the state through real use-case paths, or label it a deliberate impossible-state test with the reason it is reachable |
| Retry wrapped around a failing assertion — a retry/loop added around the ACT+ASSERT pair after a red run | CRITICAL | Revert the retry; adjudicate the intermittency (/integration-test-verify) before any change |
| Fidelity improvement that weakened the protected invariant — the scenario became realistic and the assertion became looser in the same change | HIGH | Keep the assertion; find a DIFFERENT realistic scenario that still exercises the rule |
PASS: Every actor step in ARRANGE could occur in production in that order and at that pacing; every wait is an ARRANGE-phase barrier on a real observable (or a commented fixed delay where no observable exists); any deliberately impossible state carries a comment naming WHY production could reach it (upstream bug, partial write, legacy data).
FAIL: Any trigger row fires with no recorded justification.
Verify: Read the test end-to-end as a production trace — per actor step ask "what separates this from the previous step in real life, and what does the test wait on?" Then grep the test file (and, when reviewing a change, its diff) for sleep/delay calls, retry/poll wrappers, and enlarged timeout arguments; every hit is a candidate row above. A trigger that fires in the DIFF (a wait that grew, a retry that appeared) outranks one that merely pre-existed — it is evidence of masking in progress.
Review Protocol (9 Phases)
Use TaskCreate for EACH phase before starting.
Phase 1 — Collect: Split the change set: production files (Gate 7 coverage targets) vs test files. Categorize test files: new (full review), modified (changed methods only), new projects (infra + samples). Categorize production files: behavior-changing vs excluded (with reason).
MANDATORY task — "Validate: 100% Section-8 TC coverage for {feature doc(s)} — not just diff-touched TCs." Create as OWN named
TaskCreateitem in Phase 1 breakdown. Non-skippable whenever this review runs inside a workflow, current git changes are present (staged/unstaged), or by direct user request — every invocation of this skill except a scope explicitly narrowed by the user to a single named TC/test. Identify feature doc(s) implicated by the change set (or named by the user) up front — their full Section 8 TC list is the audit scope for Phase 3's addendum below, independent of which TCs the diff touches.
Phase 2 — Gate Review: Per test file, apply Gates 1-6 and Gate 8. Apply Gate 7 once across the change set and produce the Coverage Mapping Table. Record per-file verdict table:
| Gate | Verdict | Evidence |
|---|---|---|
| 1. Assertion Value | PASS/FAIL | {file:line} |
| 2. Data State | PASS/FAIL | {file:line} |
| 3. Repeatability | PASS/FAIL | {file:line} |
| 4. Domain Logic | PASS/FAIL | {file:line} |
| 5. Traceability | PASS/WARN | {file:line} |
| 6. Three-Way Sync | PASS/WARN/FAIL/ESCALATE | {file:line} |
| 7. Change Coverage (per change set) | COVERED/COVERED-UNIT/GAP/SPEC-GAP | {coverage mapping table} |
| 8. Scenario Fidelity | PASS/FAIL | {file:line} |
Phase 3 — Spec Cross-Check + Three-Way Diff: Two directions — from tests AND from changed code.
For each TC ID in code:
- Verify TC entry exists in both
docs/specs/(Section 8) anddocs/specs/ - Read what TC describes in each doc
- Read what implementation code actually does
- Read what test asserts
- Classify conflict pattern (Gate 6 table) and record action
- Flag gaps both directions: TC in code but not in docs, or "Implemented" TC in docs but no test found
For each behavior-changing production file in the review target (reverse direction — spec-driven development check):
- Verify a TC exists describing the changed behavior AND read it to confirm it describes the CURRENT behavior — run this even when a covering test was already found and the row is otherwise COVERED. If the TC still describes the OLD behavior, downgrade the row from COVERED to SPEC-GAP and flag it stale (route to
/spec [mode=tests]UPDATE). Finding a covering test never excuses re-checking the TC's correctness. - New behavior with no TC anywhere → SPEC-GAP finding (Gate 7); recommend
/spec [mode=tests](and/specwhen business rules changed)
Phase 3 addendum — Feature-Area-Wide TC Audit (MANDATORY, satisfies Phase 1 "100% Section-8 TC coverage" task). Steps 1-8 above diff-scoped: map changed production files → TC. This addendum TC-scoped instead, covers TCs the diff never touched:
- Enumerate every
TC-{FEATURE}-{NNN}in Section 8 of all feature doc(s) implicated by the change set (or named by the user) — FULL list, not the subset steps 1-8 already visited. - For each TC not already resolved by steps 1-8, find covering test(s) the same way Gate 7 does (graph query / grep the handler or class the TC describes; read test — name match alone NOT coverage) and read TC to confirm it still describes CURRENT behavior.
- Classify each using SAME verdicts and Coverage Mapping Table format as Gate 7 (
COVERED/COVERED-UNIT/GAP/SPEC-GAP) — append rows to the SAME Coverage Mapping Table Gate 7 produces, so review emits one unified table covering both diff-touched and pre-existing TCs. - A
GAPsurfaced here (pre-existing TC, zero covering test — coverage regressed or never written) carries the SAME Phase 5 "WRITE the missing test" obligation as a diff-touched Gate 7 GAP — not a lesser finding merely because the diff didn't touch it. Severity: HIGH minimum per Gate 7's rule. - Zero
GAProws across the WHOLE unified table (diff-touched + feature-area-wide) required before review can report PASS on Gate 7 / the Phase 1 mandatory task.
Phase 4 — Initial Report: Write to plans/reports/integration-test-review-{date}-{slug}.md
Phase 5 — Fix All Issues (MANDATORY — fix ONLY findings already validated per the embedded double-round-trip-review validate-before-fix contract): Fix every CRITICAL and HIGH issue. MEDIUM: fix if straightforward, document as tech debt otherwise.
- Prioritize: CRITICAL → HIGH → MEDIUM
- Per fix: read handler source, understand domain logic, write/fix assertion
- Gate 7 GAP fixes: WRITE the missing test — integration test first (route through
/integration-testpatterns); unit test only with recorded justification. SPEC-GAP fixes: run/spec [mode=tests]UPDATE to add/correct the TC before or alongside writing the test - Gate 8 fidelity fixes: repair the SCENARIO, never the assertion — add the ARRANGE-phase settle barrier on a real observable, restore any widened assertion timeout to its original window, remove any retry wrapped around a failing assertion, and comment a deliberate impossible-state setup with why production could reach it
- NEVER weaken assertions to make tests pass — fix root cause (timing, data, setup) instead
- Re-read changed files to verify fix correctness
- Record each fix with
file:lineunder## Fixes Applied
Phase 6 — Validated Fix + Full Re-Review (MANDATORY when fixes are applied):
Do not spawn a fresh reviewer to re-review the same findings before validation/fix. After Phase 5 applies validated fixes, run a full fresh review over the current test scope. When that review uses sub-agents, spawn fresh integration-tester sub-agents (parallel by module for 10+ files; single agent otherwise) using canonical Agent template from SYNC:review-protocol-injection. Each sub-agent re-reads ALL target test files from scratch with ZERO memory of Phase 2/5. When constructing Agent call prompt:
- Copy Agent call shape from
SYNC:review-protocol-injectiontemplate verbatim - Set
subagent_type: "integration-tester" - Embed full verbatim body of 9 SYNC blocks (all present inline in this skill file):
SYNC:evidence-based-reasoning,SYNC:bug-detection,SYNC:design-patterns-quality,SYNC:logic-and-intention-review,SYNC:test-spec-verification,SYNC:fix-layer-accountability,SYNC:rationalization-prevention,SYNC:graph-assisted-investigation,SYNC:understand-code-first - Task field:
"Run a full fresh integration-test review pass over {file-list} after validated fixes were applied. Review against 8 quality gates: assertion value, data state, infinite repeatability, domain logic, test-spec traceability, three-way sync, change coverage, scenario fidelity. Read handler source AND feature docs before judging assertions. Flag smoke-only, existence-only, dead assertions, and repository-created invalid test data as FAIL. Gate 3 also flags tests that are not parallel-safe: assertions hung off a shared mutable entity another test can change, or off a parent a bulk cross-cutting consumer (re-sync/recompute/rebuild/cascade) can wipe even without this test mutating it — each test must own fresh per-test data; prove by grepping other tests on that shared data and every consumer over it. Gate 7: map every behavior-changing production file in {changed-production-file-list} to a covering test (integration-first; unit fallback requires justification) AND a spec TC — uncovered behavior is a HIGH finding minimum, missing/stale TC is a SPEC-GAP finding. Gate 8 (Scenario Fidelity): read each ARRANGE block as a production trace and flag setups production could never reach — distinct actor actions chained with no settle barrier where real life separates them by seconds/minutes/hours, a fixed sleep standing in for a real observable, an assertion timeout widened instead of an ARRANGE barrier added, a retry wrapped around a failing assertion, or a setup state with no explanation of how production reaches it; the finding is on the SCENARIO, never on the assertion. Source-of-truth hierarchy: feature docs > test-spec docs > implementation code > test code. Classify every disagreement as: wrong test, code bug, stale docs, or escalate (three-way conflict)." - Target Files: explicit file list (never pass inline contents)
- Reference Docs: include
docs/project-reference/integration-test-reference.md - Report path
…(truncated)