zuvo:test-audit — Test Quality Triage
Systematic evaluation of unit and integration test files through the Q1-Q25 binary checklist and AP anti-pattern catalog. Each test file is paired with its production source, scored against behavioral coverage standards, and assigned a tier.
Scope: Unit and integration tests only. E2E tests (*/e2e/*, *.e2e.*) are excluded by default. Use --include-e2e to include them.
When to use: After mass test writing, when test quality is uncertain, before releases, when test failures are hard to diagnose, periodic health check.
Out of scope: Single-file code review (use zuvo:review), writing new tests (use zuvo:write-tests), fixing systematic anti-patterns across many files (use zuvo:fix-tests).
Argument Parsing
| Argument | Effect |
|---|---|
all |
Audit every test file in the project |
[path] |
Audit test files under a specific directory |
[file] |
Audit a single test file with full evidence (forces deep mode) |
[file file2 …] |
Audit exactly the listed test files (space-separated, forces deep mode) — the form the Test Quality Gate (../../shared/includes/test-quality-gate.md, called from build/refactor/execute/write-tests/write-e2e) uses to scope to touched files |
--deep |
Collect evidence and fix recommendations for every file |
--quick |
Binary pass/fail only, skip evidence |
--include-e2e |
Include E2E test files in scope |
--details |
Save per-file reports to zuvo/audits/test-audit-details/ |
--commit=ask|auto|off |
Commit behavior after fix workflow (default: ask) |
--read-only |
Report only: skip Phases 4-6 (coverage.md/backlog.md writes) and Phase 7; no repo mutation beyond the zuvo/audits/ report |
Default: all --quick --commit=ask
Dispatch is already authorized — do not ask, and do not substitute. Invoking this skill IS the
request for the gates it mandates. A session-level instruction like "do not use the Agent tool unless
the user asked" does NOT apply here: the user asked, by invoking this skill. Reading it as a
prohibition and recording a self-scored result is the substituted gate this step forbids — it
happened twice in the field (2026-08-07, 2026-08-08), the second time invented as
WARN:substituted-inline, a value no vocabulary defines. If the harness genuinely has no dispatch
capability (Cursor, Antigravity — NOT Codex, which dispatches mechanical workers), follow the ONE documented exception in
test-quality-gate.md; otherwise dispatch.
| Mode | Scope | Depth | Commit | Notes |
|---|---|---|---|---|
all |
Entire project | Standard | --commit=ask |
Default |
[path] |
Directory | Standard | --commit=ask |
Scoped |
[file] |
Single file | Deep | --commit=ask |
Full evidence |
[file file2 …] |
Listed files only | Deep | --commit=ask |
File-list scope (Test Quality Gate callers pass --read-only --commit=off) |
--deep |
Any scope | Full evidence + fixes | Per flag | Thorough |
--quick |
Any scope | Binary only | --commit=off |
Fast triage |
--include-e2e |
+ E2E files | Standard | Per flag | Expanded scope |
--details |
Any scope | + per-file reports | Per flag | Save individual files |
Mandatory File Loading
First resolve ../../shared/includes/execution-policy.md and
../../shared/includes/evidence-reuse.md. Reuse the parent policy and verified evidence for
a nested stage. Load only the rules needed now, with the read-once receipt protocol.
Load the applicable definitions using the read-once protocol. Defer logging/retro includes until completion.
CORE FILES LOADED:
1. ../../rules/testing.md -- READ/MISSING
2. ../../shared/includes/test-edge-cases.md -- READ/MISSING
3. ../../shared/includes/env-compat.md -- READ/MISSING
4. ../../shared/includes/run-logger.md -- READ/MISSING
5. ../../shared/includes/retrospective.md -- READ/MISSING
6. ../../shared/includes/no-pause-protocol.md -- READ/MISSING (HARD: no mid-batch pauses)
7. ../../shared/includes/test-quality-gate.md -- READ/MISSING (carries the dispatch-authorization rule)
8. ../../shared/includes/test-metrics.md -- READ/MISSING (frozen quality/cost/speed formulas — cite TIER_DIST/ECHO_COUNT, never restate)
If any file is missing: Stop. The quality gate definitions are required for scoring.
Environment Compatibility
Read ../../shared/includes/env-compat.md for agent dispatch patterns, path resolution, and progress tracking.
MANDATORY TOOL CALLS — Test Audit Validity Gate
INVALID if any tool below is skipped when trigger holds. "DEFERRED", "N/A", "--quick mode" NOT valid reasons.
| Tool | Trigger | Skip allowed? |
|---|---|---|
find_dead_code |
Always | NO — AP17 unused test data |
find_clones |
Always | NO — AP18 duplicate test names / copy-pasted bodies |
find_references |
Always | NO — orphan tests + untested public methods (no AP id) |
search_patterns |
Always | NO — Q-checklist anti-patterns |
audit_scan |
Always | NO — compound check |
scan_secrets |
Always | NO — hardcoded credentials in fixtures (security, not an AP) |
| Stack-specific tools | Framework/language detected | NO when matches |
Forbidden: find_dead_code: skipped, codesift: unavailable (when deferred), retrospective: skipped — all REJECTED.
POSTAMBLE: report on disk → retro appended → ~/.zuvo/append-runlog exit 0. Every Q-fail/AP finding needs path/to/file.ext:LINE (verify-audit gate).
Mandatory-tools-acknowledgment: I will run find_dead_code + find_clones + find_references + search_patterns + audit_scan + scan_secrets + stack-specific tools for this test audit. Every finding will cite a `path/to/file.ext:LINE` resolving in the current tree.
CodeSift Integration
Use the deterministic preload helper FIRST. Run ~/.zuvo/compute-preload test-audit "$PWD" before any ToolSearch. Copy [CodeSift matching trace] verbatim, issue printed ToolSearch(query="select:..."). Math gate enforced.
Read ../../shared/includes/codesift-setup.md for the full initialization sequence.
Summary: Run the CodeSift setup from codesift-setup.md at skill start. Use CodeSift for file discovery and production code analysis when available. If unavailable, fall back to standard tools.
CodeSift Optimizations
| Task | CodeSift | Fallback |
|---|---|---|
| Find test files | get_file_tree(repo, name_pattern="*.test.*") |
find command |
| Understand test structure | get_file_outline(repo, file_path) |
Read each file |
| Batch-read test cases | get_symbols(repo, symbol_ids=[...]) |
Multiple Read calls |
| Find production file for test | search_symbols(repo, query, kind="function") |
Path-based heuristic |
| Verify test imports | find_references(repo, symbol_name) |
Grep for imports |
| Pre-scan for weak assertions | search_text(repo, "toBeTruthy|toBeDefined", file_pattern="*.test.*") |
Grep |
Degraded Mode (CodeSift unavailable)
All steps fall back to find/Read/Grep/Glob. File discovery is slower and production file pairing relies on path conventions rather than symbol resolution.
Phase 0: Discovery and Pairing
0.1 Locate Test Files
When CodeSift is available: get_file_tree(repo, name_pattern="*.test.*") with path filters excluding node_modules, .next, e2e (unless --include-e2e).
When unavailable:
find . \( -name "*.test.ts" -o -name "*.test.tsx" -o -name "*.spec.ts" -o -name "*.spec.tsx" \
-o -name "test_*.py" -o -name "*_test.py" \) \
! -path "*/node_modules/*" ! -path "*/.next/*" ! -path "*/__pycache__/*" ! -path "*/e2e/*" | sort
If count exceeds 50 and --deep was not explicitly requested, auto-switch to --quick. Explicit --deep always takes precedence.
0.2 Pair with Production Files
For each test, identify its production counterpart:
__tests__/api/projects/[id]/route.test.ts->app/api/projects/[id]/route.tstests/unit/services/bar.test.ts->lib/services/bar.ts
If production file not found: flag as ORPHAN (test without source).
When CodeSift is available, use search_symbols or find_references for more reliable pairing in non-standard project layouts.
0.3 Pre-Batch Grouping
Before splitting into batches, group test files by production file. If multiple test files target the same production code (foo.test.ts + foo.errors.test.ts), they MUST go into the same batch so suite-aware Q7/Q11 scoring works correctly.
0.4 Golden File Calibration (recommended for first audit)
If this is the first audit of a project or agent scores seem inconsistent:
- Pick 2-3 test files with known quality (one good, one bad, one mid)
- Run a single calibration agent on those files
- Compare scores to expectations. If drift >2 points, adjust prompt wording
- Proceed with full evaluation
0.5 Batch Output Directory
mkdir -p zuvo/audits/.test-audit-batch
Each batch agent writes results here. Cleaned up after the final report.
Phase 1: Batch Evaluation
Split grouped files into batches of 8-10. For each batch, spawn a Task agent or process inline.
Each Task agent dispatch:
Agent: Test Quality Auditor (per batch)
model: "sonnet"
type: "general-purpose" # read-only: Read + CodeSift only, no Edit/Write (Explore lacks mcp__codesift__*)
instructions: evaluate test files against Q1-Q25 and AP anti-patterns (see Agent Prompt below)
input: batch file list with paired production files, CODESIFT_AVAILABLE
Agent Prompt (provided to each batch agent)
You are a test quality auditor. Evaluate each test file below against Q1-Q25.
RED FLAG PRE-SCAN (do FIRST, before full evaluation):
- Tests with zero expect() calls (AP13) -> AUTO TIER-D. RTL exception: getByRole/getByText/getByLabelText are implicit assertions.
- Fixture:assertion ratio > 20:1 (AP16) -> AUTO TIER-D
- 50%+ of tests use toBeTruthy()/toBeDefined() as sole assertion (AP14) -> AUTO TIER-D
QUICK HEURISTICS:
- 0 CalledWith in entire file -> likely score <=4
- 10+ DI providers in test setup -> likely score <=5
- Tests calling __privateMethod() directly -> likely score <=5
POSITIVE INDICATORS:
- Factory with named overrides -> likely >=8
- Regression anchor in test name -> mature suite
- it.each with table-driven data -> Q8+Q9+Q11 likely pass
PRODUCTION CODE ANALYSIS (do BEFORE scoring):
Read the production file and extract:
1. Public API surface: all exported functions/methods
2. Branch map: all if/else, switch, ternary with line numbers
3. Enum/union values with their members
4. Error handling: try/catch, thrown errors, rejected promises
5. Complexity: THIN (<50 LOC, <=3 branches), STANDARD (50-200 LOC), COMPLEX (>200 LOC or >10 branches)
COMPLEXITY EXPECTATIONS:
| Complexity | Expected tests | Q11 depth | Edge case scope |
|------------|---------------|-----------|-----------------|
| THIN | 8-15 | cache/wiring only | null/undefined on params |
| STANDARD | 15-40 | all branches | full edge case checklist |
| COMPLEX | 40-80 (split files) | all branches + combos | full checklist + matrix |
CHECKLIST (score 1=YES, 0=NO):
<!-- GATES:BEGIN kind=q-prompt -->
Q1: Every test name describes expected behavior?
Q2: Tests grouped in logical describe blocks?
Q3: Every mock has CalledWith + not.toHaveBeenCalled?
Q4: Assertions use exact matchers (toEqual/toBe, not toBeTruthy)?
Q5: Mocks are typed (no `as any`)?
Q6: Mock state fresh per test (beforeEach, no shared mutable)?
Q7: CRITICAL -- Every contract negative case tested: throws/rejections type+message; filter/sentinel/fallback exact result? Derive accepted input domain from callers/validation; no invented out-of-domain throw requirements. No negative cases: pass only with exhaustive contract/branch/caller evidence.
Q8: Null/empty/edge inputs tested?
Q9: Repeated setup (3+ tests) extracted to helper/factory?
Q10: No magic values -- test data is self-documenting?
Q11: CRITICAL -- All code branches exercised?
Q12: Symmetric: "does X when Y" has "does NOT do X when not-Y"?
Q13: CRITICAL -- Tests import actual production function?
Q14: Behavioral assertions (not just mock-was-called)?
Q15: CRITICAL -- Content/values assertions, not just counts/shape?
Q16: Cross-cutting isolation: change to A verified not to affect B?
Q17: CRITICAL -- Assertions verify COMPUTED output, not input echo?
Q18: No flaky signals? No Date.now() without fake timers, no setTimeout for timing, no Math.random(), no real network?
Q19: Tests fully isolated? No shared mutable state between tests; each runs independently in any order?
Q20: CONDITIONAL -- Test level declared (small/medium/large) and not mixed?
Q21: CONDITIONAL -- Mutation score >= 70% on changed files, or every survivor triaged?
Q22: CONDITIONAL -- Pure/validator unit has a property test with a recorded seed?
Q23: CONDITIONAL -- Cross-service contract verified against a shared artifact, not a hand-written mock?
Q24: CONDITIONAL -- Suite green under randomized order, seed logged?
Q25: CONDITIONAL -- Patch coverage >= 90% on changed lines, enforced in CI?
<!-- GATES:END kind=q-prompt -->
ANTI-PATTERNS (each unique AP = -1 from score, max -5):
<!-- GATES:BEGIN kind=ap-list -->
AP1: try/catch in test swallowing errors
AP2: Conditional assertions (if/else in test)
AP3: Re-implementing production logic in test
AP4: Snapshot as only test for component
AP5: `as any` -> `as never` bypassing types
AP6: Testing CSS classes instead of behavior
AP7: .catch(() => {}) swallowing errors
AP8: document.querySelector bypassing Testing Library
AP9: Always-true assertion (expect(true).toBe(true))
AP10: Tautological mock (call mock -> verify mock called, no production code)
AP11: vi.mocked(vi.fn()) -- mock targeting fresh fn
AP12: waitForTimeout(N) hardcoded delays
AP13: Test with zero expect() calls -- AUTO TIER-D
AP14: toBeTruthy()/toBeDefined() as sole assertion on complex object
AP15: Testing private methods directly
AP16: Fixture:assertion ratio > 20:1 -- AUTO TIER-D
AP17: Unused test data declared but never used
AP18: Duplicate test names (copy-paste indicator)
AP19: expect.anything() hiding callback contract
AP20: Mock returns same data for ALL methods
AP21: .calls[N] magic index (fragile)
AP22: CSS selector in test
AP23: Inline mockRestore() with afterEach present (redundant)
AP24: consoleSpy typed as `any`
AP25: `expect(x.length).toBe(N)` instead of `.toHaveLength(N)` — worse failure output, masks a missing property (Q4). JS-only: pytest/Go/Rust have no equivalent, mark N/A there
AP26: Real timers in time-dependent tests (Date.now/setTimeout without useFakeTimers)
AP27: `expect(x.length).toBeGreaterThan(0)` when the fixture's exact count is known — masks off-by-one and duplicates (Q4/Q15)
AP28: Persistent `it.skip`/`describe.skip`/`@Ignore`/`#[ignore]`/`@pytest.mark.skip` with no ticket or expiry — dead code plus a silent coverage gap
AP29: Mock return value echoed in the assertion — proves the mock setup, not production logic (Q17). The most common audit failure
AP30: Mocking own code that could run with a real implementation (was AP25 until the numbering fork was resolved; overlaps `fix-tests` P-68 and Q13)
AP31: Committed focus marker — `it.only`/`describe.only`/`test.only`/`fit`/`fdescribe`/`@pytest.mark.only`-style filter left in a committed test file. Silently disables the REST of the suite while CI reports green — a whole-suite coverage collapse, worse than a skip. Deterministic detector: grep / Biome `noFocusedTests`. **AUTO TIER-D**
AP32: Flake-masking retries — per-test/per-suite `retries:`/`@Retry`/`flaky=True` annotation with no ticket or expiry (same contract as AP28's skip rule). Retry hides the nondeterminism Q18/Q24 exist to surface
<!-- GATES:END kind=ap-list -->
N/A HANDLING: N/A items excluded from both numerator and denominator. Score = passed / applicable.
Q16 N/A: score N/A when test covers single function/hook with no shared mutable state.
Q17 PASS-THROUGH: For thin controllers that are pure delegation, `expect(result).toEqual(mockReturn)` with CalledWith on service mock = Q17=1.
CRITICAL GATE: Q7, Q11, Q13, Q15, Q17 -- any = 0 -> Tier C floor (see TIER CLASSIFICATION).
SCORING MATH:
Applicable = 25 - N/A-count - out-of-scope-count
Passed = count(Q score == 1)
Deduction = min(5, count(unique AP IDs))
Adjusted = max(0, Passed - Deduction)
Score = Adjusted / Applicable (percentage); Applicable == 0 => INCOMPLETE
If Applicable == 0, status=INCOMPLETE and tier=none; skip numeric classification.
Report Passed, Deduction, Adjusted, N/A, out-of-scope and Applicable separately.
ONE SCALE ONLY: the percentage below is the verdict. Do not also compare raw counts —
that produced two answers for one file (14/17 was simultaneously "PASS" and "Tier B")
and left score 9 belonging to no tier at all.
### Scoring Q21 — read the number, never estimate it
Q21's text lives in the generated region above; how to *answer* it does not, so it lives
here. Source of truth: the newest `$ZUVO_DIR/audits/mutation-test-*.json` written by
`zuvo:mutation-test` (§4.3b of that skill).
| Condition | Q21 value |
|---|---|
| `plan_completed: false` | `N/A (mutation run incomplete — <executed>/<planned>)` — check this FIRST |
| JSON present, `commit` == HEAD sha7 | score from **`score_triaged`** |
| JSON present, `commit` != HEAD | `N/A (mutation data STALE — <json sha7>, HEAD is <sha7>)` |
| JSON present, `tier2_ran: false` | score it, and append `(--quick: survivors never checked against the full suite)` |
| No JSON at all | `N/A (no mutation run)` — legitimate; Q21 is CONDITIONAL on a runner existing |
**Per-file evidence is mandatory.** Verify that the artifact's changed-tree identity, production
file path, executed mutation rows and symbol scope match this audit. A current HEAD alone is
insufficient when the working tree differs from the recorded run. Prefer the matching per-file
score; an aggregate score is valid for a file only if that artifact's entire measured scope is
that file. Mutations of a parser helper do not certify a ranking or graph algorithm that imports
it. If the requested file/symbol has no attributable completed mutation evidence, use
`N/A (no matching mutation evidence for <file:symbol>)`, never borrow another file's score.
Native/hybrid and LLM results retain their measured scope and engine labels.
Use `score_triaged`, never `score_raw`. An *equivalent mutant* cannot be killed by any
test, so counting it against the suite fails work nobody can fix — and a gate people
cannot satisfy is a gate people learn to route around.
`plan_completed` is checked before anything else because a truncated run produces a score
that looks exactly like a complete one. A user hit this on 2026-08-10: the budget stopped
the run after 3 of 10 mutants and a score was printed anyway. A sample scored as the whole
plan is worse than no data — no data is honestly `N/A`.
Until 2026-08-09 `zuvo:mutation-test` wrote nothing to disk: the report went to chat and
vanished. This gate asked for a number the system never produced in readable form, so the
only available answers were a guess or `N/A`. Estimating it from test-file appearance is
not a third option — if the JSON is absent or stale, say so and score `N/A`.
FOR AUTO TIER-D FILES, use SHORT format:
### [filename]
Production file: [path or ORPHAN]
Red flags: [AP13/AP14/AP16] -> AUTO TIER-D
Phantom mocks: [list mocked modules not called by production code, or "none"]
Reason: [brief]
Top 3 gaps: [brief]
FOR ALL OTHERS, use FULL format:
### [filename]
Production file: [absolute branch/worktree path or ORPHAN]
Evidence scope: [production symbols; matching test paths; commit and dirty-tree identity]
Verification: [command, cwd, run ID/artifact, exit/result summary, skips/unrun checks]
Complexity: [THIN/STANDARD/COMPLEX] ([LOC] LOC, [N] branches)
Red flags: ["none"]
Phantom mocks: [list or "none"]
Untested methods: [list of public methods with no test coverage, or "all covered"]
Score: Q1=[0/1] Q2=[0/1] ... Q25=[0/1]
Anti-patterns: [AP IDs found, or "none"]
Total: passed=[N], N/A=[N], out-of-scope=[N], applicable=[N], AP deduction=[N], adjusted=[N]/[applicable] ([%])
Critical gate: Q7=[0/1] Q11=[0/1] Q13=[0/1] Q15=[0/1] Q17=[0/1] -> [PASS/FAIL]
Tier: [A/B/C/D]
Top 3 gaps: [brief]
TIER CLASSIFICATION (derived from the percentage above — no separate count scale):
A (>= 82%, all critical gates = 1): No action needed
B (>= 53% and < 82%, all critical gates = 1): Fix gaps -- 2-5 targeted fixes
C (< 53%, OR any critical gate = 0): Major rewrite needed
D (AUTO TIER-D red flag: AP13, AP16, or AP31 (committed `.only`/focus marker — it disables the REST of the suite while CI stays green, so it is the most destructive of the three)): Delete and rewrite from scratch
A very low score without an auto red flag remains Tier C; D is the specified red-flag classification, not a second raw-count threshold.
A critical gate at 0 is a FLOOR (Tier C), not a ceiling. Previously it "capped at Tier B",
so a tautological suite scoring 16/17 landed in the same bucket as an honest 10/17 — and
test quality punished a critical failure LESS than code quality does (where critical = 0
is a hard FAIL). Now the two families agree.
IMPORTANT:
- Read BOTH the test file AND its production file
- Red flag pre-scan first
- COVERAGE COMPLETENESS: List all public methods in production file. For each, check if test exercises it. Flag untested methods. Exclude control flow keywords, built-ins, SQL keywords. API endpoint exception: test calling client.get("/path") IS testing the handler. Page component exception: render(<Component />) IS testing the export. Re-export exception: only test functions DEFINED in the file, not re-exports.
- PHANTOM MOCK DETECTION: List all mocked modules in test. For each: does production code actually call it? Unused mock = phantom mock.
- SUITE-AWARE MODE: Sibling test files for the same production file may supply Q7/Q11 evidence; name each contributing test and actual production branch. A helper test is not evidence for its caller unless it executes and asserts that caller behavior. Coverage/mutation claims require matching file/symbol and tree identity.
- Q7 DOMAIN: List accepted inputs and every specified reject/filter/sentinel/fallback path, with production branch locations and exact test assertions. Do not invent throws for out-of-domain typed null; malformed inputs at real untrusted boundaries still require testing.
- Q17 ECHO vs COMPUTED: mock returns X, test asserts X = echo (Q17=0). Mock returns raw data, test asserts transformation = computed (Q17=1).
- Q15 API ROUTE CALIBRATION: Status code checks, error body checks, response field checks, auth guard verification all count as Q15=1 for API routes.
- AP21 CALIBRATION: `.mock.calls[N]` = fragile (AP21). `.toHaveBeenNthCalledWith(N, ...)` = Jest API, not AP21.
Write complete output to: zuvo/audits/.test-audit-batch/batch-{N}.md
Files to audit:
[BATCH FILE LIST]
Phase 2: Aggregate Results
Read all batch files from zuvo/audits/.test-audit-batch/:
- Glob for
zuvo/audits/.test-audit-batch/batch-*.md - Parse summary tables for tier counts
- Parse per-file blocks for detailed analysis
- If any batch file is missing (agent failure), log the gap
- Count INCOMPLETE files separately; exclude them from numeric tier averages and passing totals, and keep the overall audit INCOMPLETE until their evaluation is resolved
Build the summary report:
# Test Quality Audit Report
Date: [date]
Project: [name]
Files audited: [N]
Total tests: [count from test runner]
Checkout: [absolute branch/worktree path]
Tree: [branch, commit, dirty-tree identity if applicable]
Verification: [commands + cwd + run IDs/artifacts + exit/result summaries; skips and unrun checks]
Evidence map: [per production file/symbol → test branch citations → matching coverage/mutation artifact scope]
## Summary by Tier
| Tier | Count | % | Action |
|------|-------|---|--------|
| A (>=82% of applicable) | [N] | [%] | No action |
| B (>= 53% and < 82% of applicable) | [N] | [%] | Fix gaps |
| C (<53% of applicable OR any critical gate = 0) | [N] | [%] | Major rewrite |
| D (auto Tier-D red flag) | [N] | [%] | Delete + rewrite |
| INCOMPLETE (no tier) | [N] | [%] | Finish missing evaluation |
| ORPHAN | [N] | [%] | Verify or delete |
## Critical Gate Failures
| File | Score | Failed Qs | Top Gap |
|------|-------|-----------|---------|
## Red Flag Summary (Auto Tier-D)
| File | Red Flag | Details |
|------|----------|---------|
## Untested Public Methods
| File | Untested Methods | Impact |
|------|-----------------|--------|
## Top Failed Questions (across all files)
| Question | Fail count | % of files | Pattern |
|----------|-----------|------------|---------|
## Anti-pattern Hot Spots
| Anti-pattern | Files affected | Instances |
|-------------|---------------|-----------|
## Tier D -- Rewrite Queue
## Tier C -- Major Fix Queue
## Tier B -- Targeted Fix Queue
## Tier A -- No Action
Save to: zuvo/audits/test-quality-audit-[date].md — at the project root (zuvo/ resolves via git rev-parse --show-toplevel; override $ZUVO_OUTPUT_DIR. See ../../shared/includes/report-output-location.md).
If --details flag: also save per-file reports to zuvo/audits/test-audit-details/
Phase 3: Cleanup Batch Files
rm -rf zuvo/audits/.test-audit-batch
Phase 3b: Adversarial Review on Audit Report (MANDATORY — do NOT skip)
After the audit report is generated, run cross-model validation to catch Q-score inflation and coverage theater. Runs on ALL audits (not just --deep).
~/.zuvo/adversarial-review --mode tests --files "zuvo/audits/test-quality-audit-[date].md"
If adversarial-review is not in PATH: ~/.zuvo/adversarial-review (stable; the versioned cache path breaks after any release)
Wait for complete output. Verify each actionable finding against the actual source/test branches and artifact scope before changing scores. Record rejected false positives with source evidence; severity alone does not establish validity. Then, for confirmed findings:
- CRITICAL (passing Q-score contradicted by evidence) → fix in report before delivery
- WARNING (coverage theater not flagged) → correct affected evidence, scores and tier; record unresolved gaps explicitly
- INFO → ignore
Apply validated corrections to the audit report, including per-file scores, aggregate counts and evidence attribution, before delivery. Do not start a recursive review loop of report corrections unless the user requested one. The existing mandatory review is one bounded review of the report; record its findings and dispositions. Link the actual branch/worktree files so reviewers can open the source that was audited.
Phase 4: Coverage Registry Update
Under --read-only: skip Phases 4-6 entirely and present the report (Phase 7 fix workflow is also off).
Read memory/coverage.md. If it does not exist, create it now.
For each audited test file, find its production file row in coverage.md:
| Audit Tier | Coverage Status | Rationale |
|---|---|---|
| A (>=82% of applicable, critical gates PASS) | COVERED | Tests are solid |
| B (>=53% and <82% of applicable, critical gates PASS) | PARTIAL-QUALITY | Has tests but quality issues |
| C (<53% of applicable OR any critical gate = 0) | PARTIAL-QUALITY | Major quality gaps |
| D (auto Tier-D red flag) | PARTIAL | Effectively untested |
| INCOMPLETE (no tier) | Leave existing row unchanged | Evaluation unavailable; never register as COVERED |
Only downgrade coverage status, never upgrade. If production file is not yet in coverage.md, add it.
Output: COVERAGE UPDATE: [N] rows updated ([N] downgraded, [N] confirmed, [N] new)
Phase 5: Backlog Persistence
Persist findings to memory/backlog.md:
- Read
memory/backlog.md. If missing, create with template. - Fingerprint each finding:
file|Q/AP-id|signature. Dedup: existing = incrementSeen. - Delete resolved items.
Full protocol: ../../shared/includes/backlog-protocol.md.
What to persist:
- Tier C/D files: all findings. Source:
test-audit/{date}. Category: Test. - Tier B critical gate failures (Q7/Q11/Q13/Q15/Q17=0): separate item per gate
- Auto Tier-D red flags (AP13/AP14/AP16): always persist as HIGH
Phase 6: Persistence Verification
Before presenting the report, verify all writes completed:
PERSISTENCE VERIFICATION
coverage.md updated: [N] rows ([N] downgraded, [N] confirmed, [N] new)
backlog.md updated: [N] entries ([N] new, [N] deduped)
batch files cleaned: [yes/no]
If any step is incomplete, go back and finish it before continuing.
Phase 7: Post-Audit Fix Workflow
After presenting the report, the user may request fixes:
- Fix -- rewrite test files following the quality rules
- Test -- run the test suite to confirm all tests pass
- Verify -- for each fixed file:
- Only test files modified (no production code changes)
- Full test suite green
- All modified test files <= 400 lines
- Q1-Q25 self-eval on each fixed file
- Tier improvement confirmed (D->C+, C->B+, B->A)
- Commit -- behavior per
--commitflag (ask/auto/off) - Re-audit -- optionally re-run on fixed files to verify improvement
Next-Action Routing
| Finding | Action | Command |
|---|---|---|
| Tier D files (any AUTO TIER-D red flag, or a critical Q gate = 0) | Rewrite tests | zuvo:write-tests [path] |
| Same AP across 10+ files | Batch fix | zuvo:fix-tests --pattern [AP-ID] |
| Tier B-C with Q7=0 | Add error tests | zuvo:write-tests [path] |
| Coverage gaps (methods untested) | Write missing tests | zuvo:write-tests [path] |
| Test infra issues (runner config) | Optimize runner | zuvo:tests-performance |
Completion Gate Check
Before printing the final output block, verify every item. Unfinished items = pipeline incomplete.
COMPLETION GATE CHECK
[ ] Red flag pre-scan ran on every batch
[ ] Phantom mock detection ran: unused mocks listed
[ ] Untested public methods listed per file
[ ] Adversarial review ran on audit report
[ ] Coverage registry updated: memory/coverage.md rows written
[ ] Backlog updated for critical gate failures
[ ] Report saved to zuvo/audits/
[ ] Run: line printed and appended to log
TEST AUDIT COMPLETE
Validity Gate (REQUIRED — print BEFORE Run line, AFTER retro append + append-runlog)
VALIDITY GATE
triggers_held: language=<X> framework=<X> test_runner=<X>
required_tool_calls:
find_dead_code: [<N> orphan helpers | NOT_CALLED — VIOLATES_TRIGGER]
find_clones: [<N> dup tests | NOT_CALLED — VIOLATES_TRIGGER]
find_references: [<N> ref-checks | NOT_CALLED — VIOLATES_TRIGGER]
search_patterns: [<N> hits | NOT_CALLED — VIOLATES_TRIGGER]
audit_scan: [<N> findings | NOT_CALLED — VIOLATES_TRIGGER]
scan_secrets: [<N> hits | NOT_CALLED — VIOLATES_TRIGGER]
stack_specific: [<result> | not_required | NOT_CALLED — VIOLATES_TRIGGER]
postamble:
retros_log_appended: [yes(bytes_added=N) | NOT_APPENDED]
retros_md_appended: [yes(entry_count=N) | NOT_APPENDED]
verify_audit_pass: [yes(<verified>/<total>) | NOT_RUN | REJECTED]
gate_status: [PASS | FAIL — <which gates missing>]
If gate_status = FAIL → VERDICT = INCOMPLETE.
Append the Run line via the retro-gated wrapper (NOT direct >> runs.log):
printf '%b\n' "$RUN_LINE" | ~/.zuvo/append-runlog
Run: \ttest-audit\t\t\t\t\t-\t-dimensions\t\t\t\t\t
Retrospective (REQUIRED)
Follow the retrospective protocol from retrospective.md.
Gate check → structured questions → TSV emit → markdown append.
If gate check skips: print "RETRO: skipped (trivial session)" and proceed.
After printing this block, append the Run: line value (without the Run: prefix) to the log file path resolved per run-logger.md.
VERDICT: PASS (0 critical findings), WARN (1-3 critical), FAIL (4+ critical).
Execution Notes
- Use Sonnet for batch agents in both QUICK and DEEP modes
- Process batches sequentially in Cursor/Codex. Claude Code may parallelize with up to 6 Task agents.
- Run the project's test suite first to confirm baseline passes. Auto-detect runner from config files.
- Estimated durations: QUICK ~2 min for 50 files, DEEP ~10 min for 50 files