zuvo:mutation-test -- LLM-Guided Mutation Testing
Intelligent mutation testing that targets meaningful behavioral gaps rather than random code changes. For each production file, the LLM generates mutations in 7 categories (boundary, logic, null, error, state, async, security), runs only the tests that cover that file, and closes the gaps it finds: every survivor is triaged as a real gap or an equivalent mutant, and each real gap gets the missing assertion added and re-probed in the same run.
Scope: Production files that have associated test files. Measures how well existing tests detect real behavioral changes.
When to use: After writing tests, before releases, when mutation score is unknown, when test suite feels shallow despite high line coverage.
Out of scope: Writing a test suite from scratch (use zuvo:write-tests), fixing systematic test
anti-patterns across many files (use zuvo:fix-tests), auditing test quality without execution (use
zuvo:test-audit), code quality review (use zuvo:review). Adding the single missing assertion that
a surviving mutant exposes is IN scope — that is the point of finding it.
Argument Parsing
Parse $ARGUMENTS as: [path | full | continue] [--max N] [--category CATEGORY] [--runner MODE] [--break N] [--no-install] [--dry-run] [--quick] [--report-only]
| Flag | Env equivalent | Effect |
|---|---|---|
[path] |
— | Scope to a specific directory or file |
continue |
— | Resume an interrupted run from its checkpoint (Phase 3.0). Re-runs nothing already resolved. |
full |
— | All production files that have test coverage |
--max N |
— | Max total LLM mutations to execute (default: 50). Does not bound the native runner, which mutates exhaustively by design. |
--category CATEGORY |
— | Only generate LLM mutations of this category: BOUNDARY, LOGIC, NULL, ERROR, STATE, ASYNC, SECURITY |
--runner MODE |
ZUVO_MUTATION_RUNNER |
auto (default) / native / llm / hybrid — see 0.1d |
--break N |
ZUVO_MUTATION_BREAK |
Fail the run when the final score_triaged is below N% (4.2c — evaluated after the 4.2b fix loop, not at 4.1). The flag overrides the env value; both are recorded with their source. |
--no-install |
ZUVO_MUTATION_NO_INSTALL=1 or ZUVO_NO_INSTALL=1 |
Never offer to install a mutation runner (disables the 0.1c consent gate). Detection still runs. |
--dry-run |
— | Generate mutations and show the plan, but do not execute any. Also suppresses 0.1c entirely. |
--quick |
— | Max 3 LLM mutations per file, max 20 total |
--report-only |
— | Report the score; do NOT fix surviving gaps (skips 4.2b). The only way to skip the fix loop. |
Flags can be combined: zuvo:mutation-test src/services/ --max 30 --category SECURITY
Default (no arguments): the CHANGED production files, not the whole project —
git diff --name-only $(git merge-base HEAD <default-branch>)..HEAD plus uncommitted production
files, filtered to those that have tests. Then --max 50 --runner auto. If that set is empty, say
so and stop; do NOT silently widen to everything.
full still exists and still means every covered production file — it just has to be asked for.
Why the default moved. Measured 2026-08-28 across the local fleet: the median mutation run is 1 minute and normal scoped runs top out around 31 minutes, but 32 runs exceeded an hour and the longest reached 258 minutes — all from a main checkout, all unscoped, mutating a 2,784-file test suite to answer a question about one module. Nothing was learned in those four hours that the scoped one-minute run would not have said, and each one occupied a farm slot. A default that costs four hours when the user meant "check what I just changed" is a defect in the default, not in the farm that ran it.
Callers must pass a scope. zuvo:refactor and zuvo:write-tests probe mutation per file
(--mutate <file>) and are unaffected. Any skill invoking this one must name its file set; an
unscoped invocation from another skill is a bug in that skill.
Mandatory File Loading
First resolve ../../shared/includes/execution-policy.md and
../../shared/includes/evidence-reuse.md. Reuse the parent policy and verified evidence for
a nested stage. Load only the rules needed now, with the read-once receipt protocol.
Load the applicable definitions using the read-once protocol. Defer logging/retro includes until completion.
CORE FILES LOADED:
1. ../../rules/testing.md -- READ/MISSING
2. ../../rules/testing.md (M1-M5 + Assertion Strength + Self-Eval Evidence) -- READ/MISSING
3. ../../shared/includes/env-compat.md -- READ/MISSING
4. ../../shared/includes/run-logger.md -- READ/MISSING
5. ../../shared/includes/retrospective.md -- READ/MISSING
6. ../../shared/includes/report-output-location.md -- READ/MISSING (canonical $ZUVO_DIR for 4.3b)
7. ../../shared/includes/terminal-state.md -- READ/MISSING (HARD: no completion over a live runner)
If any file is missing: Proceed in degraded mode. Note "DEGRADED -- [file] unavailable" in the final report.
Environment Compatibility
Dispatch follows the resolved execution policy. Record actual independence and any unavailable gate; session restrictions take precedence.
Read ../../shared/includes/env-compat.md for agent dispatch patterns, path resolution, and progress tracking across all supported platforms.
CodeSift Integration
Read ../../shared/includes/codesift-setup.md for the full initialization sequence.
Key tools for this skill:
| Phase | Task | CodeSift tool | Fallback |
|---|---|---|---|
| 0 | Find production files | get_file_tree(repo, file_pattern=<detected_ext>) |
Glob with detected extension |
| 0 | Find test files | get_file_tree(repo, name_pattern=<detected_test_pattern>) |
Glob with detected test pattern |
| 0 | Understand file structure | get_file_outline(repo, file_path) |
Read the file |
| 0 | Detect complexity hotspots | analyze_complexity(repo, top_n=20) |
Line count heuristic |
| 2 | Read production code for mutation targeting | get_symbol(repo, symbol_id) |
Read the file |
| 2 | Batch-read multiple functions | get_symbols(repo, symbol_ids=[...]) |
Multiple Read calls |
| 2 | Find references to identify test coverage | find_references(repo, symbol_name) |
Grep for imports |
Phase 0: Discovery
Detect the project's test infrastructure and build the production-to-test file map.
0.1 Framework Detection
Detect the test framework and runner from config files:
| Signal | Framework | Runner command |
|---|---|---|
jest.config.* or "jest" in package.json |
Jest | npx jest |
vitest.config.* or "vitest" in package.json |
Vitest | npx vitest run |
pytest.ini, pyproject.toml [tool.pytest], conftest.py |
Pytest | pytest |
phpunit.xml |
PHPUnit | vendor/bin/phpunit |
_test.go files |
Go testing | go test |
*_spec.rb files |
RSpec | bundle exec rspec |
*.test.rs or #[cfg(test)] |
Rust | cargo test |
If framework cannot be detected: ask the user for the test runner command.
0.1b Native mutation runner — DETECTION (read-only)
Every stack this skill supports has a maintained mutation runner. A project that already
configured one has a reproducible score this skill must READ rather than re-derive; the
LLM engine measuring a repo that ships stryker.conf.json with its own hand-rolled
mutants was the gap this step closes.
Detection touches nothing. It reads config files and manifests. No install, no write,
no command execution beyond a --version probe.
| Stack | Runner | Config signals (any one) | Scoped run | Machine-readable report |
|---|---|---|---|---|
| TS / JS | StrykerJS ≥ 10 | stryker.conf.{json,js,mjs,cjs}, stryker key in package.json, @stryker-mutator/core in devDependencies |
npx stryker run --incremental --mutate <files> |
--reporters json |
| PHP | Infection ≥ 0.35 | infection.json, infection.json5, infection.json.dist, infection/infection in composer.json require-dev |
vendor/bin/infection --threads=max -- <paths> (POSITIONAL — --filter is deprecated since 0.34) |
logs.json key in infection.json (there is no --logger-json CLI flag) |
| Python | mutmut ≥ 3.7 | [tool.mutmut] in pyproject.toml, [mutmut] in setup.cfg |
mutmut run — whole project only, see the mutmut note below |
mutmut results |
| JVM (Java, Kotlin) | PIT ≥ 1.25 | pitest plugin in pom.xml / build.gradle{,.kts} |
mvn test-compile org.pitest:pitest-maven:mutationCoverage (the test-compile phase is required — a bare goal has no compiled classes to mutate) or ./gradlew pitest |
outputFormats=XML |
| Rust | cargo-mutants ≥ 27 | .cargo/mutants.toml, or cargo mutants --version succeeds |
cargo mutants -f <file> |
--json (mutants.out/) |
| .NET / Scala | Stryker.NET / Stryker4s | stryker-config.{json,yaml} |
per tool docs | Stryker JSON |
mutmut 3.x cannot scope a run from the CLI. --paths-to-mutate was a mutmut 2.x flag and was
REMOVED in 3.x — the error message still suggests it, which is how it survives in documentation
that was never re-tested. In 3.x the mutated paths come only from [tool.mutmut] paths_to_mutate
in pyproject.toml / setup.cfg. Consequences, and they are not cosmetic:
- A scoped native run on Python means editing the project's mutmut config, which is a write
outside test files and therefore belongs to the 0.1c consent gate — not to a per-file loop.
Until that is wired, treat Python native runs as whole-project only: run
mutmut runas the project configures it, and take the score for the whole project rather than claiming a per-file number the tool cannot produce. test-mutation-probes.md's per-file native path is therefore unavailable for mutmut. That include says so explicitly; a Python project falls back to the hand-picked probes there.
Shell-quote every interpolated path. These run commands are templates with <file>
placeholders, and a repository controls its own filenames — a path holding a space, a quote or a
; becomes argument injection the moment a template is pasted into a shell. Quote the
substitution (--mutate '<file>') or pass the file list as separate argv entries; never build
the command by string concatenation and hand it to bash -c.
Versions are MINIMUMS, not pins — an older configured runner is still used, with its
version recorded. A runner present but BELOW the minimum is used and flagged
native_runner.state: detected (below-minimum <ver>); do not silently upgrade a project's
tooling to satisfy a floor written here.
Record on the run and in the 4.3b artifact:
native_runner: { name, version, config_path, state }
state ∈ detected | installed | absent | declined | unsupported_stack | failed
0.1c Consent-gated install (skipped by --no-install, either no-install env var, and --dry-run)
Mirrors the DD-3 consent gate in
skills/infra-audit/SKILL.md— offer, consent, log with the uninstall command, degrade loudly on decline. The wording is deliberately parallel so the two can later be extracted into one shared include; they are not shared today, and unifying them is a separate change to a separate skill.
ZUVO_MUTATION_NO_INSTALL=1is this skill's own switch.ZUVO_NO_INSTALL=1is honoured as a broader user policy, but only this skill reads it today — do not describe it to a user as a machine-wide guarantee.
Fires only when ALL hold: no runner detected, the stack HAS one, none of the suppressors above
is set, and the engine is not llm. --runner llm says the run will not touch a native
runner at all — prompting to install one it has already declined to use is a prompt for nothing,
and a consent prompt that appears when it cannot matter is how consent stops being read.
One prompt per run — never per file.
This is the only step in this skill that writes outside test files, so it is fenced harder than anything else here:
Print both commands before asking. The exact install command AND the exact uninstall command, verbatim, so the consent is informed and reversible in one line.
Dev scope only, and use the project's OWN package manager. Never a runtime dependency, never
-g/global for a project-scoped manager. Detect the manager from its lockfile (pnpm-lock.yaml→ pnpm,yarn.lock→ yarn,package-lock.json→ npm) and use the matching row — runningnpm iin a pnpm or yarn workspace writes a competingpackage-lock.jsonand a nestednode_modules, which is a lasting mess left behind by a tool that was only measuring:Manager Install Uninstall npm npm i -D @stryker-mutator/core @stryker-mutator/<jest|vitest|mocha>-runner(pick the plugin matching the framework detected in 0.1)npm rm @stryker-mutator/core @stryker-mutator/<…>-runnerpnpm pnpm add -D @stryker-mutator/core @stryker-mutator/<…>-runnerpnpm remove @stryker-mutator/core @stryker-mutator/<…>-runneryarn yarn add --dev @stryker-mutator/core @stryker-mutator/<…>-runneryarn remove @stryker-mutator/core @stryker-mutator/<…>-runnercomposer composer require --dev infection/infectioncomposer remove --dev infection/infectionuv uv add --dev mutmutuv remove --dev mutmutpip pip install mutmutpip uninstall -y mutmutcargo cargo install cargo-mutants— user-global toolchain, NOT the project'sCargo.toml. Say that in the prompt.cargo uninstall cargo-mutantsJVM is MANUAL and is never installed. PIT is a build-plugin edit to
pom.xml/build.gradle.kts— a change to how the project builds, not a dev dependency. Print the exact snippet, do NOT edit the build file, and recordstate: absent (manual wiring required). A skill that edits a build file to measure test quality has exceeded its mandate.Config: generate a MINIMAL config only when none exists, only for the consented tool, and only enough to run — no opinionated thresholds, no ignore-lists.
Everything written is reported.
installed_this_run[]in the 4.3b artifact carries{tool, version, command, uninstall, files_touched[]}, and the completion block prints the uninstall line. A manifest or lockfile modified without appearing there is a bug.Failure and decline are loud, never silent. Declined →
state: declined. Install command exits non-zero →state: failedwith its stderr. Both fall back to the LLM engine and label the runnative: skipped (<reason>). A run that wanted native and got LLM must never report as if it had a native score.
Consent is a human decision and must stay one. --no-install only ever makes a run
more conservative, so an agent may set it freely; there is deliberately no flag that
grants consent, because a flag an agent can type is not consent
(no-agent-typable-bypass). Non-interactive hosts therefore take the declined path.
0.1d Engine selection (--runner)
| Mode | Behaviour |
|---|---|
auto (default) |
Native if detected or installed; otherwise LLM. |
native |
Native only. Unavailable → ABORT with BLOCKED_NO_NATIVE_RUNNER naming why. Never silently falls back — that is the whole point of asking for it. |
llm |
The LLM engine only. This is the guaranteed floor and is unchanged from previous versions. |
hybrid |
Native for the score, PLUS LLM mutations restricted to ERROR, STATE, ASYNC, SECURITY — the classes syntactic mutators do not generate. Report both numbers separately; never average them, they measure different mutant populations over the same code. |
Two invariants hold in every mode:
- Phase 4.2b still runs. A native survivor is triaged and closed in-run exactly like an LLM survivor. A native runner's HTML report is a hand-off, and handing off is the drift this skill exists to stop.
- The LLM engine is never removed. It is the fallback for
absent,declined,failed, andunsupported_stack, and it is the only engine for the four categories above.
0.2 File Mapping
Build a map of production files to their test files. Discovery patterns are language-aware — use the detected framework from 0.1:
| Language | Production ext | Test patterns |
|---|---|---|
| TypeScript/JavaScript | *.ts, *.tsx, *.js, *.jsx |
*.test.*, *.spec.*, __tests__/* |
| Python | *.py |
test_*, *_test.py, tests/ |
| PHP | *.php |
*Test.php, tests/ |
| Go | *.go (non-test) |
*_test.go |
| Ruby | *.rb |
*_spec.rb, spec/ |
| Rust | *.rs (non-test) |
#[cfg(test)] blocks, tests/ |
For each detected language:
- Scan for all test files using the language-specific patterns
- For each test file, identify the production file it covers:
- By import/require statements in the test
- By naming convention (
foo.ts->foo.test.ts,foo.py->test_foo.py) - By directory convention (
src/foo.ts->__tests__/foo.test.ts)
- Build the map:
{ production_file: [test_file_1, test_file_2, ...] } - Exclude production files with no test coverage (nothing to validate mutations against)
If no language matches or discovery produces 0 files: ask the user for the file patterns.
0.3 Prioritization
Order files for mutation testing by priority:
- Critical paths first: Files matching keywords:
auth,login,session,token,payment,billing,charge,transaction,password,encrypt,decrypt,sanitize,validate,permission,role,access - High complexity: Files with the most functions, branches, or cyclomatic complexity
- Recent changes: Files with commits in the last 30 days (active development = higher risk)
- Everything else: Alphabetical
If --category SECURITY is set, promote files matching security-related keywords to the top.
Output:
DISCOVERY
Framework: [name] | Runner: [command]
Native runner: [name vX.Y | none] ([detected <path> | installed | absent | declined | failed | unsupported_stack])
Engine: [auto->native | auto->llm | native | llm | hybrid]
Production files with tests: [N]
Files excluded (no tests): [N]
Priority order: [top 5 files listed]
Scope: [path or "full project"]
Max mutations: [N]
If --quick: reduce max mutations per file to 3, total to 20.
Phase 1: Baseline
Establish that all tests pass before introducing mutations.
1.1 Run Full Test Suite
Execute the detected test runner command against the scoped files:
# Examples:
npx jest --passWithNoTests # Jest
npx vitest run # Vitest
pytest # Pytest
go test ./... # Go
1.2 Validate Baseline
- All tests pass: Record the total execution time. Proceed to Phase 2.
- Any test fails: STOP immediately. Do not proceed with mutation testing.
If tests fail:
BASELINE FAILED
[N] test(s) failing
Cannot run mutation testing against a failing test suite.
Suggestion: run zuvo:fix-tests to repair failing tests first.
1.3 Send the TIER 1 loop to the farm as ONE invocation — never per-mutant, never local
Wrap the LOOP, not each mutant. The wrapper's cost is a fixed per-invocation charge (mirror sync + queue), and Tier 1 makes N short invocations — so wrapping each one multiplies the charge by N, while wrapping the loop pays it once:
rt --light bash -c '<the whole mutant loop>'
An earlier version of this section concluded "therefore run Tier 1 locally". That was the
wrong lesson from a correct measurement, and 2026-08-29 measured what it costs: 109 local test
processes at 421% CPU, load 34, macOS suspending the workstation with Dark Wake Thermal Emergency, and a native mutation run dying ten minutes in when a concurrent worktree pulled
shared node_modules out from under it — while the farm sat idle with ~18 free slots. Nothing
runs on the workstation.
Measured 2026-08-10 on the same single test file:
| Invocation | Wall clock |
|---|---|
npx vitest run <file> |
1.4 s (2.2 s cold) |
rt npx vitest run <file> |
103.4 s |
~50-75x, and a single wrapped call alone exceeds the whole run budget below. A
10-mutant plan that would finish in ~20 s locally cannot complete a single mutant
through the wrapper. Wrap the LOOP once instead and the same ten mutants pay one charge.
If the farm is unreachable (rt exits 21), this skill is not usable for that run — say so
and stop; never move the loop to the workstation, and never report a wrapper timeout as a
test-quality result.
Tier 2 is the opposite case, and the ban used to swallow it. A Tier 2 pass is ONE
long full-suite run — exactly the shape the global rt rule was written for — and the
budget formula in 1.3b already accounts for it separately (TIER2_RUNS * BASELINE_TIME,
never PER_RUN). The routing rule simply did not follow its own split. Measured
2026-08-17 on rs_be, same suite, same commit:
| Tier 2 full suite | Wall clock |
|---|---|
| local, 4 passes in one run | 673 s + 270 s + 323 s + 312 s = 1578 s |
| farm, warm mirror + cache hit | 142–264 s per pass (deps 2–4 s, setup < 0.8 s) |
Route Tier 2 by BASELINE_TIME, measured in 1.2, not by habit:
BASELINE_TIME >= 120s→ run Tier 2 throughrt(rt --lightwhen the suite needs no services). The one-time mirror/queue charge is amortised across a run that long, and it takes the heaviest load off the workstation that is also running the Tier 1 loop.BASELINE_TIME < 120s→ keep Tier 2 local. Below that the wrapper's fixed cost is a large fraction of the run and the farm wins nothing.
Record which side the run took and why: tier2_runner: "rt (baseline 187s)" or
tier2_runner: "local (baseline 41s)". A Tier 2 result whose runner is unstated cannot be
compared against the next run's.
Three rules that do not relax when Tier 2 goes to the farm:
- Restoration stays local and unconditional. The farm run is read-only with respect to
the working tree:
rtships the tree as it stands (mutation applied) and the verdict comes back, but thecprestore in 3.2 step 4 and the hash verify in step 5 happen here, on this machine, exactly as before. Never let a remote step own the restore. - A wrapper failure is not a mutation result.
rtexiting non-zero for a queue timeout, an evicted run, or an unreachable host means the mutation was not measured — it is neither killed nor survived. Follow the execution policy: keep queued work attached on the required runner. A pending run isNOT_EXECUTED, not a terminal refusal or a result to score around. TEST_RANdiscipline applies to the farm too. A farm run that is evicted or never scheduled exits 0 having run nothing, and "0 failures" from a suite that did not execute reads as a SURVIVED mutation — the single most expensive misread this skill can make. Require a summary line proving the suite executed before recording any Tier 2 verdict.
Off-tailnet, rt refuses with exit 21 rather than running locally. That is a routing
failure, not a test result. Follow execution-policy.md; a project restriction on workstation execution requires
waiting/reconnecting to the authorized runner. Never infer permission from a timeout.
1.3b Calculate Timeouts — budget per-invocation cost, not suite time
Measure the real per-invocation cost first. Run the mapped tests for the first
target file ONCE, unmutated, and record wall clock as PER_RUN. That number carries
runner startup, transform and setup — which dominate a short targeted run and are
paid again for every mutation. The full-suite baseline time does NOT predict it.
PER_RUN = measured wall clock of one unmutated targeted run
TIER2_RUNS = expected survivors (unknown up front — budget 30% of MUTATION_COUNT)
BUDGET = MUTATION_COUNT * PER_RUN * 1.5 # tier 1, +50% slack
+ TIER2_RUNS * BASELINE_TIME * 1.5 # tier 2 full-suite passes
Per-file timeout = max(10s, 3 * PER_RUN)
The old formula was 3 * baseline, minimum 60s — it budgeted three suite runs while
the loop actually spends N * (startup + short run). With a 5 s suite the budget was
60 s regardless of whether the plan had 5 mutants or 50, so a plan was aborted
mid-way on a limit that had nothing to do with its size. Reported by a user on
2026-08-10: 7 of 10 mutants dropped, budget consumed by invocation overhead.
If BUDGET looks unreasonable, shrink the PLAN, not the budget — lower --max
or use --quick, and say which. Silently truncating a plan produces a mutation score
computed over a sample the report presents as the whole plan.
Output:
BASELINE
Tests: [N] passing | [N] suites
Baseline time: [N]s (full suite, once)
Per-run cost: [N]s (one targeted run — what each mutation actually costs)
Runner: rt (farm) (the whole loop in ONE invocation — see 1.3)
Plan: [N] mutations -> budget [N]s
Phase 2: Mutation Generation
For each production file (in priority order from Phase 0), generate intelligent mutations.
2.1 Read Production Code
Read the full production file. Identify:
- Functions, methods, and their signatures
- Conditional branches (if/else, switch, ternary)
- Guard clauses and validation
- Error handling (try/catch, throw, reject)
- State mutations and assignments
- Async operations (await, Promise, callback)
- Security-relevant code (auth checks, sanitization, access control)
2.2 Generate Mutations
For each file, generate 5-10 mutations across these categories:
| Category | Tag | Mutation type | Example |
|---|---|---|---|
| Boundary | BOUNDARY |
Off-by-one, < vs <=, >= vs >, +1/-1 on limits |
i < arr.length -> i <= arr.length |
| Logic | LOGIC |
true -> false, && -> ||, negate condition |
if (isValid) -> if (!isValid) |
| Null/empty | NULL |
Return null instead of value, empty array instead of data |
return users -> return [] |
| Error path | ERROR |
Remove try/catch, swap error types, skip validation | Remove if (!input) throw guard |
| State | STATE |
Remove state update, swap assignment values | count += 1 -> count += 0 |
| Async | ASYNC |
Remove await, swap resolve/reject |
await save() -> save() (fire-and-forget) |
| Security | SECURITY |
Remove auth check, skip validation, remove sanitization | Remove if (!user.isAdmin) return 403 |
Mutation quality rules:
- Each mutation must change observable behavior (not just cosmetic)
- Skip trivial mutations: comments, whitespace, logging-only statements, console.log
- Skip mutations in generated code, type definitions, and pure configuration
- Each mutation targets one specific behavioral change
- Prefer mutations at decision points (branches, guards, returns)
If --category is set: Only generate mutations of the specified category.
2.3 Mutation Plan
For each mutation, record:
MUT-NNN: Sequential IDfile: Production file pathline: Line numbercategory: One of BOUNDARY, LOGIC, NULL, ERROR, STATE, ASYNC, SECURITYoriginal: Original code (1-3 lines)mutated: Mutated code (1-3 lines)rationale: Why a test should catch this (1 sentence)test_files: Which test file(s) to run
Cap at --max total mutations (default 50). If more mutations are possible, prioritize by:
- SECURITY mutations (most important to catch)
- ERROR mutations (error paths are commonly under-tested)
- BOUNDARY mutations (off-by-one errors are common and subtle)
- LOGIC, NULL, STATE, ASYNC (remaining categories)
2.3b Anchor every mutation to CONTENT, not to a line number
line is a hint that expires. It is recorded before Phase 3 runs, and by the time a mutation is
applied — or re-applied on resume — the file may have moved underneath it: Phase 4.2b writes tests
and can touch production code, a fix commit lands between passes, or a resumed run meets a file
edited since the plan was made. Applying mutated at a stale line number does not fail loudly; it
corrupts a DIFFERENT statement and the run then measures a mutation nobody designed.
So each mutation carries an anchor that survives movement:
symbol: the enclosing function/method/class name (from CodeSiftget_file_outline, or the nearest preceding definition line when unavailable).original_norm:originalwith leading/trailing whitespace stripped per line and internal runs of whitespace collapsed to one space. This is what you MATCH on — never the raw text, because a formatter run would otherwise invalidate every anchor in the plan.occurrence: 1-based index oforiginal_normwithinsymbol, for the case where the same statement appears more than once in one function.file_sha:git hash-object <file>at plan time.
Resolve before applying, every time:
- If
git hash-object <file>still equalsfile_sha, the plan is current — apply atline. - Otherwise re-locate: find
symbol, then theoccurrence-th match oforiginal_norminside it. Exactly one match → apply there and recordanchor: relocated(<old-line> → <new-line>). - Zero matches, or more than one after the occurrence filter → SKIP this mutation and record
anchor: lost (<reason>). A skipped mutation isnot_run, and per Phase 3.3 a plan that did not run in full is not a score — it must be reported as such, never averaged away.
If --dry-run: Print the mutation plan and STOP. Do not execute.
MUTATION PLAN (--dry-run)
Files: [N]
Mutations: [N] total
[list each mutation with ID, file, line, category, original, mutated, rationale]
To execute: zuvo:mutation-test [same args without --dry-run]
Phase 3: Mutation Execution
For each mutation in the plan, apply it, run tests, and record the result.
3.0 Checkpoint — write it after EVERY mutation, not at the end
A mutation run is a long loop of expensive, individually-meaningful results, and until now it kept all of them in context only. Anything that ended the run — an API error, a 137, a timeout, the user stopping it — threw away every mutation already executed and left the next attempt to redo the lot. Worse, Phase 3.1 restores the file from a temp copy after each mutation, so a run killed mid-apply can leave a MUTATED file on disk with nothing on record saying so.
State file: zuvo/context/mutation-<target-hash>.json (<target-hash> = first 8 of the SHA-1 of
the scope argument, so concurrent runs on different scopes do not collide).
{
"version": 1,
"scope": "src/services/",
"baseline": { "passed": 412, "failed": 0, "sha7": "a1b2c3d" },
"applied_to": null,
"mutations": [
{ "id": "MUT-001", "file": "src/x.ts", "symbol": "calcTax", "line": 88,
"original_norm": "if (n > 0) {", "occurrence": 1, "file_sha": "e4f5…",
"status": "killed|survived|fixed|not_run|lost", "anchor": "exact|relocated(88→91)|lost(<reason>)" }
]
}
BEFORE anything else — including on a FRESH run — load any existing state file for this scope and
honour its applied_to. Writing a new plan first would overwrite the only record that a previous
run died with a mutation on disk, and that mutated file then stays in the working tree, silently
poisoning every later baseline in a way that looks like a real regression. continue is not the
only path into a crashed run; the far more likely one is a user who re-runs the same command.
Write it at three moments, and the middle one is the one that matters:
- After the plan is generated — the full list at
status: not_run. Only after the recovery above has run andapplied_tois back tonull. applied_to: "<file>"BEFORE writing a mutation, back tonullAFTER restoring it. This is the crash-safety record: a non-nullapplied_toon startup means the previous run died with a mutation on disk. Restore that file from its temp copy (orgit checkoutit if the copy is gone and the file is tracked and otherwise clean) BEFORE doing anything else, and say so.- After each mutation resolves — its
status, immediately, not batched at the end.
continue mode: load the state file, restore any applied_to leftover, then execute only
mutations whose status is not_run, re-resolving each anchor per 2.3b (the tree has moved since
the plan — that is why you are resuming). Print what you skipped and why:
[MUTATION] resumed from checkpoint: 31/50 already resolved (24 killed, 5 survived, 2 fixed)
[MUTATION] restored a mutated file left by the interrupted run: src/x.ts
[MUTATION] 19 remaining
If no state file exists for the scope, say so and start fresh — never silently treat continue as
a new full run, because the score of a partial resume and the score of a fresh run are different
numbers and only one of them answers the question that was asked.
3.1 Safety Protocol
Before starting execution:
- Require cleanliness only where it actually matters: the files that will be MUTATED.
git status --porcelain -- <each production file in scope>must be empty. Those are the only files where the post-run hash check cannot tell a leftover mutant from an edit of yours, which is the entire hazard this step exists for.- Dirty scoped file →
BLOCKED_DIRTY_TREE, naming it. Do not mutate it. - Uncommitted changes ANYWHERE ELSE are not a blocker. Record their sha256 with the rest of
the pre-run snapshot and proceed. Demanding a globally clean tree blocked runs on the
pipeline's OWN output — the test file
zuvo:write-testshad just written andmemory/coverage.md— which is a gate firing on the work it was invoked to measure. - Never stop to ask permission to commit. If this run authored those files, the auto-commit policy already covers them; if they are the user's, they are none of this skill's business. A question here costs a whole turn and the answer is always the same.
- Dirty scoped file →
- Restoration strategy (temp copy — NOT stash):
- For each production file being mutated, copy the original to a temp location:
cp [file] /tmp/zuvo-mutation-[hash]-[filename] - After each mutation: restore from the temp copy:
cp /tmp/zuvo-mutation-[hash]-[filename] [file] - After ALL mutations complete (or on error): verify every original is restored, then delete temp copies
- Do NOT use
git stash(pop consumes the stash on first iteration) - Do NOT use
git checkout -- [file](destructive to local changes)
- For each production file being mutated, copy the original to a temp location:
- NEVER commit a mutated file. NEVER leave a mutation in place after execution.
3.2 Execution Loop
For each mutation MUT-NNN:
1. APPLY: Write the mutated code to the production file
2. RUN (two-tier strategy):
TIER 1 — Run mapped test files first (fast, targeted):
- Jest: npx jest [test_file_1] [test_file_2] --no-coverage
- Vitest: npx vitest run [test_file_1] [test_file_2]
- Pytest: pytest [test_file_1] [test_file_2] -x
- Go: go test [package] -run [test_pattern]
TIER 2 — If TIER 1 passes (mutation survived), run the FULL test suite:
- This catches integration tests, black-box tests, and indirect callers
- If full suite fails -> mutation KILLED (integration test caught it)
- If full suite passes -> mutation truly SURVIVED
--quick mode: skip TIER 2 (only mapped tests). Mark survivors as
"SURVIVED (mapped tests only)" with a warning that score may be optimistic.
3. RECORD result:
- Test FAILED (tier 1) -> mutation KILLED (good: direct test caught it)
- Test FAILED (tier 2) -> mutation KILLED-INDIRECT (good: integration test caught it)
- Test PASSED (both tiers) -> mutation SURVIVED (bad: no test caught it)
- Test TIMEOUT (>per-file timeout) -> mutation TIMEOUT (counts as killed)
- Test ERROR (crash/compile error) -> mutation KILLED (counts as killed)
4. RESTORE: Copy original from temp location back to production file
5. VERIFY: Diff check to confirm restoration is clean
Error recovery: If restoration fails for any reason:
- Copy from temp file:
cp /tmp/zuvo-mutation-[hash]-[filename] [file] - If temp file missing:
git checkout HEAD -- [file](safe: working dir was clean at start) - If both fail, STOP execution and alert the user
Progress tracking: After every 10 mutations, print a progress line:
PROGRESS: [N]/[total] mutations executed | [killed] killed | [survived] survived
3.2b Native runner execution (--runner native / hybrid / auto when one is available)
The native runner replaces the 3.2 loop for the mutants it generates; it does not replace any rule around that loop. Run it scoped to the SAME file set built in 0.2, and inherit every constraint below without exception:
ON THE FARM, in an isolated checkout — reversed 2026-08-29 by measurement. This rule used to say "local, always", and following it is what produced the incident below. A native run is the single heaviest thing this repo starts: one long-lived process re-running the suite once per mutant, 730 mutants and ~24 minutes in the case that broke.
What actually happened when it ran locally: the worktree's
node_moduleswas shared with another worktree, the other run moved underneath it, and Stryker died ten minutes in on vanished dependencies (mutation-testing-report-schema,balanced-match) — a crash that looks like a test failure and is not. At the same moment the laptop carried 109 local test processes at 421% CPU and load 34, macOS was putting it to sleep withDark Wake Thermal Emergency, and the farm sat idle with ~18 free slots.The old justification was 1.3's per-invocation wrapper overhead — real, and it does not apply here. 1.3 measures a wrapper charge paid PER SHORT CALL; a native run pays it ONCE across twenty-plus minutes, where it rounds to nothing. Send it to the farm:
rt npx stryker run <config> # or the project's own native runnerGenerate
<config>with the helper — do NOT hand-roll it and do NOT rely on--mutate.bash "$ZUVO_BASE/scripts/stryker-scoped-config.sh" \ --file <f1> --file <f2> ... # or --files-from <list>, from the 0.2 scope setIt prints
config_path,report_path,temp_dir,test_runner,coverage_analysis,mutate_countand a readyrun_command.mutate_countis a check, not decoration: Stryker reports an empty mutate set as a successful run with a 100% score, so a typo in the scope set is indistinguishable from a perfect suite unless the count is read."Scoped Stryker config" is the most re-invented artifact in this fleet's retro log (~30 names for one thing), because it is five decisions that each fail SILENTLY:
--mutatealone does not scope the run. Stryker still loads the project config, which routinely carries a repo-widemutatearray, its own reporters, its owntempDirName. The CLI flag merges over one key; the rest still applies..stryker-tmpis shared by every run in the repo. Two scoped runs on one box corrupt each other's sandbox, and it surfaces as dependencies vanishing mid-run (Cannot find module 'balanced-match') — which reads as a test failure and is not. This is the same incident rule 1 above was reversed for; the config is the other half of the fix.coverageAnalysis: perTestmismarks module-level ("static") mutants as SURVIVED, because per-test coverage cannot a
…(truncated)