TensileLite Mutation Rerun
Use mutation testing only as an opt-in analysis technique for finding gaps in
TensileLite's characterization coverage. Treat surviving mutants as diagnostic
leads. When a survivor exposes an observable gap, make the durable result a
focused characterization test in the normal pytest suite.
Do not treat mutation scores or reports as CI or release gates. Do not change
production code merely to improve a mutation score. The scripts and Dockerfile
bundled here are executable documentation for maintainers and coding agents;
ordinary TensileLite builds and tests do not depend on them.
Prepare
- Read references/execution.md before running a
campaign. It contains the supported commands, manifest format, and recovery
rules.
- Use a Linux environment. On Windows, use WSL or a Linux container; native
Windows is unsupported. The documented workflow uses the optional image in
scripts/Dockerfile.
- Start from a clean tracked worktree. Keep one verifier process active at a
time because mutation is temporarily applied to that worktree.
- Do not push, edit pull requests, or update external trackers unless the user
explicitly authorizes that action.
Choose the next target
When the user has not selected a module, read
references/prioritization.md. First make a
fixed list of modules to compare. Use measured test coverage, mutation results,
and run time. Mark missing measurements instead of inventing values. Record
which module a person selected and why. Do not present a weighted score as
objective fact unless the team has reviewed its inputs and formula.
Run one investigation
- Define the slice, following
references/slice-planning.md: record the
source version, target module, selected tests, container, worker limit, and
output directory before editing any configuration. Assemble the test set
with references/covering-set.md and measure
its coverage of that exact file. Treat the threshold as a scheduling
heuristic, not as evidence that the tests detect each mutation. A run whose
selected tests do not meet the reviewed threshold is deferred, not started.
- Record the source and container state with
scripts/slice-preflight.sh.
- Use
scripts/pyproject-mutmut.sh backup and set to configure the bounded
campaign.
- Run mutmut with an explicit worker limit. Review every result that was not
killed, following references/survivor-triage.md.
Save the complete set of mutant IDs before grouping work, inspect each one
with
mutmut show <id>, and keep exactly one review row per ID.
- Restore
pyproject.toml and require assert-clean before survivor
verification.
- For a meaningful survivor, add a focused characterization test that passes
on unchanged source and fails on the changed behavior, following
references/test-authoring.md. A new untracked test
file can be verified directly; when an existing tracked test must change,
use a separate clean worktree or a deliberate local commit first.
- Verify that evidence with
scripts/mutmut-verify.sh. Collection, usage,
timeout, transport, interruption, and restoration failures are inconclusive;
they never prove a kill.
- Run
scripts/tests/run-selftests.sh after changing any bundled helper.
- Compare a report against an earlier one only when their inputs are
compatible, following
references/regression-comparison.md.
Require nonempty, unique mutant identities and an explicit decision for
every missing or new mutant. Comparison does not block any automated GitHub
check today; the repository has neither a standard report generator nor a
workflow that consumes one.
Report the outcome
Follow references/reporting.md. Build the report from
saved mutmut output, the complete review table, verifier output, and the
restoration result, keeping mutmut's original statuses separate from review
decisions. Record the source commit, container image, mutmut version, selected
source and tests, complete result counts, characterization tests added, and any
unresolved survivors or infrastructure failures. End with one explicit state:
- Unresolved survivor: the exact mutation has not yet been classified;
preserve its ID and evidence without calling it a coverage gap.
- Gap found: a survivor exposed missing behavioral coverage and a focused
characterization test was added.
- No gap demonstrated: the reviewed mutants were already detected or a
survivor was shown to be equivalent or otherwise unobservable; record that
reasoning without treating the score as a project quality gate.
- Inconclusive: an environment, collection, timeout, tool, or restoration
failure prevented trustworthy evidence. The verifier may label some of these
failures
BAD; neither label is valid kill evidence.
- Blocked: a named dependency or user decision is required.
Preserve unrelated work and never describe incomplete or partially counted
results as successful evidence.
1---2name: tensilelite-mutation-rerun3description: Guide optional, offline TensileLite mutation analysis used to find behavioral coverage and assertion gaps that should be closed with focused characterization tests. Use when planning, running, resuming, or auditing a TensileLite mutmut campaign; inspecting a surviving mutant; verifying that a characterization test detects it; or recording reproducible mutation evidence.4metadata:5 source: https://skillmd.com/skills/rocm/tensilelite-mutation-rerun6---78# TensileLite Mutation Rerun910Use mutation testing only as an opt-in analysis technique for finding gaps in11TensileLite's characterization coverage. Treat surviving mutants as diagnostic12leads. When a survivor exposes an observable gap, make the durable result a13focused characterization test in the normal pytest suite.1415Do not treat mutation scores or reports as CI or release gates. Do not change16production code merely to improve a mutation score. The scripts and Dockerfile17bundled here are executable documentation for maintainers and coding agents;18ordinary TensileLite builds and tests do not depend on them.1920## Prepare21221. Read [references/execution.md](references/execution.md) before running a23 campaign. It contains the supported commands, manifest format, and recovery24 rules.252. Use a Linux environment. On Windows, use WSL or a Linux container; native26 Windows is unsupported. The documented workflow uses the optional image in27 `scripts/Dockerfile`.283. Start from a clean tracked worktree. Keep one verifier process active at a29 time because mutation is temporarily applied to that worktree.304. Do not push, edit pull requests, or update external trackers unless the user31 explicitly authorizes that action.3233## Choose the next target3435When the user has not selected a module, read36[references/prioritization.md](references/prioritization.md). First make a37fixed list of modules to compare. Use measured test coverage, mutation results,38and run time. Mark missing measurements instead of inventing values. Record39which module a person selected and why. Do not present a weighted score as40objective fact unless the team has reviewed its inputs and formula.4142## Run one investigation43441. Define the slice, following45 [references/slice-planning.md](references/slice-planning.md): record the46 source version, target module, selected tests, container, worker limit, and47 output directory before editing any configuration. Assemble the test set48 with [references/covering-set.md](references/covering-set.md) and measure49 its coverage of that exact file. Treat the threshold as a scheduling50 heuristic, not as evidence that the tests detect each mutation. A run whose51 selected tests do not meet the reviewed threshold is deferred, not started.522. Record the source and container state with `scripts/slice-preflight.sh`.533. Use `scripts/pyproject-mutmut.sh backup` and `set` to configure the bounded54 campaign.554. Run mutmut with an explicit worker limit. Review every result that was not56 killed, following [references/survivor-triage.md](references/survivor-triage.md).57 Save the complete set of mutant IDs before grouping work, inspect each one58 with `mutmut show <id>`, and keep exactly one review row per ID.595. Restore `pyproject.toml` and require `assert-clean` before survivor60 verification.616. For a meaningful survivor, add a focused characterization test that passes62 on unchanged source and fails on the changed behavior, following63 [references/test-authoring.md](references/test-authoring.md). A new untracked test64 file can be verified directly; when an existing tracked test must change,65 use a separate clean worktree or a deliberate local commit first.667. Verify that evidence with `scripts/mutmut-verify.sh`. Collection, usage,67 timeout, transport, interruption, and restoration failures are inconclusive;68 they never prove a kill.698. Run `scripts/tests/run-selftests.sh` after changing any bundled helper.709. Compare a report against an earlier one only when their inputs are71 compatible, following72 [references/regression-comparison.md](references/regression-comparison.md).73 Require nonempty, unique mutant identities and an explicit decision for74 every missing or new mutant. Comparison does not block any automated GitHub75 check today; the repository has neither a standard report generator nor a76 workflow that consumes one.7778## Report the outcome7980Follow [references/reporting.md](references/reporting.md). Build the report from81saved mutmut output, the complete review table, verifier output, and the82restoration result, keeping mutmut's original statuses separate from review83decisions. Record the source commit, container image, mutmut version, selected84source and tests, complete result counts, characterization tests added, and any85unresolved survivors or infrastructure failures. End with one explicit state:8687- **Unresolved survivor:** the exact mutation has not yet been classified;88 preserve its ID and evidence without calling it a coverage gap.89- **Gap found:** a survivor exposed missing behavioral coverage and a focused90 characterization test was added.91- **No gap demonstrated:** the reviewed mutants were already detected or a92 survivor was shown to be equivalent or otherwise unobservable; record that93 reasoning without treating the score as a project quality gate.94- **Inconclusive:** an environment, collection, timeout, tool, or restoration95 failure prevented trustworthy evidence. The verifier may label some of these96 failures `BAD`; neither label is valid kill evidence.97- **Blocked:** a named dependency or user decision is required.9899Preserve unrelated work and never describe incomplete or partially counted100results as successful evidence.