Testing

Testing

Build the smallest executable argument that can disagree with a wrong implementation. A passing test matters only when its setup, observation, and oracle can expose the named risk.

Route before choosing examples

Inspect the requested contract, relevant production code, types, callers, and existing tests before choosing evidence.

Choose one primary search mechanism for each independently falsifiable risk:

A primary search mechanism is not exclusive evidence ownership. Explicit examples may still document named boundaries, minimized regressions, or separate risks. Every branch returns to this skill's final evidence audit.

Load branch-specific guidance only when its matching dimension applies:

Load more than one branch when dimensions combine, such as Rust code crossing a database boundary.

Complete when: each independently falsifiable risk has a primary search mechanism, or the missing information needed to choose one is named.

1. Inventory and frame the contract

Write one contract card for each independently observable requested claim and each source-grounded preservation risk:

Contract: <requested behavior>
Preservation: <adjacent existing behavior, or none>
Observation: <public surface where the contract is visible>
Risk: <plausible failure>
Oracle: <independent source of the expected result>
Counterfeit: <wrong behavior the evidence can reject>
Primary search: <Examples | Property-based testing | Fuzzing>
Evidence: <existing test, transient probe, durable test, or command>

Use one card for a single claim. Several cards may share one invariant or evidence route when each card states why it applies. A contract card does not require a new test file.

Ground cards in specifications, public documentation, types, callers, accepted behavior, and existing tests. Treat names and comments as search leads, not proof. Confirm any production-interface or scope-expanding seam before adding it.

Complete when: every requested claim and grounded preservation risk has an evidence route, or is marked unverified with the missing evidence named.

2. Choose the smallest discriminating surface

Place the test at the narrowest surface that still contains the risk:

• use an example test for a small set of named cases
• cross a real boundary when serialization, schema, configuration, lifecycle, or protocol behavior is the risk
• use a browser only when browser behavior or cross-system wiring is part of the claim
• use generated search only when a broad domain and compact oracle create leverage

Choose real dependencies and test doubles by what the test must detect. A fake is appropriate when it preserves the contract under test and removes an unrelated, destructive, unavailable, or prohibitively expensive dependency. A real dependency is required when its actual behavior is the risk.

Narrow evidence may localize a fault, but it does not discharge a contract whose observation is a runnable artifact, serialized boundary, user-facing entry point, or downstream consumer. Exercise an exact journey through that surface when the surface itself is part of the contract.

Complete when: a narrower surface would miss the named risk, every replaced dependency has an explicit realism tradeoff, and every contract observed through an outer surface has an evidence route through that surface.

3. Build an independent test

Follow the repository's existing framework, naming, fixture, and file-layout conventions. Test through the selected seam and observe public behavior rather than private calls or intermediate state.

Keep the oracle independent:

• derive the expected result from a specification example, worked literal, reference implementation, invariant, or public postcondition before consulting production output
• keep expected-value calculation structurally independent from production; avoid repeating the same algorithm
• assert the meaningful result, not merely that execution completed or returned a value
• control time, randomness, locale, ordering, scheduling, and external responses when they affect the result
• make test data reveal the contract; omit irrelevant fields and accidental noise

Extract setup behind helpers only when the helper names a domain operation or hides substantial mechanics. Keep the behavior and decisive assertion readable from the test.

Complete when: the test can fail while the code still compiles, and a reader can explain the expected result without opening production internals.

4. Prove discrimination

Use the strongest safe discrimination proof available:

1. For a bug, run the test against the buggy or pre-fix revision and then against the fix.
2. For new behavior, run the test before implementing the behavior and again after implementation.
3. For behavior that already exists, use an available mutation tool or safely introduce a local temporary counterfeit—for example, suppress the state transition, return the wrong boundary value, or invert the condition—then restore the implementation.
4. When none of those proofs is safe or proportionate, record the unproven counterfeit and the constraint that prevented exercising it. Prefer an honest limitation over changing production code merely to manufacture a red result.

Inspect every red result. A setup error, timeout, unrelated exception, or failure in a different assertion does not prove the intended behavior. Leave no deliberate defect in the final change.

Complete when: the strongest safe proof goes red for the named reason and green after the correct behavior is restored, or the unproven counterfeit and blocking constraint are explicitly recorded.

5. Validate and operate

Run the narrow test while iterating. For every new or renamed test, use runner collection, listing, or focused execution output to confirm that the intended command selects it. Account for focus, skip, todo, tag, feature, and exclusion markers; a green suite does not prove an unseen test ran.

Then run the repository's required type checks, linters, and full test suite once the change is complete. When order, concurrency, retry, or shared state matters, also repeat the test under the relevant seed, order, worker count, or scheduler.

Use coverage to locate unexercised risk, not as a target by itself. Use mutation testing on changed or risk-bearing code when the tool exists and survivors can become concrete test goals. Investigate small score changes and flaky results before drawing conclusions.

Complete when: new and renamed tests are collected, the intended test command and project validation are green, relevant nondeterminism has been stressed, and failures leave enough context to reproduce.

Review checklist

• The test names an observable contract and plausible failure.
• The chosen surface is the narrowest one that contains the risk.
• The oracle is independent of the implementation.
• New and renamed tests are collected by the intended command.
• The test avoids private state and incidental call choreography.
• The strongest safe discrimination proof was exercised, or its limitation was recorded.
• Time, randomness, concurrency, external responses, and durable state are controlled where relevant.
• Coverage, mutation, retries, and snapshots support a decision rather than replace one.
• The test remains useful after an internal refactor that preserves behavior.