Property-based testing
Property-based search challenges a quantified domain for counterexamples. A green run is evidence over the searched cases, not proof over every value.
Choose the branch
- Write or design properties — run the process below.
- Review existing property tests — load reviewing property tests and the matching framework adapter.
- Investigate a failure — load triaging counterexamples and the matching framework adapter.
Process
1. Commit or return
Inspect the production seam and existing test infrastructure, then write one outcome before mutation:
Commit: Use <generated property | repository law runner | exhaustive enumeration>
to search <domain> for <risk>; continue to the oracle and search contracts below.
or:
Return: Use Examples because <missing leverage>.
Commit when generated search covers a broad domain, sequence, or schedule more effectively than a small explicit table. Repository-native generators and law runners qualify. A tractable finite domain may commit to exhaustive enumeration. Return when the meaningful cases are a small table, the oracle would duplicate production logic, effects cannot be isolated within budget, or the required dependency is unavailable and not authorized.
Complete when: one concrete outcome is written. A specialist read without a Commit or Return outcome is incomplete.
2. State the contract
Write the property before its test code:
For every
xin domainDsatisfying preconditionP, observing the system produces relationRunder equivalence≈.
Ground D, P, R, and ≈ in specifications, public documentation, types, callers, and established tests. Treat names as search leads. Separate supported inputs from invalid inputs when their contracts differ, including the required error or rejection behavior.
Complete when: every term in the quantified statement has a source, and unresolved product decisions have been surfaced to the user.
3. Choose an oracle
Write the oracle contract before the generator:
Oracle source: <specification, reference, worked literal, invariant, law, or public postcondition>
Observation: <public result or relation>
Counterfeit: <plausible defect the oracle could reject>
Choose among a direct postcondition, independent differential model, round trip, metamorphic relation, preservation relation, algebraic law, or specified invalid-input behavior. A safety observation detects crashes and hangs; semantic correctness needs a semantic relation.
Keep the oracle structurally independent from production. Read designing properties when the compact oracle remains unclear or distinct risks require a portfolio.
Complete when: the oracle has a grounded public observation and could reject the named counterfeit.
4. Design the search
Write the search contract before framework code:
Generator: <supported and invalid partitions, including dependent inputs>
Distribution: <risk-bearing categories that must occur>
Failure handling: <shrinker, smallest failing case, or repository replay mechanism>
Replay: <seed, serialized case, explicit case, or framework replay artifact>
Budget: <case count or wall-time bound>
Generate dependent values jointly. Keep small and boundary values reachable. Use repository-native generators and the installed framework when available. Load only the matching framework adapter for syntax and shrink behavior.
When no property-testing dependency exists, follow repository dependency policy and obtain authorization before changing the manifest. A tractable finite domain may use exhaustive enumeration; label that evidence as enumeration rather than claiming generative shrinking.
Load generators and shrinking when the domain or shrink strategy remains unclear. Load stateful and concurrent testing for operation sequences or schedules, and advanced search only after ordinary generation cannot reach a named risk.
Use the adapter for the repository's installed framework:
- Python/Hypothesis — Hypothesis adapter
- JavaScript or TypeScript/fast-check — fast-check adapter
- Rust/proptest — proptest adapter
- Go/rapid — rapid adapter
- Java/jqwik — jqwik adapter
- Another ecosystem — other frameworks
Complete when: the search contract is concrete, the framework or repository mechanism is selected, and every intentional domain limit has a reason.
5. Prove discrimination
Run the property against the target defect, pre-fix behavior, or a safe temporary counterfeit. Keep temporary mutations out of the final change.
Complete when: the search goes red for the named relation on every safely exercisable counterfeit, or each unproven counterfeit and constraint is recorded.
6. Operate the test
Run the repository's normal test command. When the ordinary suite is unavailable or already red, use current base-revision or CI evidence when available and record the limitation. Confirm that risk-bearing categories occur, failures replay through the declared mechanism and reduce when that mechanism supports reduction, and the search fits its declared budget. Keep a durable explicit regression when a minimized case communicates a named boundary better than replay metadata alone.
Use triaging counterexamples before changing production code in response to a generated failure. When another skill or plan owns the parent evidence, return the result to its final audit.
Complete when: the corrected implementation is green; observed search reach matches the stated contract; failures replay and reduce as supported by the declared mechanism; ordinary example-test results are green or accounted for; and any parent evidence is closed.
Product seams
- Test through existing public observations whenever they express the contract.
- Change a production seam only when the change improves the product design independently of enabling a property.
- Preserve example tests that document named scenarios; properties add domain search rather than replacing useful examples.
- Honor an explicit request for example-only tests.