Test Gap Review
Audit verification sufficiency without silently becoming a test-writing or production implementation workflow.
Workflow
- Establish the authoritative behavioral contract, comparison scope, and claimed verification.
- Map invariants, assumptions, normal behavior, important boundaries, meaningful failures, preserved state, cleanup, and external effects to existing evidence.
- Identify weak assertions, fixtures that cannot fail meaningfully, implementation-detail tests, unsupported manual claims, and skipped required commands. Flag example-only evidence when the claim spans a broad input space, operation sequence, concurrent ordering, restart, or recovery, and check that generated failures are reproducible.
- Review changed test code as maintained product code. Flag unclear naming or structure, repeated setup that should use a focused helper, helpers that hide scenario data or mix responsibilities, and table-driven cases that do not share one execution path.
- Report each material gap with the affected claim, missing evidence, consequence, smallest proving check, and residual risk.
- If remediation is authorized, use
test-design for test changes and behavior-implement for production behavior; otherwise remain read-only.
For each material claim report:
VERIFIED | NOT VERIFIED | INCONCLUSIVE
Claim: ...
Evidence: ...
Reasoning: ...
Do not chase coverage percentages, demand a stronger method without a concrete risk, add fake-confidence tests, broaden infrastructure, or change production behavior directly.
Finish when material claims have supported statuses and remaining verification risk is explicit.
1---2name: test-gap-review3description: Review whether existing tests and verification credibly prove a scoped behavior contract. Use during validation when evidence may omit important behavior or provide false confidence.4---56# Test Gap Review78Audit verification sufficiency without silently becoming a test-writing or production implementation workflow.910## Workflow11121. Establish the authoritative behavioral contract, comparison scope, and claimed verification.132. Map invariants, assumptions, normal behavior, important boundaries, meaningful failures, preserved state, cleanup, and external effects to existing evidence.143. Identify weak assertions, fixtures that cannot fail meaningfully, implementation-detail tests, unsupported manual claims, and skipped required commands. Flag example-only evidence when the claim spans a broad input space, operation sequence, concurrent ordering, restart, or recovery, and check that generated failures are reproducible.154. Review changed test code as maintained product code. Flag unclear naming or structure, repeated setup that should use a focused helper, helpers that hide scenario data or mix responsibilities, and table-driven cases that do not share one execution path.165. Report each material gap with the affected claim, missing evidence, consequence, smallest proving check, and residual risk.176. If remediation is authorized, use `test-design` for test changes and `behavior-implement` for production behavior; otherwise remain read-only.1819For each material claim report:2021```text22VERIFIED | NOT VERIFIED | INCONCLUSIVE23Claim: ...24Evidence: ...25Reasoning: ...26```2728Do not chase coverage percentages, demand a stronger method without a concrete risk, add fake-confidence tests, broaden infrastructure, or change production behavior directly.2930Finish when material claims have supported statuses and remaining verification risk is explicit.
yarlson