Test Quality Review
Use Python unittest documentation for that framework's behavior when applicable. The defect-sensitivity and quality checks below are this catalog's review contract; discover the actual framework and its version before applying framework-specific guidance.
Workflow
- Identify the behaviors and risks the tests are expected to protect.
- Trace representative tests from setup through action, oracle, cleanup, and failure output.
- Check that assertions can fail for the target defect and are not coupled only to implementation details.
- Review boundary, negative, concurrency, retry, migration, permission, and recovery coverage where relevant.
- Inspect isolation, determinism, clocks, randomness, network, shared state, fixtures, and parallel safety.
- Find skipped, quarantined, flaky, snapshot-heavy, redundant, slow, or low-signal coverage.
- Recommend focused mutations or controlled defect injection to test sensitivity when authorized.
Boundaries
- Do not equate test count, line coverage, or a green suite with meaningful behavioral protection.
- Do not weaken assertions or accept broad snapshots merely to reduce failures.
- Do not execute destructive fixtures or shared-environment tests without explicit authorization.
Output
Lead with the answer. Use the shortest format that retains the required evidence and limits; expand when requested or needed. The items below describe required content, not mandatory headings.
Return Protected behavior, Oracle quality, Missing risks, Flakiness and isolation, Maintenance cost, and Recommended evidence. Grade each finding none, note, warning, or error, and order the list with the most severe first. When a pinned change is under review, mark each finding new, unchanged, updated, or absent against the state before the change; otherwise report severity alone.
Example
Synthetic example; not an observed client result.
Input: A test asserts only that a mock method was called, not what the user receives.
Output: The test may pass with incorrect output. Add an observable result or error assertion and demonstrate that a real regression fails it.
1---2name: review-tests3description: "Reviews test quality when asked whether tests catch real regressions or merely mirror implementation; checks assertions, isolation, failure behavior, and coverage without equating count with confidence."4license: MIT5metadata:6 source: https://skillmd.com/skills/jovanipink/review-tests7 author: "Measured Studios"8 version: "0.19.0"9 plugin: "measured-quality"10 invocation: "implicit"11 provenance: "original"12 risk_class: "read-only"13---1415# Test Quality Review1617Use [Python unittest documentation](https://docs.python.org/3/library/unittest.html) for that framework's behavior when applicable. The defect-sensitivity and quality checks below are this catalog's review contract; discover the actual framework and its version before applying framework-specific guidance.1819## Workflow20211. Identify the behaviors and risks the tests are expected to protect.222. Trace representative tests from setup through action, oracle, cleanup, and failure output.233. Check that assertions can fail for the target defect and are not coupled only to implementation details.244. Review boundary, negative, concurrency, retry, migration, permission, and recovery coverage where relevant.255. Inspect isolation, determinism, clocks, randomness, network, shared state, fixtures, and parallel safety.266. Find skipped, quarantined, flaky, snapshot-heavy, redundant, slow, or low-signal coverage.277. Recommend focused mutations or controlled defect injection to test sensitivity when authorized.2829## Boundaries3031- Do not equate test count, line coverage, or a green suite with meaningful behavioral protection.32- Do not weaken assertions or accept broad snapshots merely to reduce failures.33- Do not execute destructive fixtures or shared-environment tests without explicit authorization.3435## Output3637Lead with the answer. Use the shortest format that retains the required evidence and limits; expand when requested or needed. The items below describe required content, not mandatory headings.3839Return Protected behavior, Oracle quality, Missing risks, Flakiness and isolation, Maintenance cost, and Recommended evidence. Grade each finding `none`, `note`, `warning`, or `error`, and order the list with the most severe first. When a pinned change is under review, mark each finding `new`, `unchanged`, `updated`, or `absent` against the state before the change; otherwise report severity alone.4041## Example4243Synthetic example; not an observed client result.4445Input: A test asserts only that a mock method was called, not what the user receives.4647Output: The test may pass with incorrect output. Add an observable result or error assertion and demonstrate that a real regression fails it.