Test Coverage Audit
Purpose
Judge whether the right things are tested — critical paths, error and authorization-denial branches, the required cases — rather than whether a coverage percentage is high. Coverage % is a hint, never the goal.
When to Use
- Before release, or when assessing an existing suite's adequacy (
../../existing-project-audit).
- When deciding where to invest test effort next.
- Not to enforce a blanket coverage threshold (that's the anti-pattern this skill guards against).
Inputs
- The application's critical paths + risk map (
testing-selection, ../../../knowledge/).
- The existing suites + optional coverage data (as a hint, not a verdict).
Discovery Questions
- What are the highest-risk paths (auth, payments, data integrity, permissions) — and are they tested, including their failure/denial branches?
- For key behaviors, are the required cases present: happy path, invalid input, error path, authorization denial, regression?
- Where does high line-coverage hide untested branches (error handling, edge cases)?
Responsibilities
- Map coverage to risk: identify critical paths and check each is tested at an appropriate level (
testing-selection pyramid) — a well-tested trivial util and an untested payment path can share the same coverage %.
- Check the required-case set on important behaviors: happy path, invalid input, error path, authorization denial, regression — flag missing ones (especially error/denial branches, which line-coverage tools count as covered when only the happy branch ran).
- Treat coverage numbers as a diagnostic: low coverage on a critical module is a flag; high coverage is not proof (assertion-free or happy-path-only tests inflate it).
- Recommend targeted tests for the gaps that matter, prioritized by risk — not a sweep to hit a number.
- Feed findings to
testing-selection (missing levels) and the per-type skills (write the gap tests); regression gaps to regression-testing.
Required Workflow
- Build/confirm the critical-path + risk map.
- Check each critical path is tested at the right level.
- Audit required-case completeness on key behaviors (esp. error/denial).
- Use coverage data as a hint to find blind spots.
- Recommend prioritized, risk-justified tests; hand to the per-type skills.
Decision Rules
- Never chase a coverage percentage without risk justification — 100% on plumbing while auth is untested is failure dressed as success.
- Error and authorization-denial branches are the usual blind spots — check them explicitly.
- High coverage with weak assertions is worse than honest gaps — inspect test quality, not just the number.
- Prioritize gap-filling by blast radius (money, auth, data loss) first.
Rules
- Recommendations are risk-justified, not threshold-driven.
- Coverage tools inform; they don't decide adequacy.
- Missing required cases on critical paths are flagged as defects.
Anti-Patterns
- Mandating "80% coverage" as the quality bar.
- Counting happy-path-only tests as covering a behavior.
- Trusting high line-coverage that skips error/denial branches.
- Writing low-value tests to raise a number.
- Auditing coverage without a risk map.
Validation Checklist
Definition of Done
A risk-based coverage assessment naming the untested critical paths and missing required cases, with prioritized, risk-justified test recommendations — and no coverage-percentage target treated as the goal.
Related Skills
testing-selection, unit-testing, integration-testing, api-integration-testing, regression-testing, ../../existing-project-audit, ../../final-quality-audit, ../../security/security-regression-testing.
Related Knowledge
../../../knowledge/ (risk map, critical paths).
Related References
../../../references/testing/ (coverage-audit notes, when populated).
Context Loading Guidance
- Requires: critical-path/risk map, existing suites, optional coverage data.
- Does not require: rewriting tests here (hand to per-type skills), full source.
- May load:
testing-selection, regression-testing.
- Stop when: risk-based gaps + prioritized recommendations are recorded.
Token Efficiency Guidance
The critical-path × required-case matrix (tested? gap?) is the artifact; audit the high-risk paths, not every file.
1---2name: test-coverage-audit3description: Use to assess test coverage by risk, not by percentage — find untested critical paths (auth, money, data integrity, error/denial paths) and missing required cases, and recommend targeted tests. Never chase a coverage number without risk justification.4---56# Test Coverage Audit78## Purpose910Judge whether the *right things* are tested — critical paths, error and authorization-denial branches, the required cases — rather than whether a coverage percentage is high. Coverage % is a hint, never the goal.1112## When to Use1314- Before release, or when assessing an existing suite's adequacy (`../../existing-project-audit`).15- When deciding where to invest test effort next.16- **Not** to enforce a blanket coverage threshold (that's the anti-pattern this skill guards against).1718## Inputs1920- The application's critical paths + risk map (`testing-selection`, `../../../knowledge/`).21- The existing suites + optional coverage data (as a hint, not a verdict).2223## Discovery Questions2425- What are the highest-risk paths (auth, payments, data integrity, permissions) — and are they tested, including their failure/denial branches?26- For key behaviors, are the **required cases** present: happy path, invalid input, error path, authorization denial, regression?27- Where does high line-coverage hide untested branches (error handling, edge cases)?2829## Responsibilities3031- Map **coverage to risk**: identify critical paths and check each is tested at an appropriate level (`testing-selection` pyramid) — a well-tested trivial util and an untested payment path can share the same coverage %.32- Check the **required-case set** on important behaviors: **happy path, invalid input, error path, authorization denial, regression** — flag missing ones (especially error/denial branches, which line-coverage tools count as covered when only the happy branch ran).33- Treat coverage numbers as a **diagnostic**: low coverage on a critical module is a flag; high coverage is not proof (assertion-free or happy-path-only tests inflate it).34- Recommend **targeted tests** for the gaps that matter, prioritized by risk — not a sweep to hit a number.35- Feed findings to `testing-selection` (missing levels) and the per-type skills (write the gap tests); regression gaps to `regression-testing`.3637## Required Workflow38391. Build/confirm the critical-path + risk map.402. Check each critical path is tested at the right level.413. Audit required-case completeness on key behaviors (esp. error/denial).424. Use coverage data as a hint to find blind spots.435. Recommend prioritized, risk-justified tests; hand to the per-type skills.4445## Decision Rules4647- **Never chase a coverage percentage without risk justification** — 100% on plumbing while auth is untested is failure dressed as success.48- Error and authorization-denial branches are the usual blind spots — check them explicitly.49- High coverage with weak assertions is worse than honest gaps — inspect test quality, not just the number.50- Prioritize gap-filling by blast radius (money, auth, data loss) first.5152## Rules5354- Recommendations are risk-justified, not threshold-driven.55- Coverage tools inform; they don't decide adequacy.56- Missing required cases on critical paths are flagged as defects.5758## Anti-Patterns5960- Mandating "80% coverage" as the quality bar.61- Counting happy-path-only tests as covering a behavior.62- Trusting high line-coverage that skips error/denial branches.63- Writing low-value tests to raise a number.64- Auditing coverage without a risk map.6566## Validation Checklist6768- [ ] Critical-path + risk map built.69- [ ] Each critical path tested at the right level.70- [ ] Required-case completeness checked (esp. error/denial).71- [ ] Coverage data used as a hint, not a verdict.72- [ ] Prioritized, risk-justified gap tests recommended.7374## Definition of Done7576A risk-based coverage assessment naming the untested critical paths and missing required cases, with prioritized, risk-justified test recommendations — and no coverage-percentage target treated as the goal.7778## Related Skills7980`testing-selection`, `unit-testing`, `integration-testing`, `api-integration-testing`, `regression-testing`, `../../existing-project-audit`, `../../final-quality-audit`, `../../security/security-regression-testing`.8182## Related Knowledge8384`../../../knowledge/` (risk map, critical paths).8586## Related References8788`../../../references/testing/` (coverage-audit notes, when populated).8990## Context Loading Guidance9192- **Requires:** critical-path/risk map, existing suites, optional coverage data.93- **Does not require:** rewriting tests here (hand to per-type skills), full source.94- **May load:** `testing-selection`, `regression-testing`.95- **Stop when:** risk-based gaps + prioritized recommendations are recorded.9697## Token Efficiency Guidance9899The critical-path × required-case matrix (tested? gap?) is the artifact; audit the high-risk paths, not every file.
ahtishamshahzad