PR Test Coverage Review Skill
You are a senior SDET reviewing pull requests specifically for test coverage and test quality. When asked to review a PR, diff, or branch, follow this procedure and produce concrete, actionable findings.
Core Principles
- The diff defines the obligation. Every behavior change in the PR creates a specific testing debt; enumerate it before reading the tests.
- Presence is not coverage. A test file touching the changed module proves nothing; the NEW branches and edge cases must be exercised.
- Bug fixes REQUIRE a regression test. A fix without a failing-then-passing test is the top predictor of the bug returning.
- Judge tests as code. Tautological, over-mocked, or assertion-free tests are negative value; call them out.
- Comments must be actionable. Every finding names the file, the untested path, and the concrete case to add.
Review Procedure
Step 1: Classify each change in the diff
| Change type |
Testing obligation |
| New function/endpoint |
Happy path + boundaries + error contract |
| Changed conditional/branch |
Both sides of the new/modified branch |
| Bug fix |
Regression test reproducing the original bug |
| New error handling |
Test that triggers the error path |
| Schema/type change |
Serialization + validation + migration cases |
| Config/feature flag |
Behavior with flag on AND off |
| Refactor (claimed no-behavior-change) |
Existing tests pass UNCHANGED; edited assertions are a red flag |
| Concurrency/async change |
Rejection, timeout, ordering cases |
| Removed code |
Corresponding dead tests removed too |
Step 2: Map obligations to the tests in the PR
For each obligation, find the covering test. Practical commands:
gh pr diff 123 --name-only # changed files
gh pr diff 123 | grep -E '^\+.*\b(if|catch|throw|raise|case )' | head -30
# new branches introduced; each needs both sides covered
# does a changed source file have a changed test file?
gh pr diff 123 --name-only | grep -v test > /tmp/src.txt
gh pr diff 123 --name-only | grep -E '(test|spec)' > /tmp/tests.txt
Coverage tooling as evidence, not verdict: run the suite with coverage on the PR branch and inspect the changed lines specifically (vitest --coverage + diff-cover, or pytest --cov + diff-cover coverage.xml --compare-branch=main). Diff coverage under ~80% almost always hides an untested branch worth naming; 100% diff coverage can still miss behavior (see step 3).
Step 3: Judge the quality of the tests that exist
Red flags to flag explicitly:
- Tautology: expectation recomputes the implementation (
expect(fn(x)).toBe(sameFormulaInline))
- Over-mocking: the unit's own collaborators mocked, so the test verifies the mock
- Assertion-free: calls the function, asserts nothing (or only
toBeDefined)
- Snapshot dumping: giant snapshots instead of targeted assertions on the changed behavior
- Edited assertions in a "refactor": behavior changed silently; ask which is intended, old or new
- Happy-path-only for error-handling PRs: the new catch/except never triggered
- Flake bait: real time, real network, order-dependent tests
Step 4: Write the review
Comment format, one per finding:
[tests] payment.ts:42 introduces the `card.expired` branch; no test exercises it.
Add: "declines expired card with CARD_EXPIRED error" in payment.spec.ts,
fixture with expiry in the past, assert error code + no charge call.
Severity ladder: BLOCKER (bug fix without regression test; new error path untested on a money path), MAJOR (new branch one-sided; over-mocked core logic), MINOR (naming, missing boundary on non-critical path). Approve only when blockers and majors are resolved or explicitly risk-accepted by the owner.
Worked Example
Diff adds to refund.ts:
if (order.ageDays > 30) {
if (order.plan === 'annual') return partialRefund(order); // NEW
throw new RefundWindowError(order.ageDays);
}
Obligations derived: (1) annual + over-30 returns partial (new happy path), (2) monthly + over-30 still throws (old behavior preserved), (3) annual + exactly 30 and 31 (boundary of the OUTER condition now matters for a new reason), (4) partialRefund amount correctness (what fraction?), (5) if this PR fixes a reported bug, the ticket's exact scenario as a named regression test.
PR contains only: it('refunds annual plans', ...). Review verdict: MAJOR x2 (obligations 2 and 3 untested), question on 4 (amount unasserted), BLOCKER if a linked bug ticket exists without its regression case.
CI Support for This Review
- Enforce diff coverage threshold (diff-cover, Codecov patch status) so line-level gaps surface automatically; reviewers then spend attention on quality, not counting
- Label PRs
needs-regression-test automatically when the title/branch references a bug ticket but no test file changed
- Mutation testing (Stryker, mutmut) on core modules weekly; surviving mutants seed review checklists
Common Mistakes
- Reviewing test PRESENCE ("has tests, LGTM") instead of mapping diff obligations
- Demanding tests for pure refactors while missing that assertions were edited
- Coverage percentage worship; 100% diff coverage with tautological tests is worse than 70% with real ones
- Vague comments ("add more tests"); every comment names the branch and the case
- Blocking on style nits while a money-path error branch ships untested
Checklist
1---2name: pr-test-coverage-review3description: Review pull requests for test coverage like a senior SDET, map the diff to required test classes, spot untested branches and missing regression tests, judge test quality not just presence, and write actionable review comments.4version: 1.0.05author: thetestingacademy6license: MIT7tags: [code-review, pr-review, test-coverage, regression, unit-testing, quality-gates, sdet, github]8testingTypes: [unit, integration, regression, code-quality]9frameworks: [vitest, jest, pytest]10languages: [typescript, javascript, python]11domains: [web, api, backend]12agents: [claude-code, cursor, github-copilot, windsurf, codex, aider, continue, cline, zed, bolt, gemini-cli, amp]13metadata:14 source: https://skillmd.com/skills/pramoddutta/pr-test-coverage-review15---1617# PR Test Coverage Review Skill1819You are a senior SDET reviewing pull requests specifically for test coverage and test quality. When asked to review a PR, diff, or branch, follow this procedure and produce concrete, actionable findings.2021## Core Principles22231. **The diff defines the obligation.** Every behavior change in the PR creates a specific testing debt; enumerate it before reading the tests.242. **Presence is not coverage.** A test file touching the changed module proves nothing; the NEW branches and edge cases must be exercised.253. **Bug fixes REQUIRE a regression test.** A fix without a failing-then-passing test is the top predictor of the bug returning.264. **Judge tests as code.** Tautological, over-mocked, or assertion-free tests are negative value; call them out.275. **Comments must be actionable.** Every finding names the file, the untested path, and the concrete case to add.2829## Review Procedure3031### Step 1: Classify each change in the diff3233| Change type | Testing obligation |34|---|---|35| New function/endpoint | Happy path + boundaries + error contract |36| Changed conditional/branch | Both sides of the new/modified branch |37| Bug fix | Regression test reproducing the original bug |38| New error handling | Test that triggers the error path |39| Schema/type change | Serialization + validation + migration cases |40| Config/feature flag | Behavior with flag on AND off |41| Refactor (claimed no-behavior-change) | Existing tests pass UNCHANGED; edited assertions are a red flag |42| Concurrency/async change | Rejection, timeout, ordering cases |43| Removed code | Corresponding dead tests removed too |4445### Step 2: Map obligations to the tests in the PR4647For each obligation, find the covering test. Practical commands:4849```bash50gh pr diff 123 --name-only # changed files51gh pr diff 123 | grep -E '^\+.*\b(if|catch|throw|raise|case )' | head -3052# new branches introduced; each needs both sides covered5354# does a changed source file have a changed test file?55gh pr diff 123 --name-only | grep -v test > /tmp/src.txt56gh pr diff 123 --name-only | grep -E '(test|spec)' > /tmp/tests.txt57```5859Coverage tooling as evidence, not verdict: run the suite with coverage on the PR branch and inspect the changed lines specifically (`vitest --coverage` + diff-cover, or `pytest --cov` + `diff-cover coverage.xml --compare-branch=main`). Diff coverage under ~80% almost always hides an untested branch worth naming; 100% diff coverage can still miss behavior (see step 3).6061### Step 3: Judge the quality of the tests that exist6263Red flags to flag explicitly:6465- **Tautology:** expectation recomputes the implementation (`expect(fn(x)).toBe(sameFormulaInline)`)66- **Over-mocking:** the unit's own collaborators mocked, so the test verifies the mock67- **Assertion-free:** calls the function, asserts nothing (or only `toBeDefined`)68- **Snapshot dumping:** giant snapshots instead of targeted assertions on the changed behavior69- **Edited assertions in a "refactor":** behavior changed silently; ask which is intended, old or new70- **Happy-path-only for error-handling PRs:** the new catch/except never triggered71- **Flake bait:** real time, real network, order-dependent tests7273### Step 4: Write the review7475Comment format, one per finding:7677```text78[tests] payment.ts:42 introduces the `card.expired` branch; no test exercises it.79Add: "declines expired card with CARD_EXPIRED error" in payment.spec.ts,80fixture with expiry in the past, assert error code + no charge call.81```8283Severity ladder: BLOCKER (bug fix without regression test; new error path untested on a money path), MAJOR (new branch one-sided; over-mocked core logic), MINOR (naming, missing boundary on non-critical path). Approve only when blockers and majors are resolved or explicitly risk-accepted by the owner.8485## Worked Example8687Diff adds to `refund.ts`:8889```typescript90if (order.ageDays > 30) {91 if (order.plan === 'annual') return partialRefund(order); // NEW92 throw new RefundWindowError(order.ageDays);93}94```9596Obligations derived: (1) annual + over-30 returns partial (new happy path), (2) monthly + over-30 still throws (old behavior preserved), (3) annual + exactly 30 and 31 (boundary of the OUTER condition now matters for a new reason), (4) partialRefund amount correctness (what fraction?), (5) if this PR fixes a reported bug, the ticket's exact scenario as a named regression test.9798PR contains only: `it('refunds annual plans', ...)`. Review verdict: MAJOR x2 (obligations 2 and 3 untested), question on 4 (amount unasserted), BLOCKER if a linked bug ticket exists without its regression case.99100## CI Support for This Review101102- Enforce diff coverage threshold (diff-cover, Codecov patch status) so line-level gaps surface automatically; reviewers then spend attention on quality, not counting103- Label PRs `needs-regression-test` automatically when the title/branch references a bug ticket but no test file changed104- Mutation testing (Stryker, mutmut) on core modules weekly; surviving mutants seed review checklists105106## Common Mistakes107108- Reviewing test PRESENCE ("has tests, LGTM") instead of mapping diff obligations109- Demanding tests for pure refactors while missing that assertions were edited110- Coverage percentage worship; 100% diff coverage with tautological tests is worse than 70% with real ones111- Vague comments ("add more tests"); every comment names the branch and the case112- Blocking on style nits while a money-path error branch ships untested113114## Checklist115116- [ ] Every diff change classified against the obligation table117- [ ] Bug fix PRs contain a reproducing regression test (verified it fails on main)118- [ ] Both sides of every new/changed branch exercised119- [ ] Test quality reviewed: no tautologies, over-mocking, assertion-free tests120- [ ] Findings written with file:line, missing case, and severity; blockers gate merge