# Validate Tests

> Detects test anti-patterns based on behavior testing principles

- Skill: `ww-w-ai/validate-tests` (Agent Skill, multi-file: 3 files)
- Install (CLI): `npx skillmds@latest add ww-w-ai/validate-tests`
- Raw SKILL.md: https://api.skillmd.com/api/skills/ww-w-ai/validate-tests/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: ww-w-ai (https://skillmd.com/u/ww-w-ai)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/ww-w-ai/validate-tests

---


# validate-tests -- Test Anti-Pattern Detection

Corresponding rule: `testing.md`

## Purpose

Statically detect anti-patterns in test code based on behavior-driven testing principles (testing.md). Identify patterns that hurt maintainability, such as implementation-detail testing, excessive mocking, and shared global state.

## Input

- Target: test files (`*.test.*`, `*.spec.*`, `test_*`, `*_test.*`)
- Default search path: entire project. If CLAUDE.md specifies a test directory, that path takes precedence.

## Validation Categories

### 1. Vague Test Names

Detect patterns where test names provide no clue to the cause of failure.

Search patterns:
- `test_\d+$` (names ending only in a number: `test_1`, `test_2`)
- `test_it_works`
- `test_function`
- `test_ok`
- `test_success`
- `it\(["']works["']` (JS/TS: `it('works')`)
- `it\(["']should work["']`
- `test\(["']test["']` (JS/TS: `test('test')`)

Severity: **WARN**

### 2. Implementation-Detail Tests

Detect patterns that verify call count/existence of internal methods.

Search patterns:
- `.call_count`
- `.called`
- `.assert_called_with`
- `.assert_called_once`
- `.assert_called_once_with`
- `.assert_any_call`
- `.toHaveBeenCalledTimes` (Jest)
- `.toHaveBeenCalledWith` (Jest)
- `verify(.*).times(` (Java Mockito)

Severity: **WARN**

### 3. Excessive Mocks

Detect patterns where a single test function uses 4 or more Mock objects.

Search patterns:
- `Mock()` or `mock.patch` appears 4+ times in a single test function
- `jest.fn()` or `jest.mock` appears 4+ times in a single test
- `@mock.patch` decorators: 4+ on a single test function

Severity: **WARN** (when 4 or more)

### 4. Shared Global State

Detect patterns where test files create instances at module level, sharing state across tests.

Search patterns:
- `= new ` (instance creation) -- at the top level of a test file (outside functions/classes)
- Variables with `shared_` prefix
- Module-level `let ` declarations that are mutated by tests (JS/TS)
- Use of the `global` keyword at the top level of a test file

Severity: **WARN**

### 5. DB Dependency in Unit Tests

Detect patterns where unit test files directly call DB connections/queries. Exclude integration test files whose names contain `integration`, `e2e`, or `integ`.

Search patterns:
- `db.query`
- `db.execute`
- `db.session`
- `db.connect`
- `prisma.` (query methods)
- `mongoose.connect`
- `connection.execute`
- `pool.query`

Exclusion condition: file name contains `integration`, `e2e`, or `integ`

Severity: **WARN**

## Execution Logic

1. `Glob` to collect test files (`*.test.*`, `*.spec.*`, `test_*`, `*_test.*`)
2. Categories 1, 2, 5: `Grep` each pattern across all test files
3. Category 3: `Read` test files and tally Mock count per function
4. Category 4: `Read` test files and detect instance creation at module level (outside functions/classes)
5. Group detected anti-patterns by category and output as a table

### Mandatory Output: Test Validation Matrix

Output the matrix below before generating the final report. Do not proceed until every category has been scanned.

| Category | Status | Items Checked | Findings | Severity | Evidence |
|----------|:------:|:-------------:|:--------:|:--------:|----------|
| Vague Test Names | ? | ? | ? | WARN | {Glob/Grep, files, pattern} |
| Implementation-Detail Tests | ? | ? | ? | WARN | {Glob/Grep, files, pattern} |
| Excessive Mocks | ? | ? | ? | WARN | {Read, scanned files} |
| Shared Global State | ? | ? | ? | WARN | {Read, scanned files} |
| DB Dependency | ? | ? | ? | WARN | {Grep, files, pattern} |

Status values: PASS (verification complete, no issues), NOT_APPLICABLE (no test files), SKIPPED (plugin issue), SHALLOW (partial scan)

### Pre-Output Checklist (Mandatory Before Final Output)

Check every item before writing the report. If any item is unchecked, go back and complete it.

- [ ] Every category has a Status value (no empty Status cells)
- [ ] Every category with Status != SKIPPED has an Evidence value
- [ ] All 5 validation categories scanned (or marked NOT_APPLICABLE)
- [ ] Every finding includes file path and line number
- [ ] Legacy-allowance annotations applied where applicable
- [ ] Report language matches the user's conversation language

### Schema Compliance Check (Mandatory Before Saving to .ww-w-ai/)

Before writing to .ww-w-ai/devtools/validate-tests/, verify the JSON output:
- [ ] Every "required" field in schema.json is present and non-empty
- [ ] The findings[] array contains every detected anti-pattern
- [ ] categorySummary counts match the actual finding counts per category

## Output

Generate the validation report in the user's conversation language.

Output detection results as a Markdown table in this format:

```
===== Test Validation Report =====

| File:Line | Type | Description |
|-----------|------|-------------|
| `src/services/auth.test.ts:15` | Vague Name | `test_1` -- cannot identify failure cause |
| `src/services/order.test.ts:42` | Implementation Detail | `.assert_called_once_with` -- internal-call verification |
| `src/services/payment.test.ts:20` | Excessive Mocks | 6 Mocks used -- design review needed |
| `test_user.py:5` | Global State | `shared_cart = Cart()` -- state shared across tests |
| `src/utils/calc.test.ts:30` | DB Dependency | Direct `db.query` call -- unit test depends on DB |

Total anti-patterns: {N}
  Vague Names: {A}
  Implementation Detail: {B}
  Excessive Mocks: {C}
  Global State: {D}
  DB Dependency: {E}

===============================
```

When no anti-patterns are found:

```
===== Test Validation Report =====

No anti-patterns.

===============================
```

## Output Persistence

After generating the test validation report, save results to `.ww-w-ai/devtools/validate-tests/`:

1. Create `.ww-w-ai/devtools/validate-tests/` if missing
2. Write `latest.json` -- structured result following `templates/schema.json`
3. Write `latest.md` -- human-readable report following `templates/report.template.md`
4. Archive to `history/` -- copy latest.json to `.ww-w-ai/devtools/validate-tests/history/{timestamp}.json`

`latest.md` is generated in the user's conversation language. JSON field names stay in English regardless of language.
The JSON output enables machine-parseable history tracking and cross-run comparison.
The history/ directory preserves previous executions for trend analysis.

## Permission Rationale

- **Write**: Exclusively for .ww-w-ai/ output persistence. No project source modification.
- **Bash**: Exclusively for read-only git/system queries. No file modifications.

## Notes

- Bash is permitted for directory creation (`mkdir -p .ww-w-ai/devtools/validate-tests/history`). The validation itself is read-only and does not modify project files.
- "Change-detection tests" in legacy code are temporarily allowed per testing.md exceptions. If the file or directory contains `legacy`, `migration`, or `compat`, annotate the result with "(legacy allowance)."
- Implementation-detail test detection (Category 2) may yield false positives. `.toHaveBeenCalledWith` can also be used for external-dependency verification (e.g., API calls), so results are presented as advisory.
- Excessive-Mocks judgment (Category 3) is performed per function. Mocks set up in `beforeEach`/`setUp` are distributed and tallied against each test.

## Spec Reference

For detailed validation criteria, evidence tables, and examples:
- Corresponding rule spec: `../../docs/specs/testing.md`