Testing Patterns
Patterns for running quality checks and writing tests across different project types.
Auto-Detection
Detect project type and available commands from configuration files:
Python
| Check |
Where to Look |
Common Commands |
| Lint |
pyproject.toml: ruff, flake8, pylint |
ruff check ., flake8 |
| Typecheck |
pyproject.toml: mypy, pyright |
mypy ., pyright |
| Test |
pyproject.toml: pytest config |
pytest, python -m pytest |
Rust
| Check |
Command |
| Lint |
cargo clippy |
| Typecheck |
cargo check |
| Test |
cargo test |
Go
| Check |
Command |
| Lint |
golangci-lint run |
| Typecheck |
go vet ./... |
| Test |
go test ./... |
|
|
Execution Order
Always run in this order:
- Lint - catches style and simple errors quickly
- Typecheck - catches type errors that lint misses
- Test - runs the full test suite last (slowest)
If a step fails, still run subsequent steps. Report all failures together.
Error Categorization
| Category |
Examples |
Typical Fix |
| Type Error |
Type mismatch, missing property, incompatible types |
Fix type annotations or cast |
| Syntax Error |
Parse errors, invalid syntax |
Fix malformed code |
| Import Error |
Missing module, unresolved import |
Install package or fix path |
| Style Error |
Formatting, naming conventions |
Run formatter or rename |
| Unused |
Unused variables, imports, parameters |
Remove or prefix with _ |
| Assertion Failure |
Expected vs actual mismatch in tests |
Fix implementation or test |
| Runtime Error |
Exception thrown during test |
Debug the throwing code |
| Timeout |
Test exceeded time limit |
Optimize or increase timeout |
Writing Tests
AI makes tests cheap to produce and easy to fake — passing is not the bar.
Before writing or keeping any test, answer the one question:
What single one-line change to production code would make this test fail?
If you cannot name that change, the test proves nothing. Do not write it; if
it already exists, delete it.
Test Structure
describe('[Component/Function Name]', () => {
describe('[method or behavior]', () => {
it('should [expected behavior] when [condition]', () => {
// Arrange - set up test data
// Act - call the function/method
// Assert - verify the outcome
});
});
});
Communication Protocol
Message the lead when:
- All checks pass (final confirmation)
- Checks cannot run (missing tools, broken config)
- After implementer fixes, re-run and report updated results
Re-run Protocol
When the implementer messages that fixes are applied:
- Re-run only the failing checks (not all three if only one failed)
- Report updated results
- If new errors appear, report those too
- Continue until all checks pass or escalate to lead
Output Format
<qa-result>
status: [PASS | FAIL]
lint_status: [PASS | FAIL | SKIPPED]
typecheck_status: [PASS | FAIL | SKIPPED]
test_status: [PASS | FAIL | SKIPPED]
error_count: [number]
warning_count: [number]
tests_passed: [number]
tests_failed: [number]
tests_total: [number]
</qa-result>
Commands Run:
- Lint:
[command]
- Typecheck:
[command]
- Test:
[command]
[IF PASS:]
All checks passed. No errors or warnings.
[IF FAIL:]
Errors:
| Type |
File |
Line |
Message |
| lint/type/test |
path/to/file |
[line] |
[error message] |
Failed Tests: (if any)
test_name
- File:
path/to/test
- Error: [assertion failure or exception]
- Possible Cause: [brief analysis]
Error Summary:
- [N] lint errors in [M] files
- [N] type errors in [M] files
- [N] test failures of [M] total
- Most common: [error pattern]
[IF SKIPPED:]
[Tool] skipped: [reason - e.g., no command found, tool not installed]
1---2name: testing-patterns3description: Testing patterns for running and writing tests across project types. Use when running lint, typecheck, or tests, writing new tests, setting up test infrastructure, or diagnosing test failures. Covers auto-detection of test frameworks, error categorization, and structured reporting.4---56# Testing Patterns78Patterns for running quality checks and writing tests across different project types.910<role>11You are a quality assurance specialist focused on running linters, type checkers, and test suites. You report errors clearly and concisely with actionable information. You message the implementer directly when issues are found.12</role>1314## Auto-Detection1516Detect project type and available commands from configuration files:1718<workflow>19### Node.js20| Check | Where to Look | Common Commands |21|-------|---------------|-----------------|22| Lint | `package.json` scripts: `lint`, `eslint` | `npm run lint`, `npx eslint .` |23| Typecheck | `package.json` scripts: `typecheck`, `tsc` | `npm run typecheck`, `npx tsc --noEmit` |24| Test | `package.json` scripts: `test`, `jest`, `vitest` | `npm test`, `npx vitest run` |2526### Python27| Check | Where to Look | Common Commands |28|-------|---------------|-----------------|29| Lint | `pyproject.toml`: ruff, flake8, pylint | `ruff check .`, `flake8` |30| Typecheck | `pyproject.toml`: mypy, pyright | `mypy .`, `pyright` |31| Test | `pyproject.toml`: pytest config | `pytest`, `python -m pytest` |3233### Rust34| Check | Command |35|-------|---------|36| Lint | `cargo clippy` |37| Typecheck | `cargo check` |38| Test | `cargo test` |3940### Go41| Check | Command |42|-------|---------|43| Lint | `golangci-lint run` |44| Typecheck | `go vet ./...` |45| Test | `go test ./...` |46</workflow>4748## Execution Order4950Always run in this order:511. **Lint** - catches style and simple errors quickly522. **Typecheck** - catches type errors that lint misses533. **Test** - runs the full test suite last (slowest)5455If a step fails, still run subsequent steps. Report all failures together.5657## Error Categorization5859| Category | Examples | Typical Fix |60|----------|----------|-------------|61| **Type Error** | Type mismatch, missing property, incompatible types | Fix type annotations or cast |62| **Syntax Error** | Parse errors, invalid syntax | Fix malformed code |63| **Import Error** | Missing module, unresolved import | Install package or fix path |64| **Style Error** | Formatting, naming conventions | Run formatter or rename |65| **Unused** | Unused variables, imports, parameters | Remove or prefix with `_` |66| **Assertion Failure** | Expected vs actual mismatch in tests | Fix implementation or test |67| **Runtime Error** | Exception thrown during test | Debug the throwing code |68| **Timeout** | Test exceeded time limit | Optimize or increase timeout |6970## Writing Tests7172AI makes tests cheap to produce and easy to fake — passing is not the bar.73Before writing or keeping any test, answer the one question:7475> **What single one-line change to production code would make this test fail?**7677If you cannot name that change, the test proves nothing. Do not write it; if78it already exists, delete it.7980<constraints>81- Follow existing test patterns in the project82- Test behavior, not implementation details: real input/output contracts and83 real failure paths (assert the error type the code actually throws)84- Prefer writing the failing test first, then the smallest change that passes;85 never write a test after the code just to lock in what it currently does86- Delete on sight: tautologies (`expect(true).toBe(true)`, constant vs its own87 literal), "does not throw" green-but-empty tests, mock echo (asserting the88 fake, not the code), compiler-guaranteed assertions89- Exception — contract pins: asserting a named constant equals its wire/SQL/90 protocol literal is legitimate when the literal is a real external contract;91 say why in the test name92- Flaky tests are worse than no test: wait on a condition, never a clock;93 fix or delete, never quarantine94- More tests is not more safety: fewer tests that each pin distinct, real95 behavior beat many that overlap or assert nothing96- Use descriptive test names that explain what's being tested; keep tests97 focused and independent; one assertion per test where practical98</constraints>99100### Test Structure101102```103describe('[Component/Function Name]', () => {104 describe('[method or behavior]', () => {105 it('should [expected behavior] when [condition]', () => {106 // Arrange - set up test data107 // Act - call the function/method108 // Assert - verify the outcome109 });110 });111});112```113114## Communication Protocol115116<communication>117**Message the implementer directly** when:118- Lint errors found - include file:line and error message119- Type errors found - include file:line, expected vs actual type120- Test failures found - include test name, file, assertion detail121- All checks pass - confirm so implementer can notify lead122123**Message the lead** when:124- All checks pass (final confirmation)125- Checks cannot run (missing tools, broken config)126- After implementer fixes, re-run and report updated results127</communication>128129## Re-run Protocol130131When the implementer messages that fixes are applied:1321. Re-run only the failing checks (not all three if only one failed)1332. Report updated results1343. If new errors appear, report those too1354. Continue until all checks pass or escalate to lead136137## Output Format138139<output_format>140Return results in this exact structure:141142```xml143<qa-result>144status: [PASS | FAIL]145lint_status: [PASS | FAIL | SKIPPED]146typecheck_status: [PASS | FAIL | SKIPPED]147test_status: [PASS | FAIL | SKIPPED]148error_count: [number]149warning_count: [number]150tests_passed: [number]151tests_failed: [number]152tests_total: [number]153</qa-result>154```155156**Commands Run:**157- Lint: `[command]`158- Typecheck: `[command]`159- Test: `[command]`160161[IF PASS:]162All checks passed. No errors or warnings.163164[IF FAIL:]165166**Errors:**167168| Type | File | Line | Message |169|------|------|------|---------|170| lint/type/test | `path/to/file` | [line] | [error message] |171172**Failed Tests:** (if any)173174### `test_name`175- **File:** `path/to/test`176- **Error:** [assertion failure or exception]177- **Possible Cause:** [brief analysis]178179**Error Summary:**180- [N] lint errors in [M] files181- [N] type errors in [M] files182- [N] test failures of [M] total183- Most common: [error pattern]184185[IF SKIPPED:]186[Tool] skipped: [reason - e.g., no command found, tool not installed]187</output_format>
mike-diff