Test layering policy
Unit tests
Purpose: verify individual functions and invariants in isolation.
- Data-driven: parameterized tables covering happy path, boundary, error, and edge cases.
- Property-based: fuzz invariants that must hold across all inputs (e.g., idempotency, sort stability, roundtrip serialization).
- Derive cases from the module's public API surface: input types/constraints, output shape, error modes, invariants.
Integration / contract tests
Purpose: verify interactions between components and external services.
- API envelope: request/response shape, status codes, content types, pagination.
- Error contract: error codes, error shapes, rate limiting, retries.
- Auth and scoping: token validation, role-based access, tenant isolation.
- Eventual consistency: verify convergence within bounded time; poll rather than sleep.
- Reuse auth state across tests where possible; avoid redundant login flows.
E2E tests
Purpose: verify real user workflows through the full stack.
- No mocks; exercise real services, databases, and APIs.
- Happy-path workflows only; save edge cases for lower layers.
- State-tolerant: never assume a clean slate; tolerate and work with prior state.
- Idempotent: safe to run repeatedly without cleanup between runs.
- Flow-oriented: validate real data paths end-to-end rather than isolated assertions.
Hard rules
- Never invent signatures, source locations, or line numbers. Only reference what you have read from the codebase.
- No fabricated fixtures. Derive test data from actual schemas, types, or seed data in the repo.
- No test-only hacks in product code. No
if (process.env.TEST) branches, no test-specific exports, no test backdoors.
- E2E must not rely on clean slate. Tests must tolerate pre-existing data, prior test runs, and shared environments.
Execution guidance
Preflight checks (before e2e)
- Verify the target environment is reachable (health endpoint, ping).
- Confirm required services are running (database, API, auth provider).
- Validate test user / credentials exist and are functional.
- Check for leftover state that could cause false failures; log it, do not fail on it.
Deterministic fixtures
- Use seeded randomness for generated data (seeded faker, deterministic UUIDs).
- Fixtures should be self-contained; avoid cross-test fixture dependencies.
- Prefer factory functions over shared mutable fixture objects.
Async handling
- Poll with bounded timeout and backoff; never use fixed
sleep/waitForTimeout.
- Set explicit timeout per operation; fail fast with a descriptive message on timeout.
- Bound retry attempts (e.g., max 3 retries with exponential backoff).
- Use framework-native waiting (Playwright
expect, async assertions) over manual loops.
Flake handling
- Single infrastructure retry per test run; if it fails twice, it is not flake.
- On retry failure, collect diagnostics: screenshots, network logs, service health, timestamps.
- Classify the failure (flaky / outdated / bug) before attempting a fix.
- Never add arbitrary delays or retry loops as a flake "fix."
API surface discovery
Before generating test cases:
- Read the module source to enumerate exports/public functions.
- Confirm scope from the user request and inspected code context; if ambiguous, state assumptions and proceed conservatively.
- For each function: input types/constraints, output shape, error modes, invariants.
- Probe for state dependencies and ordering constraints between functions.
Output format
Use markdown. Produce three sections:
Test Strategy -- one bullet per layer (unit/integration/e2e) naming the functions/flows and their coverage type.
Test Matrix -- table per function: columns ID | Category | Name | Input | Expected. Case ID scheme: {CATEGORY}-{NN} (HP, BV, ERR, EDGE). Append-only; never renumber.
Implementation Plan -- ordered steps: fixtures, unit tests, integration tests, e2e flows, run command.
CI guidance
Fast PR smoke lane
- Unit tests + linting + type-check on every PR.
- Subset of integration tests covering critical contracts.
- Target: under 5 minutes.
Nightly full lane
Full unit + integration + e2e suite with higher property-based iteration counts. Flag tests that pass on retry but failed initially.
Workflow
- Spec or code defines the module behavior (types, constraints, API surface).
- Agent (with this skill) produces test strategy, matrix, and implementation plan.
- test-writer agent translates the plan to runnable code in the target language's idiom.
- Developer implements to pass the tests.
- If implementation reveals missing cases, propose them first; append to spec only when explicitly requested.
1---
2name: testing-best-practices
3description: Use when designing tests, writing test cases, or planning test strategy for a module. Covers unit, integration, and e2e layering.
4metadata:
5 source: https://skillmd.com/skills/alleneubank/testing-best-practices
6---
7
8## Test layering policy
9
10### Unit tests
11
12Purpose: verify individual functions and invariants in isolation.
13
14- **Data-driven**: parameterized tables covering happy path, boundary, error, and edge cases.
15- **Property-based**: fuzz invariants that must hold across all inputs (e.g., idempotency, sort stability, roundtrip serialization).
16- Derive cases from the module's public API surface: input types/constraints, output shape, error modes, invariants.
17
18### Integration / contract tests
19
20Purpose: verify interactions between components and external services.
21
22- **API envelope**: request/response shape, status codes, content types, pagination.
23- **Error contract**: error codes, error shapes, rate limiting, retries.
24- **Auth and scoping**: token validation, role-based access, tenant isolation.
25- **Eventual consistency**: verify convergence within bounded time; poll rather than sleep.
26- Reuse auth state across tests where possible; avoid redundant login flows.
27
28### E2E tests
29
30Purpose: verify real user workflows through the full stack.
31
32- No mocks; exercise real services, databases, and APIs.
33- Happy-path workflows only; save edge cases for lower layers.
34- **State-tolerant**: never assume a clean slate; tolerate and work with prior state.
35- **Idempotent**: safe to run repeatedly without cleanup between runs.
36- **Flow-oriented**: validate real data paths end-to-end rather than isolated assertions.
37
38## Hard rules
39
40- **Never invent signatures, source locations, or line numbers.** Only reference what you have read from the codebase.
41- **No fabricated fixtures.** Derive test data from actual schemas, types, or seed data in the repo.
42- **No test-only hacks in product code.** No `if (process.env.TEST)` branches, no test-specific exports, no test backdoors.
43- **E2E must not rely on clean slate.** Tests must tolerate pre-existing data, prior test runs, and shared environments.
44
45## Execution guidance
46
47### Preflight checks (before e2e)
48
491. Verify the target environment is reachable (health endpoint, ping).
502. Confirm required services are running (database, API, auth provider).
513. Validate test user / credentials exist and are functional.
524. Check for leftover state that could cause false failures; log it, do not fail on it.
53
54### Deterministic fixtures
55
56- Use seeded randomness for generated data (seeded faker, deterministic UUIDs).
57- Fixtures should be self-contained; avoid cross-test fixture dependencies.
58- Prefer factory functions over shared mutable fixture objects.
59
60### Async handling
61
62- Poll with bounded timeout and backoff; never use fixed `sleep`/`waitForTimeout`.
63- Set explicit timeout per operation; fail fast with a descriptive message on timeout.
64- Bound retry attempts (e.g., max 3 retries with exponential backoff).
65- Use framework-native waiting (Playwright `expect`, async assertions) over manual loops.
66
67### Flake handling
68
69- **Single infrastructure retry** per test run; if it fails twice, it is not flake.
70- On retry failure, collect diagnostics: screenshots, network logs, service health, timestamps.
71- Classify the failure (flaky / outdated / bug) before attempting a fix.
72- Never add arbitrary delays or retry loops as a flake "fix."
73
74## API surface discovery
75
76Before generating test cases:
77- Read the module source to enumerate exports/public functions.
78- Confirm scope from the user request and inspected code context; if ambiguous, state assumptions and proceed conservatively.
79- For each function: input types/constraints, output shape, error modes, invariants.
80- Probe for state dependencies and ordering constraints between functions.
81
82## Output format
83
84Use markdown. Produce three sections:
85
86**Test Strategy** -- one bullet per layer (unit/integration/e2e) naming the functions/flows and their coverage type.
87
88**Test Matrix** -- table per function: columns `ID | Category | Name | Input | Expected`. Case ID scheme: `{CATEGORY}-{NN}` (HP, BV, ERR, EDGE). Append-only; never renumber.
89
90**Implementation Plan** -- ordered steps: fixtures, unit tests, integration tests, e2e flows, run command.
91
92## CI guidance
93
94### Fast PR smoke lane
95
96- Unit tests + linting + type-check on every PR.
97- Subset of integration tests covering critical contracts.
98- Target: under 5 minutes.
99
100### Nightly full lane
101
102Full unit + integration + e2e suite with higher property-based iteration counts. Flag tests that pass on retry but failed initially.
103
104## Workflow
105
1061. Spec or code defines the module behavior (types, constraints, API surface).
1072. Agent (with this skill) produces test strategy, matrix, and implementation plan.
1083. test-writer agent translates the plan to runnable code in the target language's idiom.
1094. Developer implements to pass the tests.
1105. If implementation reveals missing cases, propose them first; append to spec only when explicitly requested.