Testing Strategy

Testing Strategy

Overview

Define what gets tested, at what level, with what mocking — before tests proliferate. Output is .forge/testing-strategy.md — the test pyramid for this project (unit/integration/e2e ratio), what each level is responsible for, mocking philosophy, per-component coverage targets, the critical-path inventory, the flake quarantine policy, and CI gate definitions. Consumed by incremental-implementation, tdd, and code-review-and-quality.

When to Use

• A new project is starting and needs test conventions before code lands
• The test suite has grown organically and is slow, flaky, or inconsistent
• Coverage % is being treated as a goal and producing meaningless tests
• A team is mocking everything internally and tests pass while features break
• CI takes >15 minutes and the team is starting to skip tests

When NOT to Use

• Single-task work where tdd is the right skill (write the test, write the code)
• Reference implementations or learning projects where tests aren't shipped
• The strategy exists, is followed, and only one component needs adjustment

Common Rationalizations

Red Flags

• "100% coverage" as the explicit target
• A test file with expect(spy).toHaveBeenCalledWith(...) and no other assertions
• Every external service mocked at function level (mock the boundary instead)
• Zero end-to-end tests for the primary user journey
• A test that takes >30 seconds and runs on every commit
• A test that passes when the underlying feature is manually broken (snapshot rot, weak assertions)
• A retry config on pytest/vitest/jest to "fix" flakes

Core Process

Step 1: Identify critical user paths

Read .forge/prd.md if present. List the 3-7 user journeys that, if broken, ship a broken product. Each is an e2e candidate. Everything else is unit/integration.

Step 2: Decide test level per path

Default ratio: 70% unit / 20% integration / 10% e2e. Adjust based on what's risky.

Step 3: Define mocking boundaries

The rule: mock at the seam, not inside the module.

• Mock the HTTP client, not your service-that-calls-HTTP
• Mock the message broker, not the consumer-that-uses-the-broker
• Mock time, randomness, and the file system at the lowest seam (injected clock, seeded RNG, in-memory FS)
• Never mock the database in integration tests — use testcontainers

Step 4: Set per-component coverage targets

Not "80% across the project." Per-component:

• Pure business logic: 90%+ line + branch coverage
• Adapters (HTTP/DB clients): integration-test coverage (line % less meaningful)
• UI components: behavior coverage (clicks, keyboard, screen reader), not snapshot
• Glue code: lower bar, but every error path tested

Document each target in .forge/testing-strategy.md with rationale.

Step 5: Write golden tests for invariants

Identify 3-10 invariants per critical module (e.g., "ownership isolation never broken", "no negative balance possible", "no orphan rows after delete"). Write a property/golden test per invariant. These survive refactors.

Step 6: Define flake quarantine policy

When a test flakes:

1. Quarantine immediately (skip with TODO + owner + deadline).
2. Find root cause within 5 business days (timing? shared state? real bug?).
3. Re-enable only after the root cause is fixed.
4. Quarantine over 5 days = delete or rewrite.

No retry-on-fail config in CI. Flakes are bugs, not noise.

Step 7: Define CI gates

In .forge/testing-strategy.md:

• Which tests run on every push, every PR, nightly
• Total CI budget (e.g., <10 min PR, <30 min nightly)
• What's allowed to block merge (failing tests, coverage drop on critical modules)
• What's monitoring-only (flake rate, suite duration trend)

Step 8: Header

Prepend a forge:meta header to .forge/testing-strategy.md (generated_by: testing-strategy, generated_at: <ISO 8601 UTC with Z>, depends_on: [.forge/prd.md] — paths only, never hashes, generated_from: {.forge/prd.md: <upstream content_hash AT generation time>}, content_hash: <sha256 first 8 of THIS file's body>). See forge-dependency-graph.

Verification

• .forge/testing-strategy.md written with per-component coverage targets and rationale
• Critical user paths each have at least one e2e test
• Boundary modules (HTTP / DB / queue) have integration tests against real dependencies (testcontainers or similar)
• Pure logic modules have unit tests with >90% branch coverage
• No mock of an internal collaborator (only mocks at seams)
• No test takes >5 seconds without a documented exception
• Flake rate <1% over the trailing 30 days (or quarantine policy active)
• CI runs in <10 minutes for PRs
• No retries: N config used to mask flakes