Testing Strategy

Testing Strategy

Intro

Pick the cheapest test that gives you confidence the code is correct. Many fast unit tests, fewer integration tests, and a thin layer of end-to-end tests for the critical paths — that is the testing pyramid, and it still works.

Overview

Match the test type to the code

• Pure logic (calculations, transformations) → unit tests.
• Integration points (database, HTTP API, file I/O) → integration tests.
• User workflows (end-to-end paths) → E2E tests.

Apply the testing pyramid

• Many unit tests — fast, isolated, cheap to run on every commit.
• Fewer integration tests — slower, exercise the boundary between your code and external systems.
• Minimal E2E tests — slowest and most flaky, reserved for the critical happy paths a user must be able to complete.

Naming and shape

Name tests test_<function>_<scenario>_<expected_result> so failures self-document. Each test follows the AAA pattern: Arrange the inputs, Act by calling the unit, Assert the expected outcome. One logical assertion per test where practical.

Prioritize what you test

• Error paths and edge cases over happy paths
• Public API over internal implementation details
• Complex branching logic over trivial getters/setters

Coverage as a smoke alarm

Aim for ~80% coverage on business logic. Do not chase 100% everywhere — coverage is a smoke alarm, not a goal. The last 20% is usually trivial code, generated code, or defensive paths that cost more to test than they save.

Example

User says: "I added a new calculate_discount() function, what tests do I need?" The agent recommends:

• test_calculate_discount_percentage_applied_correctly
• test_calculate_discount_zero_amount_returns_zero
• test_calculate_discount_negative_amount_returns_error
• test_calculate_discount_exceeds_max_caps_at_limit

One happy path, one boundary, one error case, one limit case.

Gotchas

Agent-specific failure modes — provider-neutral pause-and-self-check items:

• Testing implementation details (private methods, internal state). Tests that verify internals couple the test suite to the implementation — any structural change breaks tests even when behavior is correct. Test through the public API at the appropriate level.
• Slow unit tests that hit disk or network. A unit test that opens a file, makes an HTTP call, or connects to a database is an integration test with a misleading name. It will be slow and flaky in CI. Mock or stub at the boundary; keep unit tests pure.
• E2E tests for every code path. E2E tests are expensive to write, slow to run, and flaky to maintain. They should cover the critical user journeys that must work — not every conditional branch. Those belong in unit and integration tests.
• Coverage goals as the only quality metric. 100% line coverage with assertions that never fail (e.g. assert True) provides zero confidence. Coverage is a smoke alarm — it tells you what was executed, not whether the behavior was verified.
• No test for the unhappy path. Error cases and edge cases are where bugs live. Happy-path-only tests give false confidence. For every behavior, consider: what happens with empty input, invalid input, a missing dependency, a limit exceeded?
• Deviating from the testing pyramid without understanding the trade-offs. If the test suite is inverted (mostly E2E) or trophy-shaped (mostly integration), that may be appropriate for the architecture — but it should be a deliberate decision with documented trade-offs, not an accident.
• Tests that pass when the production code is deleted. A test suite that stays green after the module is removed was not testing anything. Run a mutation test or delete the module temporarily to verify the suite would actually catch a regression.

Full reference

Choosing the right level

When to break the pyramid

• Inverted pyramid (lots of E2E): the system has weak internal seams; integration tests would be more valuable but the architecture makes them hard. Refactor for testability.
• Trophy shape (heavy integration): typical for thin services where most logic is glue between external systems; the unit layer is genuinely small.
• Diamond (heavy integration, light unit and E2E): common for microservices that mostly orchestrate.

The pyramid is a default, not a law.

Anti-patterns

• Testing implementation details (e.g., asserting on private method calls) — couples tests to refactoring
• Slow unit tests because they hit the disk or network
• E2E tests that exercise every code path instead of critical flows
• Coverage goals as the only quality metric
• Tests that pass when the production code is deleted (they weren't testing anything)
• Skipping tests for "trivial" code that turns out to be wrong