Testing Strategies Skill
Test the behavior, not the implementation. Test the boundaries, not the happy path.
Testing Pyramid
| Level |
Volume |
Speed |
Cost to Maintain |
What It Catches |
| Unit |
Many (70%) |
< 10ms each |
Low |
Logic errors, edge cases, regressions |
| Integration |
Some (20%) |
< 1s each |
Medium |
Wiring bugs, API contracts, data flow |
| E2E |
Few (10%) |
5-30s each |
High |
User journey failures, deployment issues |
Anti-pattern: Inverted pyramid (too many E2E, few unit) → slow CI, flaky tests, hard to debug.
Anti-pattern: Ice cream cone (manual testing on top of everything) → doesn't scale.
Unit Test Pattern (AAA)
test('should calculate discount when order exceeds $100', () => {
// Arrange
const order = createOrder({ subtotal: 150, customerTier: 'gold' });
// Act
const discount = calculateDiscount(order);
// Assert
expect(discount).toBe(15); // 10% for gold tier
});
Naming convention: should [expected behavior] when [condition] — reads as a specification.
Test Types Beyond the Pyramid
| Type |
Purpose |
When to Use |
Example |
| Snapshot |
Detect unexpected output changes |
UI components, serialized data |
expect(render(<Button/>)).toMatchSnapshot() |
| Contract |
Verify API shape between services |
Microservices, public APIs |
Pact, OpenAPI validation |
| Property-based |
Find edge cases humans miss |
Pure functions, parsers, serializers |
fc.assert(fc.property(fc.string(), s => decode(encode(s)) === s)) |
| Mutation |
Verify tests actually catch bugs |
Critical business logic |
Stryker, pitest |
| Performance |
Catch regressions in speed/memory |
Hot paths, API endpoints |
Benchmark before/after |
| Smoke |
Verify deployment didn't break basics |
Post-deploy, staging |
Hit health endpoint + key pages |
What to Mock (and What Not To)
| Mock This |
Why |
Don't Mock This |
Why |
| External HTTP APIs |
Unreliable, slow, costly |
Your own business logic |
You'd be testing your mocks |
| Database in unit tests |
Slow, stateful |
Database in integration tests |
That's the whole point |
Time (Date.now) |
Non-deterministic |
Pure functions |
Already deterministic |
| File system |
Side effects |
In-memory equivalents |
Faster than mocking |
| Random/UUID |
Non-deterministic |
Framework internals |
Not your responsibility |
Coverage Philosophy
| Range |
Interpretation |
Action |
| < 50% |
Probably missing critical paths |
Increase |
| 50-70% |
Reasonable for most projects |
Focus on changed code |
| 70-85% |
Good, diminishing returns starting |
Maintain, don't chase |
| 85-100% |
Often wasteful unless safety-critical |
Review if the effort is worth it |
The real metric: Coverage of changed code in each PR, not overall percentage.
What coverage doesn't tell you: That your tests assert the right things. 100% coverage with no assertions is useless.
What NOT to Test
| Don't Test |
Why |
Instead |
| Third-party library internals |
Not your code |
Trust it (or pick a different library) |
| Framework behavior |
Already tested upstream |
Test your code that uses the framework |
| Private implementation details |
Breaks on refactor |
Test the public interface |
| Trivial getters/setters |
No logic to break |
Only if they have side effects |
| Generated code |
Changes on regeneration |
Test the generator, not the output |
Test Quality Signals
| Good Test |
Bad Test |
| Fails when behavior breaks |
Fails when implementation changes |
| One clear reason to fail |
Multiple assertions testing different things |
| Self-contained |
Depends on other test order |
| Fast (< 100ms unit) |
Slow due to unnecessary setup |
| Readable as documentation |
Requires reading source to understand |
| Deterministic |
Flaky (passes sometimes) |
Mission-Critical Testing (NASA Standards)
For safety-critical or high-reliability projects, apply NASA/JPL Power of 10 testing patterns:
Bounded Behavior Testing
| What to Test |
Why |
Example |
| Recursion with depth |
R1: Prevent stack overflow |
test('walk() stops at maxDepth', () => { walk(deep, 5); expect(visited).length.lessThan(100); }) |
| Loop iteration limits |
R2: Prevent infinite loops |
test('parser terminates on malformed input', () => { expect(() => parse(corrupt, { maxIterations: 1000 })).not.toHang(); }) |
| Collection size bounds |
R3: Prevent memory exhaustion |
test('cache evicts when full', () => { fillCache(1000); expect(cache.size).toBeLessThanOrEqual(MAX_CACHE); }) |
Assertion Coverage Testing
| Pattern |
What to Test |
NASA Rule |
| Entry assertions |
Function preconditions hold |
R5 |
| Boundary assertions |
Range checks are enforced |
R3, R5 |
| State assertions |
Invariants preserved |
R5 |
// Test that assertions fire on invalid input
test('validateUser throws on undefined', () => {
expect(() => validateUser(undefined)).toThrow('assertion failed');
});
// Test that bounds are enforced
test('processItems rejects oversized batch', () => {
const items = new Array(10001).fill({});
expect(() => processItems(items)).toThrow('exceeds MAX_BATCH_SIZE');
});
Critical Path Coverage
| Path Type |
Coverage Target |
Testing Approach |
| Error handlers |
100% |
Force each error condition |
| Boundary conditions |
100% |
Test at limit, limit-1, limit+1 |
| Timeout/cancellation |
100% |
Test early abort, late abort |
| Resource cleanup |
100% |
Force failure after acquisition |
TDD Cycle
| Step |
Action |
Common Mistake |
| Red |
Write a failing test |
Writing too much test (test the next small behavior) |
| Green |
Make it pass with minimal code |
Over-engineering the solution |
| Refactor |
Clean up while green |
Skipping this step (accumulates debt) |
TDD is not always the right choice: It works best for well-understood requirements. For exploratory code, write tests after the design stabilizes.
Flaky Test Triage
| Pattern |
Likely Cause |
Fix |
| Fails 1 in 10 runs |
Timing/race condition |
Add proper waits, remove shared state |
| Fails only in CI |
Environment difference |
Pin versions, use containers |
| Fails after another test |
Test pollution |
Isolate setup/teardown |
| Fails on slow machines |
Hardcoded timeouts |
Use retry with backoff or event-based waits |
1---2name: testing-strategies3description: Systematic testing for confidence without over-testing — the right test at the right level4---56# Testing Strategies Skill78> Test the behavior, not the implementation. Test the boundaries, not the happy path.910## Testing Pyramid1112| Level | Volume | Speed | Cost to Maintain | What It Catches |13| ----- | ------ | ----- | ---------------- | --------------- |14| Unit | Many (70%) | < 10ms each | Low | Logic errors, edge cases, regressions |15| Integration | Some (20%) | < 1s each | Medium | Wiring bugs, API contracts, data flow |16| E2E | Few (10%) | 5-30s each | High | User journey failures, deployment issues |1718**Anti-pattern**: Inverted pyramid (too many E2E, few unit) → slow CI, flaky tests, hard to debug.19**Anti-pattern**: Ice cream cone (manual testing on top of everything) → doesn't scale.2021## Unit Test Pattern (AAA)2223```typescript24test('should calculate discount when order exceeds $100', () => {25 // Arrange26 const order = createOrder({ subtotal: 150, customerTier: 'gold' });27 28 // Act29 const discount = calculateDiscount(order);30 31 // Assert32 expect(discount).toBe(15); // 10% for gold tier33});34```3536**Naming convention**: `should [expected behavior] when [condition]` — reads as a specification.3738## Test Types Beyond the Pyramid3940| Type | Purpose | When to Use | Example |41| ---- | ------- | ----------- | ------- |42| **Snapshot** | Detect unexpected output changes | UI components, serialized data | `expect(render(<Button/>)).toMatchSnapshot()` |43| **Contract** | Verify API shape between services | Microservices, public APIs | Pact, OpenAPI validation |44| **Property-based** | Find edge cases humans miss | Pure functions, parsers, serializers | `fc.assert(fc.property(fc.string(), s => decode(encode(s)) === s))` |45| **Mutation** | Verify tests actually catch bugs | Critical business logic | Stryker, pitest |46| **Performance** | Catch regressions in speed/memory | Hot paths, API endpoints | Benchmark before/after |47| **Smoke** | Verify deployment didn't break basics | Post-deploy, staging | Hit health endpoint + key pages |4849## What to Mock (and What Not To)5051| Mock This | Why | Don't Mock This | Why |52| --------- | --- | --------------- | --- |53| External HTTP APIs | Unreliable, slow, costly | Your own business logic | You'd be testing your mocks |54| Database in unit tests | Slow, stateful | Database in integration tests | That's the whole point |55| Time (`Date.now`) | Non-deterministic | Pure functions | Already deterministic |56| File system | Side effects | In-memory equivalents | Faster than mocking |57| Random/UUID | Non-deterministic | Framework internals | Not your responsibility |5859## Coverage Philosophy6061| Range | Interpretation | Action |62| ----- | -------------- | ------ |63| < 50% | Probably missing critical paths | Increase |64| 50-70% | Reasonable for most projects | Focus on changed code |65| 70-85% | Good, diminishing returns starting | Maintain, don't chase |66| 85-100% | Often wasteful unless safety-critical | Review if the effort is worth it |6768**The real metric**: Coverage of *changed code* in each PR, not overall percentage.6970**What coverage doesn't tell you**: That your tests assert the right things. 100% coverage with no assertions is useless.7172## What NOT to Test7374| Don't Test | Why | Instead |75| ---------- | --- | ------- |76| Third-party library internals | Not your code | Trust it (or pick a different library) |77| Framework behavior | Already tested upstream | Test your code that uses the framework |78| Private implementation details | Breaks on refactor | Test the public interface |79| Trivial getters/setters | No logic to break | Only if they have side effects |80| Generated code | Changes on regeneration | Test the generator, not the output |8182## Test Quality Signals8384| Good Test | Bad Test |85| --------- | -------- |86| Fails when behavior breaks | Fails when implementation changes |87| One clear reason to fail | Multiple assertions testing different things |88| Self-contained | Depends on other test order |89| Fast (< 100ms unit) | Slow due to unnecessary setup |90| Readable as documentation | Requires reading source to understand |91| Deterministic | Flaky (passes sometimes) |9293## Mission-Critical Testing (NASA Standards)9495For safety-critical or high-reliability projects, apply NASA/JPL Power of 10 testing patterns:9697### Bounded Behavior Testing9899| What to Test | Why | Example |100| ------------ | --- | ------- |101| Recursion with depth | R1: Prevent stack overflow | `test('walk() stops at maxDepth', () => { walk(deep, 5); expect(visited).length.lessThan(100); })` |102| Loop iteration limits | R2: Prevent infinite loops | `test('parser terminates on malformed input', () => { expect(() => parse(corrupt, { maxIterations: 1000 })).not.toHang(); })` |103| Collection size bounds | R3: Prevent memory exhaustion | `test('cache evicts when full', () => { fillCache(1000); expect(cache.size).toBeLessThanOrEqual(MAX_CACHE); })` |104105### Assertion Coverage Testing106107| Pattern | What to Test | NASA Rule |108| ------- | ------------ | --------- |109| Entry assertions | Function preconditions hold | R5 |110| Boundary assertions | Range checks are enforced | R3, R5 |111| State assertions | Invariants preserved | R5 |112113```typescript114// Test that assertions fire on invalid input115test('validateUser throws on undefined', () => {116 expect(() => validateUser(undefined)).toThrow('assertion failed');117});118119// Test that bounds are enforced120test('processItems rejects oversized batch', () => {121 const items = new Array(10001).fill({});122 expect(() => processItems(items)).toThrow('exceeds MAX_BATCH_SIZE');123});124```125126### Critical Path Coverage127128| Path Type | Coverage Target | Testing Approach |129| --------- | --------------- | ---------------- |130| Error handlers | 100% | Force each error condition |131| Boundary conditions | 100% | Test at limit, limit-1, limit+1 |132| Timeout/cancellation | 100% | Test early abort, late abort |133| Resource cleanup | 100% | Force failure after acquisition |134135## TDD Cycle136137| Step | Action | Common Mistake |138| ---- | ------ | -------------- |139| **Red** | Write a failing test | Writing too much test (test the next small behavior) |140| **Green** | Make it pass with minimal code | Over-engineering the solution |141| **Refactor** | Clean up while green | Skipping this step (accumulates debt) |142143**TDD is not always the right choice**: It works best for well-understood requirements. For exploratory code, write tests after the design stabilizes.144145## Flaky Test Triage146147| Pattern | Likely Cause | Fix |148| ------- | ------------ | --- |149| Fails 1 in 10 runs | Timing/race condition | Add proper waits, remove shared state |150| Fails only in CI | Environment difference | Pin versions, use containers |151| Fails after another test | Test pollution | Isolate setup/teardown |152| Fails on slow machines | Hardcoded timeouts | Use retry with backoff or event-based waits |