Mutation Testing

Mutation Testing

Mutation testing answers: "Would my tests catch this bug?" by actually introducing bugs and running tests.

Execution Workflow

CRITICAL: This skill actually mutates code and runs tests. Follow this exact process:

Step 1: Identify Target Code

# Get changed files on the branch
git diff main...HEAD --name-only | grep -E '\.(ts|js|tsx|jsx|vue)$' | grep -v '\.test\.' | grep -v '\.spec\.'

Step 2: For Each Function to Test

Execute this loop for each mutation:

1. READ the original file and note exact content
2. APPLY one mutation (edit the code)
3. RUN tests: pnpm test --run (or specific test file)
4. RECORD result: KILLED (test failed) or SURVIVED (test passed)
5. RESTORE original code immediately
6. Repeat for next mutation

Step 3: Report Results

After all mutations, provide a summary table:

| Mutation | Location | Result | Action Needed |
|----------|----------|--------|---------------|
| `>` → `>=` | file.ts:42 | SURVIVED | Add boundary test |
| `&&` → `||` | file.ts:58 | KILLED | None |

Mutation Operators to Apply

Priority 1: Boundary Mutations (Most Likely to Survive)

Priority 2: Boolean Logic Mutations

Priority 3: Arithmetic Mutations

Priority 4: Return/Early Exit Mutations

Priority 5: Statement Removal

Practical Execution Example

Example: Testing a Validation Function

Original code (src/utils/validation.ts:15):

export function isValidAge(age: number): boolean {
  return age >= 18 && age <= 120;
}

Mutation 1: Change >= to >

export function isValidAge(age: number): boolean {
  return age > 18 && age <= 120;  // MUTATED
}

Run tests:

pnpm test --run src/__tests__/validation.test.ts

Result: Tests PASS → SURVIVED (Bad! Need test for isValidAge(18))

Restore original code immediately

Mutation 2: Change && to ||

export function isValidAge(age: number): boolean {
  return age >= 18 || age <= 120;  // MUTATED
}

Run tests:

pnpm test --run src/__tests__/validation.test.ts

Result: Tests FAIL → KILLED (Good! Tests catch this bug)

Restore original code immediately

Results Interpretation

Mutant States

Mutation Score

Score = (Killed + Timeout) / Total Mutations * 100

Fixing Surviving Mutants

When a mutant survives, add a test that would catch it:

Surviving: Boundary mutation (>= → >)

// Add boundary test
it('accepts exactly 18 years old', () => {
  expect(isValidAge(18)).toBe(true);  // Would fail if >= became >
});

Surviving: Logic mutation (&& → ||)

// Add test with mixed conditions
it('rejects when only one condition met', () => {
  expect(isValidAge(15)).toBe(false);  // Would pass if && became ||
});

Surviving: Statement removal

// Add side effect verification
it('saves to database', async () => {
  await processOrder(order);
  expect(db.save).toHaveBeenCalledWith(order);  // Would fail if save removed
});

Quick Checklist During Mutation

For each mutation, ask:

1. Before mutating: Does a test exist for this code path?
2. After running tests: Did any test actually fail?
3. If survived: What specific test would catch this?
4. After fixing: Re-run mutation to confirm killed

Common Surviving Mutation Patterns

Tests Only Check Happy Path

// WEAK: Only tests success case
it('validates', () => {
  expect(validate(goodInput)).toBe(true);
});

// STRONG: Tests both cases
it('validates good input', () => {
  expect(validate(goodInput)).toBe(true);
});
it('rejects bad input', () => {
  expect(validate(badInput)).toBe(false);
});

Tests Use Identity Values

// WEAK: Mutation survives
expect(multiply(5, 1)).toBe(5);  // 5*1 = 5/1 = 5

// STRONG: Mutation detected
expect(multiply(5, 3)).toBe(15);  // 5*3 ≠ 5/3

Tests Don't Assert Return Values

// WEAK: No return value check
it('processes', () => {
  process(data);  // No assertion!
});

// STRONG: Asserts outcome
it('processes', () => {
  const result = process(data);
  expect(result).toEqual(expected);
});

Important Rules

1. ALWAYS restore original code after each mutation
2. Run tests immediately after applying mutation
3. One mutation at a time - don't combine mutations
4. Focus on changed code - prioritize branch diff
5. Track all results - report full mutation summary

Summary Report Template

After completing mutation testing, provide:

## Mutation Testing Results

**Target**: `src/features/workout/utils.ts` (functions: X, Y, Z)
**Total Mutations**: 12
**Killed**: 9
**Survived**: 3
**Score**: 75%

### Surviving Mutants (Action Required)

| # | Location | Original | Mutated | Suggested Test |
|---|----------|----------|---------|----------------|
| 1 | line 42 | `>=` | `>` | Test boundary value |
| 2 | line 58 | `&&` | `\|\|` | Test mixed conditions |
| 3 | line 71 | `emit()` | removed | Verify event emission |

### Killed Mutants (Tests Effective)

- Line 35: `+` → `-` killed by `calculation.test.ts`
- Line 48: `true` → `false` killed by `validate.test.ts`
- ...

Related Skills

• systematic-debugging - Root cause analysis
• testing-conventions - Query priority, expect.poll()
• vue-integration-testing - Page objects, browser mode
• vitest-mocking - Test doubles and mocking patterns