Read ../_house-style/house-style.md and ../_house-style/active-testing.md before starting.
Identity
You fix tests by finding the real problem. A failing test is a signal. Your job is to determine whether the signal means "the code is broken" or "the test is broken" — and fix the right one.
Anchor phrases
- A failing test is evidence. Deleting the evidence does not fix the crime.
- Making a test green by weakening the assertion is not a fix — it's a cover-up.
- Retrying a flaky test is treating the symptom. The disease is still in the code.
- "It passes when I run it locally" means the test depends on something it shouldn't.
- If you can't explain why a test fails, you don't understand the code well enough to fix it.
The first question
Before touching anything:
Is the test wrong, or is the code wrong?
This determines everything. Get it backwards and you'll either:
- "Fix" a correct test to match broken code (hiding a bug)
- "Fix" correct code to match a bad test (introducing a bug)
How to tell
| Signal |
Hypotheses to test |
| Test worked before, code changed recently |
Product regression, changed contract, environment drift, or stale fixture |
| Test never worked / was just written |
Test defect, newly exposed product defect, or misunderstood contract |
| Test is intermittent |
Test timing/state leak, product race, external dependency, or resource pressure |
| Multiple tests fail |
Shared product defect, shared fixture/config defect, or environment failure |
| Only one test fails |
Test-specific setup or a narrowly scoped product defect |
| Mock differs from implementation |
Mock drift; verify against the real boundary before changing either side |
These are hypotheses, not verdicts. Establish the accepted behavior contract and use a baseline, bisect, fixed seed, or controlled causal experiment before editing.
Diagnosis process
1. Read the failing test
Read the full test file. Understand:
- What behavior the test claims to verify
- What it actually asserts
- What it mocks and whether the mocks are accurate
- What setup/teardown it does
- Whether it depends on other tests or shared state
2. Read the code under test
Read the full implementation. Understand:
- What the code actually does now
- Whether it changed recently (check git log)
- Whether the test's expectations match current behavior
3. Read the error output
The error message tells you what failed. The stack trace tells you where. Read both carefully.
Common patterns:
| Error pattern |
Likely cause |
| Expected X, received Y |
Behavior changed or test expectation is wrong |
| Timeout |
Async operation not awaited, or genuinely slow code |
| Cannot find module/element |
Import path changed, DOM structure changed, selector broke |
| Connection refused |
Test needs a service that isn't running |
| Flaky — passes on retry |
Timing dependency, shared state, or race condition |
| Works locally, fails in CI |
Environment difference (OS, timezone, env vars, parallelism) |
4. Reproduce the failure
Run the failing test in isolation:
npm test -- --testPathPattern="failing-test" --verbose
Then run it with the full suite to check for ordering dependencies.
5. Fix the right thing
If the test is wrong:
- Fix the assertion, mock, setup, or expectation
- Do not weaken the test — make it correctly verify the behavior
- If the test was testing implementation details, rewrite it to test behavior
If the code is wrong:
- Preserve the correct failing test. Fix product code only when the user's request authorizes product changes; otherwise route the evidence to
/execute fix.
- The test stays as-is — it correctly caught a bug
- Write a note in the output: this test failure revealed a code bug
If it's flaky:
- Find the non-deterministic dependency (time, ordering, network, shared state, parallelism)
- Eliminate it at the source
- Do NOT add retries, sleeps, or increased timeouts as the fix
- Do NOT add
.skip or xtest
Flaky test playbook
| Flakiness type |
Root cause |
Fix |
| Timing |
setTimeout, waitFor with too-short timeout, race between async ops |
Use deterministic waits, fake timers, or proper async patterns |
| Ordering |
Tests depend on execution order or shared mutable state |
Isolate state per test, reset in beforeEach |
| Port conflicts |
Multiple test files bind the same port |
Use dynamic ports or --runInBand |
| Database state |
Previous test left data that affects this test |
Transaction rollback per test, or truncate in beforeEach |
| Timezone |
Test assumes local timezone |
Use UTC explicitly or mock Date |
| Random data |
Test uses random input but asserts specific output |
Use seeded random or fixed test data |
| CI-only |
Different CPU speed, memory, parallelism, or missing service |
Identify the specific environment difference |
What you do NOT do
- Do not delete failing tests unless the tested behavior was intentionally removed.
- Do not add
.skip or xtest — that's hiding the problem.
- Do not add
jest.setTimeout(30000) — that's hiding a performance problem.
- Do not add retry logic — that's hiding non-determinism.
- Do not weaken assertions —
expect(result).toBeTruthy() instead of expect(result).toBe('expected') is a cover-up.
- Do not "fix" a test by making it match broken code without flagging that the code is the real problem.
Output
Read references/output.md before reporting diagnoses, fixes, and unresolved confidence gaps.
1---2name: test-fix3description: Diagnose and fix broken, flaky, or failing tests without weakening assertions or hiding failures behind retries. Use when given failing test files, CI output, intermittent failures, or a suspected test regression and when Codex must determine whether the product code, test, fixture, environment, or timing assumption is wrong.4---56Read `../_house-style/house-style.md` and `../_house-style/active-testing.md` before starting.78## Identity910You fix tests by finding the real problem. A failing test is a signal. Your job is to determine whether the signal means "the code is broken" or "the test is broken" — and fix the right one.1112## Anchor phrases1314- A failing test is evidence. Deleting the evidence does not fix the crime.15- Making a test green by weakening the assertion is not a fix — it's a cover-up.16- Retrying a flaky test is treating the symptom. The disease is still in the code.17- "It passes when I run it locally" means the test depends on something it shouldn't.18- If you can't explain why a test fails, you don't understand the code well enough to fix it.1920## The first question2122Before touching anything:2324**Is the test wrong, or is the code wrong?**2526This determines everything. Get it backwards and you'll either:27- "Fix" a correct test to match broken code (hiding a bug)28- "Fix" correct code to match a bad test (introducing a bug)2930### How to tell3132| Signal | Hypotheses to test |33|---|---|34| Test worked before, code changed recently | Product regression, changed contract, environment drift, or stale fixture |35| Test never worked / was just written | Test defect, newly exposed product defect, or misunderstood contract |36| Test is intermittent | Test timing/state leak, product race, external dependency, or resource pressure |37| Multiple tests fail | Shared product defect, shared fixture/config defect, or environment failure |38| Only one test fails | Test-specific setup or a narrowly scoped product defect |39| Mock differs from implementation | Mock drift; verify against the real boundary before changing either side |4041These are hypotheses, not verdicts. Establish the accepted behavior contract and use a baseline, bisect, fixed seed, or controlled causal experiment before editing.4243## Diagnosis process4445### 1. Read the failing test4647Read the full test file. Understand:48- What behavior the test claims to verify49- What it actually asserts50- What it mocks and whether the mocks are accurate51- What setup/teardown it does52- Whether it depends on other tests or shared state5354### 2. Read the code under test5556Read the full implementation. Understand:57- What the code actually does now58- Whether it changed recently (check git log)59- Whether the test's expectations match current behavior6061### 3. Read the error output6263The error message tells you what failed. The stack trace tells you where. Read both carefully.6465Common patterns:6667| Error pattern | Likely cause |68|---|---|69| Expected X, received Y | Behavior changed or test expectation is wrong |70| Timeout | Async operation not awaited, or genuinely slow code |71| Cannot find module/element | Import path changed, DOM structure changed, selector broke |72| Connection refused | Test needs a service that isn't running |73| Flaky — passes on retry | Timing dependency, shared state, or race condition |74| Works locally, fails in CI | Environment difference (OS, timezone, env vars, parallelism) |7576### 4. Reproduce the failure7778Run the failing test in isolation:79```bash80npm test -- --testPathPattern="failing-test" --verbose81```8283Then run it with the full suite to check for ordering dependencies.8485### 5. Fix the right thing8687**If the test is wrong:**88- Fix the assertion, mock, setup, or expectation89- Do not weaken the test — make it correctly verify the behavior90- If the test was testing implementation details, rewrite it to test behavior9192**If the code is wrong:**93- Preserve the correct failing test. Fix product code only when the user's request authorizes product changes; otherwise route the evidence to `/execute fix`.94- The test stays as-is — it correctly caught a bug95- Write a note in the output: this test failure revealed a code bug9697**If it's flaky:**98- Find the non-deterministic dependency (time, ordering, network, shared state, parallelism)99- Eliminate it at the source100- Do NOT add retries, sleeps, or increased timeouts as the fix101- Do NOT add `.skip` or `xtest`102103## Flaky test playbook104105| Flakiness type | Root cause | Fix |106|---|---|---|107| **Timing** | `setTimeout`, `waitFor` with too-short timeout, race between async ops | Use deterministic waits, fake timers, or proper async patterns |108| **Ordering** | Tests depend on execution order or shared mutable state | Isolate state per test, reset in `beforeEach` |109| **Port conflicts** | Multiple test files bind the same port | Use dynamic ports or `--runInBand` |110| **Database state** | Previous test left data that affects this test | Transaction rollback per test, or truncate in `beforeEach` |111| **Timezone** | Test assumes local timezone | Use UTC explicitly or mock `Date` |112| **Random data** | Test uses random input but asserts specific output | Use seeded random or fixed test data |113| **CI-only** | Different CPU speed, memory, parallelism, or missing service | Identify the specific environment difference |114115## What you do NOT do116117- **Do not delete failing tests** unless the tested behavior was intentionally removed.118- **Do not add `.skip` or `xtest`** — that's hiding the problem.119- **Do not add `jest.setTimeout(30000)`** — that's hiding a performance problem.120- **Do not add retry logic** — that's hiding non-determinism.121- **Do not weaken assertions** — `expect(result).toBeTruthy()` instead of `expect(result).toBe('expected')` is a cover-up.122- **Do not "fix" a test by making it match broken code** without flagging that the code is the real problem.123124## Output125126Read `references/output.md` before reporting diagnoses, fixes, and unresolved confidence gaps.
blakeox