Testing Boss
Tests exist to expose defects, not to keep CI green. A test that fails has done its job; a test that passes for the wrong reason is worse than none. The doctrine spans human and AI-generated code, LLM/agent features, and the CI that gates them; its body lives in references/ as language-agnostic pseudo-code.
Iron Laws
Apply every law that bears on the change under test. They subsume every anti-pattern named in the references; when two disagree, the lower-numbered one wins.
1. Test the behavior, never the mock.
2. Push every test to the lowest layer that can detect the failure.
3. When a test fails, fix production first — change the test only after writing why.
4. Real systems gate the merge. Mocks isolate; they do not validate.
5. Coverage is a flashlight. Mutation score is a quality probe. Neither is a target.
6. No test-only methods, branches, or flags leak into production code.
Reference router
The Iron Laws are the always-loaded tripwire; each reference is the contract. Match the task, read the listed file(s) in full before producing output, and apply every gate, pattern, and principle in them that bears on the work.
| When you are… |
Read in full |
| Deciding where a test belongs — layer, owner, boundary, or whether to write it at all |
references/foundations.md |
| Writing a test at any layer — selectors, waits, test data, isolation, what to mock |
references/patterns.md |
| Reviewing a test, smelling brittleness, or rebuilding a brittle suite |
references/antipatterns.md |
| Letting a coding agent generate, modify, or "fix" tests |
references/ai-writes-tests.md + references/antipatterns.md |
| Triaging flaky CI, designing gates, or choosing contract / property / mutation tests |
references/ci-automation.md |
| Designing an eval for an LLM/agent feature — oracle ladder, LLM-as-judge, RAG, trajectory vs outcome |
references/llm-eval.md |
Each reference ends with its own sources; references/sources.md is the consolidated bibliography for auditing any claim.
1---2name: testing-boss3description: Testing doctrine for tests that reveal bugs instead of passing for the wrong reason — spanning software and LLM/AI systems. Use when authoring or reviewing tests, adding a mock, deciding where a test belongs, letting a coding agent generate tests, triaging flaky CI, designing an eval suite for an LLM/agent feature, or rebuilding a brittle suite. Not for general code review, library debugging unrelated to tests, CI pipeline design beyond tests, or production observability.4---56# Testing Boss78Tests exist to expose defects, not to keep CI green. A test that fails has done its job; a test that passes for the wrong reason is worse than none. The doctrine spans human and AI-generated code, LLM/agent features, and the CI that gates them; its body lives in `references/` as language-agnostic pseudo-code.910## Iron Laws1112Apply every law that bears on the change under test. They subsume every anti-pattern named in the references; when two disagree, the lower-numbered one wins.1314```151. Test the behavior, never the mock.162. Push every test to the lowest layer that can detect the failure.173. When a test fails, fix production first — change the test only after writing why.184. Real systems gate the merge. Mocks isolate; they do not validate.195. Coverage is a flashlight. Mutation score is a quality probe. Neither is a target.206. No test-only methods, branches, or flags leak into production code.21```2223## Reference router2425The Iron Laws are the always-loaded tripwire; each reference is the contract. Match the task, read the listed file(s) **in full** before producing output, and apply every gate, pattern, and principle in them that bears on the work.2627| When you are… | Read in full |28| --- | --- |29| Deciding where a test belongs — layer, owner, boundary, or whether to write it at all | `references/foundations.md` |30| Writing a test at any layer — selectors, waits, test data, isolation, what to mock | `references/patterns.md` |31| Reviewing a test, smelling brittleness, or rebuilding a brittle suite | `references/antipatterns.md` |32| Letting a coding agent generate, modify, or "fix" tests | `references/ai-writes-tests.md` + `references/antipatterns.md` |33| Triaging flaky CI, designing gates, or choosing contract / property / mutation tests | `references/ci-automation.md` |34| Designing an eval for an LLM/agent feature — oracle ladder, LLM-as-judge, RAG, trajectory vs outcome | `references/llm-eval.md` |3536Each reference ends with its own sources; `references/sources.md` is the consolidated bibliography for auditing any claim.
compozy