Test Manager

Test Manager

Overview

Owns a project's automated tests as a portfolio. Decides which level each behavior belongs at, routes the work to the matching specialist skill, triages defects in the existing suite, and verifies the whole suite before the work is called done.

Three specialists do the authoring:

Full taxonomy, classification rules, and the triage catalogue: references/test-taxonomy.md The six house rules below are duplicated in every skill in this family so each is usable standalone — verbatim except where a level inserts a clause naming its own failure modes. The canonical copy is references/house-rules.md.

House Rules

Routing

digraph routing {
    "Test work requested" [shape=doublecircle];
    "Whole suite / audit / cleanup?" [shape=diamond];
    "Spans 2+ test levels?" [shape=diamond];
    "Level obvious?" [shape=diamond];
    "Inventory + triage, then fan out" [shape=box];
    "Fan out per level" [shape=box];
    "Classify the behavior" [shape=box];
    "Invoke the one specialist inline" [shape=box];

    "Test work requested" -> "Whole suite / audit / cleanup?";
    "Whole suite / audit / cleanup?" -> "Inventory + triage, then fan out" [label="yes"];
    "Whole suite / audit / cleanup?" -> "Spans 2+ test levels?" [label="no"];
    "Spans 2+ test levels?" -> "Fan out per level" [label="yes"];
    "Spans 2+ test levels?" -> "Level obvious?" [label="no"];
    "Level obvious?" -> "Invoke the one specialist inline" [label="yes"];
    "Level obvious?" -> "Classify the behavior" [label="no"];
    "Classify the behavior" -> "Invoke the one specialist inline";
}

Single level → invoke that specialist directly, in this context. Do not dispatch a subagent to cover one function. Orchestration costs a round trip and a context copy; spend it only when there is parallel work to win.

Default shape. Google's ratio is roughly 70% unit / 20% integration / 10% end-to-end (Just Say No to More End-to-End Tests); the SWE Book states it as 80/20 small tests to larger ones. A change that seems to need a new E2E test almost always needs a unit test instead. Push work down the pyramid until the level genuinely cannot observe the behavior.

The Upsert Decision

Google's rule: strive for unchanging tests. After a test is written, it should not need to change as the system is refactored, extended, or fixed. This matrix governs every level.

That matrix governs what a test asserts about product behavior. Repairing a test that is itself defective — no assertion, tautological, asserting copy, asserting a generated query, unfailable, flaky — is a different axis, and is always allowed regardless of what the product did. If the test is currently failing, establish why before repairing it: a defective test can still be failing for a real product reason, and repairing it green hides that. Confirm the product behavior is correct first, then repair, then observe the repaired test fail against the broken behavior. Name the defect from the catalogue (references/test-taxonomy.md) before you touch the test.

Deleting a test is legitimate in four cases: the behavior it covers no longer exists, it duplicates another test at the same level, it is itself defective with no salvageable assertion, or the behavior is now covered at a cheaper level and that replacement was written and observed failing first. Say which case applies. "It was failing" and "it was slow" are never reasons to delete a test.

Inventory and Triage

Run this whenever the scope is the suite rather than a single change.

1. Discover. Locate test roots, runner config, and the commands that run each level. Record the exact command per level — specialists need it to run their own slice.
2. Classify. Assign every existing test file to a level using references/test-taxonomy.md. Mislabeled tests are common: an "integration" test with every collaborator mocked is a unit test, and a "unit" test that opens a socket is not.
3. Run each level once and record failures, flakes, and skips. Re-run any suspected flake at least five times — more when the rate looks low — and report the measured rate, not a guess.
4. Triage. Catalogue defects against references/test-taxonomy.md — flaky, skipped, duplicated, obsolete, assertion-free, copy-asserting, library-testing, change-detector, unfailable.
5. Assign. Every triaged defect goes to the specialist that owns its level, with the same ownership rule attached. Nothing is filed as a follow-up.

Dispatch Rules

When fanning out:

• One subagent per level, each told to load its specialist skill by name before doing anything.
• Name the model explicitly. Sonnet for the inventory sweep — it is a search. Opus for the authoring agents — test design is judgment work.
• Assign disjoint paths. Two agents editing the same fixtures, factories, or conftest.py will collide. Shared helper files stay under this skill's ownership; a specialist that needs one changed reports the change back instead of making it. Each specialist skill states this constraint too, so a dispatched agent sees it in the skill it actually loaded.
• Each subagent runs only its own scoped test command. Concurrent full-suite runs fight over ports, databases, snapshot files, and coverage output.
• No subagent runs the code review. It reports back; the review happens once, here.
• Each subagent reports what it added, what it updated, what it deleted and why, every defect it triaged, and the pass/fail output it actually saw.

Completion Gate

1. The full suite runs once, after every specialist has reported. If a specialist ran inline in this context and already completed that run and the review below, gates 1 and 4 are satisfied by it — do not repeat either.
2. Zero failures and zero flakes, except any explicitly reported under House Rule 6 with file:line and a size estimate. Every skip is fixed and re-enabled, deleted under one of the four deletion cases with the case named, or reported the same way; a skip survives unmentioned only on an explicit user instruction to keep it.
3. Every new test has been observed failing for its stated reason at least once.
4. necturalabs:iterative-code-review runs once over the union of all changes. Three partial reviews are not a review. If the changes fall in necturalabs:iterative-security-audit's trigger list — see its description, which is broader than auth and crypto — that audit runs first and chains into the review.

If that review returns findings about tests, fix them here, under these house rules. Do not re-enter a test skill to handle review feedback — the review runs once per invocation chain, and re-entering restarts a loop that never terminates.

Rationalizations — All Rejected

Captured from agents doing this work without this skill.

Red Flags — Stop

• About to write "flagging this for a follow-up"
• About to report done without running the full suite
• About to dispatch a subagent for a single file's worth of tests
• About to let a dispatched subagent run the whole suite instead of its own slice
• About to skip the code review because "it's only tests"
• Counting a test as a regression test without having watched it fail on the unfixed code