E2E Test Manager
Overview
Authors and maintains tests that drive the assembled system the way a user does. This level buys
fidelity — proof that the pieces are wired together and the critical journeys work — at the highest
cost per test in runtime, flakiness, and diagnosis. The first job of this skill is to keep the level
small; the second is to make the tests that belong here deterministic.
Is This the Right Skill?
| The behavior needs |
Skill |
| A user driving the assembled system through its real interface |
this one |
| A real DB, HTTP server, queue, or filesystem, but no user |
necturalabs:integration-test-manager |
| Only our code, one process, no I/O |
necturalabs:unit-test-manager |
| More than one of the above, or a suite-wide audit |
necturalabs:test-manager |
Add one only for a journey that is critical to the business AND crosses boundaries no lower level
can observe. Validation rules, calculations, error mapping, and conditional rendering are all
cheaper and sharper one or two levels down.
House Rules
Workflow
- Discover the setup — the runner and its config, base URL, how the app under test is started,
how authentication is obtained, where fixtures and page objects live, and the command that runs
one spec.
- Justify the level. Write the journey as a sentence. If any step's risk is coverable lower
down, cover it there instead and keep this test to the wiring.
- Decide add / update / leave using the upsert matrix below. A UI restructure that preserves
behavior should not change a test that was written against roles and ids; if it does, the test
was written against the DOM.
- Set up state through the API, not the UI. Log in programmatically, seed data
programmatically, and start the test at the screen the journey begins on.
- Write the test with auto-waiting assertions and no fixed delays. See
references/e2e-test-rules.md.
- Observe it fail. Break the behavior it covers and confirm the failure names the right step.
- Run it repeatedly — at least five consecutive passes, and in the CI configuration if it
differs. A single green run says nothing at this level.
- Triage anything else the run surfaced under House Rule 6, classifying each defect
against
../test-manager/references/test-taxonomy.md — flaky, skipped, assertion-free,
unfailable, tautological, copy-asserting, library-testing, change-detector, duplicated,
obsolete, mislabeled, excessive setup.
When dispatched by necturalabs:test-manager
- Run only your level's command, never the full suite.
- Do not edit shared fixtures, factories, auth setup files, or any helper outside your assigned
paths — another specialist may be editing it right now. Report the change you need instead.
- Do not run the code review. Report back; the orchestrator runs it once over everything.
The Upsert Decision
Google's rule: strive for unchanging tests. After a test is written, it should not need to change as
the system is refactored, extended, or fixed.
| Change |
Existing tests |
New tests |
| Pure refactoring |
unchanged. If they break, either the refactoring changed behavior or the tests were written against implementation details — fix that cause, not the assertion. |
none |
| New feature |
unchanged |
add, covering the new behaviors only |
| Bug fix |
unchanged |
add a regression test, written first and observed failing |
| Behavior change |
the only case where editing an existing test is legitimate |
add for genuinely new behavior |
That matrix governs what a test asserts about product behavior. Repairing a test that is itself
defective — no assertion, tautological, asserting copy, asserting a generated query, unfailable,
flaky — is a different axis, and is always allowed regardless of what the product did. If the test
is currently failing, establish why before repairing it: a defective test can still be failing for
a real product reason, and repairing it green hides that. Confirm the product behavior is correct
first, then repair, then observe the repaired test fail against the broken behavior. Name the
defect from the catalogue (../test-manager/references/test-taxonomy.md) before you touch the test.
Deleting a test is legitimate in four cases: the behavior it covers no longer exists, it duplicates
another test at the same level, it is itself defective with no salvageable assertion, or the
behavior is now covered at a cheaper level and that replacement was written and observed failing
first. Say which case applies. "It was failing" and "it was slow" are never reasons to delete a test.
Quick Reference
Full rules with sources: references/e2e-test-rules.md
| Concern |
Rule |
| Selectors |
Role and accessible name first; a dedicated data-testid when there is no accessible handle. Never CSS class chains, never XPath, never nth-child, never a text sentence. |
| Waiting |
Web-first assertions that retry until the condition holds. Never a fixed sleep. Wait on a network alias or a state change, not a duration. |
| Auth |
Programmatic login, reused storage state. Never drive the login form in beforeEach. |
| Data |
The test creates and owns its data, keyed uniquely. Never rely on what a previous test left. |
| Isolation |
Fresh browser context per test: no shared cookies, storage, or session. |
| Cleanup |
Reset before the test, not after — an aborted run leaves after hooks unexecuted. |
| Assertions per test |
Batch them. A journey test asserts at each meaningful checkpoint; splitting into one-assertion tests multiplies setup cost with no benefit. |
| Third parties |
Mocked or stubbed at the network layer. |
| Debugging |
Traces and videos on failure, from CI. Screenshots alone rarely explain a race. |
| Runtime |
Shard in CI and push coverage down the pyramid. If the suite cannot finish in the time the team will actually wait, move journeys to cheaper levels — deletion only under the four cases above, and never because a test is slow. |
Rationalizations — All Rejected
| Excuse |
Reality |
| "An E2E test is quicker to write than restructuring for a unit test" |
You pay the write cost once and the run cost forever, with worse diagnosis. |
| "It's flaky, so I'll add a retry" |
Retries hide races. Find the condition the test should wait on. |
| "A short sleep is fine here" |
It is the number one cause of E2E flake and it will be copied into every new test. |
| "Checking the headline text proves the page rendered" |
Assert the URL, the landmark role, or a data-testid. Copy changes without defects. |
| "Logging in through the UI is more realistic" |
Cover login once, in its own test. Everywhere else it is slow, flaky setup. |
| "The staging data already has a suitable user" |
Then the test breaks when someone else edits that user. Create your own. |
| "Cleaning up in afterEach keeps it tidy" |
An aborted run skips it. Reset at the start. |
| "This test only fails in CI" |
That is a real difference in timing or environment, and it is a defect in the test. |
| "One assertion per test is best practice" |
That is a unit-test rule. At this level it multiplies the most expensive part. |
Red Flags — Stop
waitForTimeout, cy.wait(3000), Thread.sleep, or any bare duration- A selector containing a CSS class chain,
nth-child, XPath, or a full sentence
- A
beforeEach that logs in through the form
- A test that depends on another test having run first
- A retry count raised to make CI pass
- Conditional branching around a step that "sometimes" appears
- A new E2E test covering logic that a unit test could pin down
When Done
Report the journey covered and why it needs this level, what you added, updated, and deleted with
reasons, the failing observation for each new test, the exact command and its output, the number of
consecutive clean runs, and every defect triaged.
If you are running as a subagent dispatched by necturalabs:test-manager, stop here and report
back — it runs the full suite and the review once, over everything. Otherwise, including when
test-manager invoked this skill inline in your own context, run the full suite yourself and then
necturalabs:iterative-code-review over the changes — preceded by
necturalabs:iterative-security-audit when the changes fall in that skill's trigger list (see its
description; it is broader than auth and crypto), since it chains into the review. That run satisfies the orchestrator's completion
gate rather than adding a second one. If the review returns test findings, fix them here
under these rules rather than re-entering a test skill, so the review runs once per invocation
chain.
1---2name: e2e-test-manager3description: MUST invoke when writing, updating, or fixing end-to-end or browser tests — Playwright, Cypress, Selenium, WebDriver, or device automation that drives the assembled application as a user. Also use when an E2E suite is slow, flaky, or failing, or when someone proposes covering a change with a UI test.4---56# E2E Test Manager78## Overview910Authors and maintains tests that drive the assembled system the way a user does. This level buys11fidelity — proof that the pieces are wired together and the critical journeys work — at the highest12cost per test in runtime, flakiness, and diagnosis. The first job of this skill is to keep the level13small; the second is to make the tests that belong here deterministic.1415## Is This the Right Skill?1617| The behavior needs | Skill |18|--------------------|-------|19| A user driving the assembled system through its real interface | this one |20| A real DB, HTTP server, queue, or filesystem, but no user | `necturalabs:integration-test-manager` |21| Only our code, one process, no I/O | `necturalabs:unit-test-manager` |22| More than one of the above, or a suite-wide audit | `necturalabs:test-manager` |2324<HARD-GATE>25**Before writing an E2E test, prove it cannot be a lower-level test.** Google's published ratio is26roughly 70% unit, 20% integration, 10% end-to-end. An E2E test that fails does not say what broke —27the defect could be anywhere in the system — and its cost is paid on every CI run forever.2829Add one only for a journey that is critical to the business AND crosses boundaries no lower level30can observe. Validation rules, calculations, error mapping, and conditional rendering are all31cheaper and sharper one or two levels down.32</HARD-GATE>3334## House Rules3536<HARD-GATE>371. **Test our code, never a library or framework.** Assume third-party code works. If a test would38 fail when a dependency changes its output while our code is unchanged, it is testing the39 dependency — a component library's markup and a chart library's SVG included. Wrap the dependency40 and test our wrapper's behavior instead.412. **Never assert on human-readable copy.** Marketing sentences, labels, error prose, formatted42 dates, currency strings, and whole rendered or serialized documents change for non-behavioral43 reasons. Assert roles, ids and `data-*` attributes, element counts, status codes, error codes and44 types, and state. When the copy genuinely IS the requirement — a legal disclaimer, a regulated45 notice, a wire-protocol constant — assert the identifier the application renders it from (the46 message key, id, or code), never a duplicated literal sentence.473. **Every new test must be observed failing** for the reason it claims before it counts as passing.484. **Never weaken a test to get green.** Not by relaxing an assertion, widening a tolerance,49 marking `skip`/`only`/`xfail`, raising a retry count to hide a race, wrapping a step in a50 conditional, or deleting it. A failing test means the code is wrong (fix the code) or the51 requirement changed (see the upsert matrix below).525. **Never encode a known bug as expected behavior.** If writing a test reveals a defect, fix the53 defect. A green test that pins a bug in place is worse than no test.546. **You ran it, you own it.** A test that failed, flaked, or was skipped in a run you performed is55 yours to deal with. The only question is whether you fix it or report it — never whether to56 mention it. Fix it in the current work when it sits in what you changed; fix it in its own commit57 when it is small and self-contained, which a flaky or skipped test almost always is. Report it,58 with file:line and how big the fix looks, when fixing would swamp the intended change, when the59 problem spans the whole suite, or when the failure exposes a bug in untested or legacy code whose60 current behavior may be load-bearing — there, establish coverage before changing what you cannot61 verify. "Different module", "predates my change", "unrelated to this task"62 and "out of scope" are never reasons to stay silent, and rarely reasons not to fix. Dismissing it63 requires establishing it is not a defect, and citing the evidence for that.647. **Test our application, never a third party.** Do not navigate to sites you do not control, and65 do not drive an external identity provider's UI. Mock the external service or use a controlled66 test tenant. Their cookie banner is not your test's problem, but it will be your failure.67</HARD-GATE>6869## Workflow70711. **Discover the setup** — the runner and its config, base URL, how the app under test is started,72 how authentication is obtained, where fixtures and page objects live, and the command that runs73 one spec.742. **Justify the level.** Write the journey as a sentence. If any step's risk is coverable lower75 down, cover it there instead and keep this test to the wiring.763. **Decide add / update / leave** using the upsert matrix below. A UI restructure that preserves77 behavior should not change a test that was written against roles and ids; if it does, the test78 was written against the DOM.794. **Set up state through the API, not the UI.** Log in programmatically, seed data80 programmatically, and start the test at the screen the journey begins on.815. **Write the test** with auto-waiting assertions and no fixed delays. See82 `references/e2e-test-rules.md`.836. **Observe it fail.** Break the behavior it covers and confirm the failure names the right step.847. **Run it repeatedly** — at least five consecutive passes, and in the CI configuration if it85 differs. A single green run says nothing at this level.868. **Triage anything else the run surfaced** under House Rule 6, classifying each defect87 against `../test-manager/references/test-taxonomy.md` — flaky, skipped, assertion-free,88 unfailable, tautological, copy-asserting, library-testing, change-detector, duplicated,89 obsolete, mislabeled, excessive setup.9091### When dispatched by `necturalabs:test-manager`9293- Run only your level's command, never the full suite.94- Do not edit shared fixtures, factories, auth setup files, or any helper outside your assigned95 paths — another specialist may be editing it right now. Report the change you need instead.96- Do not run the code review. Report back; the orchestrator runs it once over everything.9798## The Upsert Decision99100Google's rule: strive for unchanging tests. After a test is written, it should not need to change as101the system is refactored, extended, or fixed.102103| Change | Existing tests | New tests |104|--------|----------------|-----------|105| **Pure refactoring** | unchanged. If they break, either the refactoring changed behavior or the tests were written against implementation details — fix that cause, not the assertion. | none |106| **New feature** | unchanged | add, covering the new behaviors only |107| **Bug fix** | unchanged | add a regression test, written first and observed failing |108| **Behavior change** | **the only case where editing an existing test is legitimate** | add for genuinely new behavior |109110That matrix governs what a test asserts about *product* behavior. Repairing a test that is itself111defective — no assertion, tautological, asserting copy, asserting a generated query, unfailable,112flaky — is a different axis, and is always allowed regardless of what the product did. If the test113is currently failing, establish *why* before repairing it: a defective test can still be failing for114a real product reason, and repairing it green hides that. Confirm the product behavior is correct115first, then repair, then observe the repaired test fail against the broken behavior. Name the116defect from the catalogue (`../test-manager/references/test-taxonomy.md`) before you touch the test.117118Deleting a test is legitimate in four cases: the behavior it covers no longer exists, it duplicates119another test at the same level, it is itself defective with no salvageable assertion, or the120behavior is now covered at a cheaper level and that replacement was written and observed failing121first. Say which case applies. "It was failing" and "it was slow" are never reasons to delete a test.122123## Quick Reference124125Full rules with sources: `references/e2e-test-rules.md`126127| Concern | Rule |128|---------|------|129| Selectors | Role and accessible name first; a dedicated `data-testid` when there is no accessible handle. Never CSS class chains, never XPath, never nth-child, never a text sentence. |130| Waiting | Web-first assertions that retry until the condition holds. Never a fixed sleep. Wait on a network alias or a state change, not a duration. |131| Auth | Programmatic login, reused storage state. Never drive the login form in `beforeEach`. |132| Data | The test creates and owns its data, keyed uniquely. Never rely on what a previous test left. |133| Isolation | Fresh browser context per test: no shared cookies, storage, or session. |134| Cleanup | Reset before the test, not after — an aborted run leaves `after` hooks unexecuted. |135| Assertions per test | Batch them. A journey test asserts at each meaningful checkpoint; splitting into one-assertion tests multiplies setup cost with no benefit. |136| Third parties | Mocked or stubbed at the network layer. |137| Debugging | Traces and videos on failure, from CI. Screenshots alone rarely explain a race. |138| Runtime | Shard in CI and push coverage down the pyramid. If the suite cannot finish in the time the team will actually wait, move journeys to cheaper levels — deletion only under the four cases above, and never because a test is slow. |139140## Rationalizations — All Rejected141142| Excuse | Reality |143|--------|---------|144| "An E2E test is quicker to write than restructuring for a unit test" | You pay the write cost once and the run cost forever, with worse diagnosis. |145| "It's flaky, so I'll add a retry" | Retries hide races. Find the condition the test should wait on. |146| "A short sleep is fine here" | It is the number one cause of E2E flake and it will be copied into every new test. |147| "Checking the headline text proves the page rendered" | Assert the URL, the landmark role, or a `data-testid`. Copy changes without defects. |148| "Logging in through the UI is more realistic" | Cover login once, in its own test. Everywhere else it is slow, flaky setup. |149| "The staging data already has a suitable user" | Then the test breaks when someone else edits that user. Create your own. |150| "Cleaning up in afterEach keeps it tidy" | An aborted run skips it. Reset at the start. |151| "This test only fails in CI" | That is a real difference in timing or environment, and it is a defect in the test. |152| "One assertion per test is best practice" | That is a unit-test rule. At this level it multiplies the most expensive part. |153154## Red Flags — Stop155156- `waitForTimeout`, `cy.wait(3000)`, `Thread.sleep`, or any bare duration157- A selector containing a CSS class chain, `nth-child`, XPath, or a full sentence158- A `beforeEach` that logs in through the form159- A test that depends on another test having run first160- A retry count raised to make CI pass161- Conditional branching around a step that "sometimes" appears162- A new E2E test covering logic that a unit test could pin down163164## When Done165166Report the journey covered and why it needs this level, what you added, updated, and deleted with167reasons, the failing observation for each new test, the exact command and its output, the number of168consecutive clean runs, and every defect triaged.169170If you are running as a subagent dispatched by `necturalabs:test-manager`, stop here and report171back — it runs the full suite and the review once, over everything. Otherwise, including when172test-manager invoked this skill inline in your own context, run the full suite yourself and then173`necturalabs:iterative-code-review` over the changes — preceded by174`necturalabs:iterative-security-audit` when the changes fall in that skill's trigger list (see its175description; it is broader than auth and crypto), since it chains into the review. That run satisfies the orchestrator's completion176gate rather than adding a second one. If the review returns test findings, fix them here177under these rules rather than re-entering a test skill, so the review runs once per invocation178chain.
NecturaLabs