Test

Test Skill — Test Plan, Cases, Fixtures, and Playwright E2E

Usage

/test
/test --plan-only
/test --e2e-only
/test --from-cr <file.md>
/test --from-cr --issue N

• /test — generate all test artefacts: plan, cases, fixtures, and E2E tests.
• /test --plan-only — generate test plan, test cases, and test data fixtures (everything in specs/test_artefacts/). Stop before writing Playwright files. Does NOT require source code — only needs stories from /spec.
• /test --e2e-only — skip plan/cases, go straight to Playwright E2E generation from specs/test_artefacts/verification-matrix.json; record specs/test_artefacts/e2e-traces.json. Use when plan already exists and source code has been built.
• /test --from-cr <file.md> / --from-cr --issue N — brownfield CR lane. Turn a change request (a markdown file, or a GitHub issue) against existing code into a regression-pin set (behavior that must stay identical) plus a delta test plan (new behavior the CR introduces), grounded against the CR. See "Brownfield CR Lane" below. Run this before /change when a CR document exists.

Prerequisites

For --plan-only (test planning phase — runs parallel with /design):

• specs/stories/ — user stories with acceptance criteria (one file per story).
• project-manifest.json — for service configuration context.

For full run or --e2e-only (Playwright generation — runs after /auto):

• specs/stories/ — user stories with acceptance criteria.
• backend/ and/or frontend/ — source code that the E2E tests will target.
• project-manifest.json — for base URLs and service port configuration.
• specs/test_artefacts/ — test plan, cases, and fixtures (generated by --plan-only).

If required prerequisites are missing, stop and report what is absent.

Steps

Step 1 — Read Patterns

Read .claude/skills/code-gen/SKILL.md for quality principles (typing, error handling, test structure), plus .claude/skills/code-gen/references/test-strategy.md for the test-layer model and boundary checklist.

Read .claude/skills/test/references/test-design.md for the case-derivation method — equivalence partitioning, boundary-value analysis, state-transition testing, error-path enumeration, and concurrency/idempotency. This is how Step 4's test cases reach positive/negative/boundary completeness instead of happy-path-only. Its adversarial/malformed fixtures come from .claude/skills/code-gen/references/test-data.md.

Read .claude/skills/evaluate/SKILL.md for the contract verification approach used by the evaluator.

Read .claude/skills/evaluate/references/playwright-patterns.md for all selector, assertion, and waiting rules — it is the single source of truth (shared with /evaluate). Read .claude/skills/code-gen/references/test-playwright.md secondarily, for config and file-structure patterns only.

Step 2 — Read Acceptance Criteria

Read every story file in specs/stories/. For each story, extract:

• Story ID and title
• Acceptance criteria (AC) — each criterion becomes one or more test cases.
• Edge cases and error paths documented in the story.

Every test case generated must trace to a specific AC. Record the mapping explicitly.

Step 3 — Spawn the generator (test-authoring)

Spawn the generator agent with the full context: story files, source code structure, and the patterns read in Step 1. Point it at .claude/skills/test/references/test-authoring.md for the test-plan / test-case / Playwright-E2E / fixture authoring guidance, and at .claude/skills/test/references/test-design.md for deriving the positive/negative/boundary cases each AC requires.

The agent operates in the forked context. It must not modify source code.

Step 4 — Generate Test Artefacts (specs/test_artefacts/)

Create specs/test_artefacts/ if it does not exist.

specs/test_artefacts/test-plan.md

• Scope: what is being tested and what is explicitly out of scope.
• Test levels: unit, integration, E2E.
• Environment assumptions: base URLs, test DB, seed state.
• Pass/fail criteria for the sprint.

specs/test_artefacts/test-cases.md

• One section per story.
• Each test case: ID, AC reference, preconditions, steps, expected result.
• Cover success paths, error paths, and boundary conditions. Apply the derivation checklist in test-design.md to each AC: partition every input, add boundary triples (N-1, N, N+1) for every bound, enumerate the error-status checklist, and add adversarial/concurrency cases where the AC mutates shared state or crosses a trust boundary. Happy-path-only is incomplete.

specs/test_artefacts/test-data/

• One fixture file per domain entity (e.g., orders.json, users.json).
• Data must be domain-representative: real-looking emails, valid UUIDs, plausible amounts.
• Never use "foo", 123, or "test" as stand-in values.
• In --plan-only mode, fixtures are schema-free: /design runs in parallel, so specs/design/api-contracts.md may not exist yet. Derive field names from story acceptance criteria, and reconcile fixtures against api-contracts.md in Step 5 (or when /build Phase 4 begins) — field-name drift between fixtures and contracts is expected until then and must be resolved before any Playwright file uses them.

specs/test_artefacts/test-traces.json — the trace spine: one entry per test case, each tracing to the acceptance-criterion id(s) it verifies:

[
  { "id": "TC-1", "text": "register returns 201 on valid input", "traces": ["E1-S1-AC1"] },
  { "id": "TC-2", "text": "register rejects duplicate email with 409", "traces": ["E1-S1-AC2"] }
]

Every test case must trace to at least one {story}-AC{n} id from specs/stories/story-traces.json. A test case tracing to no AC tests behavior nobody asked for; an AC with no test case is an untested requirement. A negative test may also trace a constraint-obligation id (OBL-…, see Step 4.4):

{ "id": "TC-3", "text": "username under 3 chars rejected with 422", "traces": ["E1-S1-AC2", "OBL-User.username-minLength"] }

specs/test_artefacts/verification-matrix.json — one requirement row per AC, with stable matrix ids, story/AC references, required layers (unit, api, e2e), group, implementation_paths (the production files expected to satisfy the AC, empty during pure plan-only work if unknown), and planned checks. This is the shared oracle for /auto, generator teammates, and evaluator runtime checks. The executed gate treats evidence older than any declared implementation_paths file as stale and blocks.

Trace sidecars — /test must generate these files alongside the plan. Empty arrays are acceptable when no concrete test artifacts exist yet; as implementation and E2E tests are authored, record the executed matrix coverage in:

• specs/test_artefacts/unit-traces.json
• specs/test_artefacts/integration-traces.json
• specs/test_artefacts/e2e-traces.json

Each sidecar entry must name the test artifact and its matrix_id.

Step 4.4 — Constraint Obligation Extraction [when design schemas exist]

The design schemas encode validation rules (required, minLength, pattern, enum, minimum, …) the test author may forget. Mine them into machine-checkable obligations — one negative-test obligation per constraint:

# Only when /design has produced the schemas (skip silently in --plan-only, where
# they do not exist yet — missing schema paths are skipped, yielding zero obligations).
node .claude/scripts/constraints-extract.js \
  --schemas specs/design/data-models.schema.json \
  --schemas specs/design/api-contracts.schema.json \
  --out specs/test_artefacts/constraint-obligations.json \
  --index-out specs/test_artefacts/obligation-index.json

constraint-obligations.json lists each obligation with a stable OBL-<field>-<rule> id and suggested_cases. Every obligation must be covered by a negative test case (trace the OBL- id from test-traces.json). Generate one representative case per obligation using the boundary points in test-design.md §2 — not one per value. obligation-index.json is the upstream index the grounding gate folds in at Step 4.5.

Step 4.5 — Grounding Gate [HARD BLOCK — when specs/stories/story-traces.json exists]

Build the AC index (the acceptance criteria are the upstream this layer must cover) and run the deterministic check. When Step 4.4 produced obligation-index.json, pass it as a second --required index so an un-covered constraint is a dropped upstream id — the same hard block as an un-covered AC:

# Flatten every story's acceptance-criterion ids into the upstream index
node -e "const fs=require('fs');const s=JSON.parse(fs.readFileSync('specs/stories/story-traces.json'));fs.writeFileSync('specs/test_artefacts/ac-index.json',JSON.stringify(s.flatMap(x=>(x.acs||[]).map(id=>({id})))))"

# Include --required specs/test_artefacts/obligation-index.json only when Step 4.4 ran.
node .claude/scripts/trace-check.js \
  --required specs/test_artefacts/ac-index.json \
  --required specs/test_artefacts/obligation-index.json \
  --downstream specs/test_artefacts/test-traces.json \
  --layer test \
  --out specs/reviews/test-grounding.json

node .claude/scripts/verification-matrix-gate.js --phase plan

specs/reviews/test-grounding.json is a hard gate: any net_new (test case tracing to no AC or obligation) or dropped (an AC or constraint obligation with no test case) blocks. Resolve before reporting the plan. (Skip when story-traces.json does not exist; omit the obligation index when Step 4.4 did not run.)

specs/reviews/verification-matrix-verdict.json is also a HARD BLOCK: every implementation-ready AC must have a matrix obligation before the plan can be reported.

Step 4.6 — Acceptance Tests First (AT Layer) [REQUIRED SUB-SKILL: writing-acceptance-tests-first]

Before implementation begins on each story, write its acceptance test(s) FIRST — a fast, business-readable layer that verifies the AC directly against business logic via a Ports-and-Adapters seam, bypassing the UI/transport stack. This is additive to Step 5's Playwright E2E generation, not a replacement: ATs become the primary per-story verification loop; E2E remains the final full-stack confirmation layer, generated later once source exists. This step runs even in --plan-only mode — writing an AT does not require the full running app, only the seam it targets, and an AT that fails because the target function does not exist yet is a legitimate first red state (see the sub-skill's Process step 5).

Follow .claude/skills/writing-acceptance-tests-first/SKILL.md in full for each story: locate or request the human-provided AT template (specs/test_artefacts/at-template.*), identify the Ports-and-Adapters seam (reuse /seam-finder output when specs/brownfield/seams-<goal-slug>.md exists for the story's goal; for a brand-new greenfield module, design the port directly from the story's acceptance criteria), write the AT against that seam with a test-double/fake adapter, run it, and confirm it fails for the right reason before implementation starts.

Write acceptance tests to specs/test_artefacts/acceptance/{story-id}.<ext> (extension matches the project's test stack) and record each in specs/test_artefacts/at-traces.json, tracing to the story's AC id(s) the same way test-traces.json does:

[
  { "id": "AT-1", "story": "E1-S1", "text": "registering with a valid email creates an account", "traces": ["E1-S1-AC1"] }
]

Every implementation-ready AC should have a corresponding AT before the story is handed to implementation. at-traces.json is the deterministic signal downstream review should check for. As of gap G23, this is also mechanically enforced at commit time: at-first-gate.js (pre-commit) blocks a story's NEW production source files from being committed unless both the AT file and a record-at-red.js receipt exist for that story (see HARNESS.md G20/G23).

If --plan-only: STOP HERE. Steps 4–4.6 (plan + cases + fixtures + trace spine + grounding gate + acceptance tests) are the complete deliverable for the planning phase. Do not proceed to Playwright generation — source code does not exist yet. Report the generated artifacts and exit.

Step 4.7 — Generate Integration Tests (tests/integration/) [when source exists]

For each story that crosses an external boundary (database, HTTP service, or LLM), generate a deterministic integration test under tests/integration/ that binds the shipped boundary-test-doubles kit (gap G34) instead of reaching real externals. On first use, copy the kit into the project's tests/ tree from .claude/templates/boundary-doubles/ (replay_transport.py, fake_llm.py, db_fixture.py, conftest.py) and record one HTTP/LLM golden per operation via the kit's record step (run once with HARNESS_TEST_REPLAY unset against the real service/model).

Rules:

• The test runs under HARNESS_TEST_REPLAY=1 so the app's DB/HTTP/LLM resolve to the doubles — HTTP via ReplayTransport at the wrapper's _call seam, LLM via FakeLLMClient golden responses, DB via the db_session transactional-rollback fixture.
• No integration test may reach a real external — the live-externals gate (G36) blocks a non-localhost URL, a real DB DSN, or a directly-constructed SDK client at commit time.
• Record each generated integration test in specs/test_artefacts/integration-traces.json with its matrix_id.

This is additive to Step 5's Playwright E2E (which remains the final full-stack layer). If source does not exist yet (--plan-only), skip this step.

Step 5 — Generate Playwright E2E Tests (e2e/)

Create e2e/ at the project root if it does not exist.

For each story, generate a Playwright test file named {story-id}.spec.ts.

In --e2e-only mode, generate Playwright specs from specs/test_artefacts/verification-matrix.json, not ad hoc story prose. Each E2E test must record its executed row in specs/test_artefacts/e2e-traces.json with the corresponding matrix_id.

Selector and assertion rules are single-sourced in .claude/skills/evaluate/references/playwright-patterns.md (the canonical Playwright reference, shared with /evaluate). The rules below summarize it — defer to that file if they differ.

Rules for Playwright tests:

• Use getByRole, getByLabel, getByText — never CSS selectors or XPath.
• No waitForTimeout. Use expect(locator).toBeVisible() with retry.
• Each test() block maps to exactly one acceptance criterion. The test name must reference the AC.
• Import fixtures from specs/test_artefacts/test-data/.
• Follow Arrange → Act → Assert structure.

Step 6 — Copy Playwright Config

Copy the config template:

cp .claude/templates/playwright.config.template.ts playwright.config.ts

Fill in the baseURL values from project-manifest.json. Configure webServer entries for each service that needs to be running during tests.

Step 6.5 — Ship the CI workflow

Copy .claude/templates/github-workflows/e2e.yml to .github/workflows/e2e.yml (skip if the target file already exists — never overwrite a team's edited workflow). This makes every future PR re-run the generated suite in CI instead of only inside harness sessions; the config's webServer block self-starts the compose stack on the runner. Requires Docker on the runner (true for ubuntu-latest). After copying, adapt the dependency-install step to the project's package layout: the template assumes a root package.json and skips npm install when none exists — a Python-backend monorepo may need its frontend workspace path installed instead.

Step 7 — Install Playwright

npx playwright install --with-deps chromium

Step 8 — Verify

npx playwright test

All tests must pass on the first run against the target environment. A failing test that was written incorrectly is not acceptable — fix the test before reporting results.

Brownfield CR Lane (--from-cr)

The greenfield lane grounds tests against story acceptance criteria. The brownfield lane grounds them against a change request over existing code, and splits the work in two: pin the behavior that must not change, and prove the behavior that must. Composes existing skills — it does not re-implement them.

Prerequisites: existing source; specs/brownfield/code-graph.json (run /code-map or /brownfield if absent). The CR is a markdown file, or a GitHub issue fetched first: gh issue view N --json title,body -q '.title + "\n\n" + .body' > specs/changes/cr-N.md.

Let <id> be the issue number or a slug of the CR title. Work under specs/test_artefacts/cr-<id>/.

CR1 — Build the CR acceptance index [HARD BLOCK if empty]

node .claude/scripts/cr-index.js --cr specs/changes/cr-<id>.md --out specs/test_artefacts/cr-<id>/cr-acceptance.json

This is the upstream the delta tests must trace to (the brownfield analogue of ac-index.json). If it is empty, STOP — the CR has no extractable acceptance lines; route to /clarify to get testable criteria before writing any tests.

CR2 — Locate the blast radius

Run /seam-finder "<cr goal>" to rank the safest cut-points, and checking-coverage-before-change on the symbols the CR touches to classify each COVERED or UNCOVERED (reads code-graph.json + coverage). This decides how each symbol gets pinned in CR3.

CR3 — Regression-pin set (behavior that must stay identical)

Write to specs/test_artefacts/cr-<id>/regression-pins.md:

• UNCOVERED symbols at a usable seam: REQUIRED SUB-SKILL: pinning-down-behavior — write characterization tests at the seam, then confirm they bite with node .claude/scripts/mutation-smoke.js --files <seam-file> --test-cmd "<pin test cmd>". A pin you never watched fail proves nothing.
• COVERED symbols: list the existing oracle tests that must stay byte-identical green across the change.
• UNCOVERED with no usable seam (seam score < 0.5 / god file): REQUIRED SUB-SKILL: sprouting-instead-of-editing — the CR's new behavior lands in a new tested unit instead of editing legacy in place.

CR4 — Delta test plan (new behavior the CR introduces)

Apply the full derivation method in references/test-design.md (equivalence partitioning, boundary triples, state transitions, error-path enumeration) to each CR acceptance line. If the CR changes a schema, run constraints-extract.js (Step 4.4) and cover every new obligation. Write delta-test-cases.md and the trace spine delta-traces.json — each delta case tracing to a CR-AC{n} id (and any OBL- id it covers).

CR5 — Delta grounding gate [HARD BLOCK]

node .claude/scripts/trace-check.js \
  --required specs/test_artefacts/cr-<id>/cr-acceptance.json \
  --downstream specs/test_artefacts/cr-<id>/delta-traces.json \
  --layer cr \
  --out specs/reviews/cr-grounding.json

Any net_new (a delta test tracing to no CR line — scope creep) or dropped (a CR line with no delta test — an unverified requirement) blocks. Resolve before handing off.

/change then implements the CR test-first against this set: the regression pins are the oracle that must stay green, the delta tests are the new behavior to make pass.

Output

Gotchas

• Test cases not mapped to acceptance criteria. Every test case must cite its AC. Unmapped tests are noise.
• CSS selectors instead of getByRole. CSS selectors break on refactors. Use ARIA-based selectors exclusively.
• Flaky waits. waitForTimeout is banned. Network or animation delays must be handled with expect(...).toBeVisible() or waitForResponse.
• Missing test data fixtures. Tests that generate random data inline produce unrepeatable results. Always use fixture files.
• Testing implementation details. Test behavior through the public interface, not internal state.
• Skipping error paths. Every documented error path in the AC must have a test.
• Leaving schema constraints untested. When design schemas exist, every OBL- obligation from Step 4.4 must be covered by a negative test — the grounding gate blocks otherwise. Generate one representative case per obligation, not one per value.
• Routing acceptance verification through the full stack when a seam exists. This is Vaccari's named E2E-as-AT-implementation antipattern: slow, flaky, and a failure means log-spelunking. Step 4.6's ATs should call business logic directly through a Ports-and-Adapters seam with a fake adapter; Step 5's Playwright suite is the final full-stack confirmation layer, not the primary acceptance loop.