# Test > [Internal pipeline stage — run by /build and /auto; invoke directly only as a power user.] Generate test plan, test cases, test data fixtures, and Playwright E2E tests mapped to acceptance criteria. - Skill: `cwijayasundara/test` (Agent Skill, multi-file: 4 files) - Install (CLI): `npx skillmds@latest add cwijayasundara/test` - Raw SKILL.md: https://api.skillmd.com/api/skills/cwijayasundara/test/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: DevOps & Infra, Coding & Dev Tools - Author: cwijayasundara (https://skillmd.com/u/cwijayasundara) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/cwijayasundara/test --- # Test Skill — Test Plan, Cases, Fixtures, and Playwright E2E ## Usage ``` /test /test --plan-only /test --e2e-only /test --from-cr /test --from-cr --issue N ``` - `/test` — generate all test artefacts: plan, cases, fixtures, and E2E tests. - `/test --plan-only` — generate test plan, test cases, and test data fixtures (everything in `specs/test_artefacts/`). Stop before writing Playwright files. Does NOT require source code — only needs stories from `/spec`. - `/test --e2e-only` — skip plan/cases, go straight to Playwright E2E generation from `specs/test_artefacts/verification-matrix.json`; record `specs/test_artefacts/e2e-traces.json`. Use when plan already exists and source code has been built. - `/test --from-cr ` / `--from-cr --issue N` — **brownfield CR lane.** Turn a change request (a markdown file, or a GitHub issue) against existing code into a **regression-pin set** (behavior that must stay identical) plus a **delta test plan** (new behavior the CR introduces), grounded against the CR. See "Brownfield CR Lane" below. Run this *before* `/change` when a CR document exists. --- ## Prerequisites **For `--plan-only` (test planning phase — runs parallel with `/design`):** - `specs/stories/` — user stories with acceptance criteria (one file per story). - `project-manifest.json` — for service configuration context. **For full run or `--e2e-only` (Playwright generation — runs after `/auto`):** - `specs/stories/` — user stories with acceptance criteria. - `backend/` and/or `frontend/` — source code that the E2E tests will target. - `project-manifest.json` — for base URLs and service port configuration. - `specs/test_artefacts/` — test plan, cases, and fixtures (generated by `--plan-only`). If required prerequisites are missing, stop and report what is absent. --- ## Steps ### Step 1 — Read Patterns Read `.claude/skills/code-gen/SKILL.md` for quality principles (typing, error handling, test structure), plus `.claude/skills/code-gen/references/test-strategy.md` for the test-layer model and boundary checklist. Read `.claude/skills/test/references/test-design.md` for the case-derivation method — equivalence partitioning, boundary-value analysis, state-transition testing, error-path enumeration, and concurrency/idempotency. This is how Step 4's test cases reach positive/negative/boundary completeness instead of happy-path-only. Its adversarial/malformed fixtures come from `.claude/skills/code-gen/references/test-data.md`. Read `.claude/skills/evaluate/SKILL.md` for the contract verification approach used by the evaluator. Read `.claude/skills/evaluate/references/playwright-patterns.md` for all selector, assertion, and waiting rules — it is the single source of truth (shared with `/evaluate`). Read `.claude/skills/code-gen/references/test-playwright.md` secondarily, for config and file-structure patterns only. ### Step 2 — Read Acceptance Criteria Read every story file in `specs/stories/`. For each story, extract: - Story ID and title - Acceptance criteria (AC) — each criterion becomes one or more test cases. - Edge cases and error paths documented in the story. Every test case generated must trace to a specific AC. Record the mapping explicitly. ### Step 3 — Spawn the generator (test-authoring) Spawn the `generator` agent with the full context: story files, source code structure, and the patterns read in Step 1. Point it at `.claude/skills/test/references/test-authoring.md` for the test-plan / test-case / Playwright-E2E / fixture authoring guidance, and at `.claude/skills/test/references/test-design.md` for deriving the positive/negative/boundary cases each AC requires. The agent operates in the forked context. It must not modify source code. ### Step 4 — Generate Test Artefacts (`specs/test_artefacts/`) Create `specs/test_artefacts/` if it does not exist. **`specs/test_artefacts/test-plan.md`** - Scope: what is being tested and what is explicitly out of scope. - Test levels: unit, integration, E2E. - Environment assumptions: base URLs, test DB, seed state. - Pass/fail criteria for the sprint. **`specs/test_artefacts/test-cases.md`** - One section per story. - Each test case: ID, AC reference, preconditions, steps, expected result. - Cover success paths, error paths, and boundary conditions. Apply the derivation checklist in `test-design.md` to each AC: partition every input, add boundary triples (`N-1`, `N`, `N+1`) for every bound, enumerate the error-status checklist, and add adversarial/concurrency cases where the AC mutates shared state or crosses a trust boundary. Happy-path-only is incomplete. **`specs/test_artefacts/test-data/`** - One fixture file per domain entity (e.g., `orders.json`, `users.json`). - Data must be domain-representative: real-looking emails, valid UUIDs, plausible amounts. - Never use `"foo"`, `123`, or `"test"` as stand-in values. - **In `--plan-only` mode, fixtures are schema-free:** `/design` runs in parallel, so `specs/design/api-contracts.md` may not exist yet. Derive field names from story acceptance criteria, and reconcile fixtures against `api-contracts.md` in Step 5 (or when `/build` Phase 4 begins) — field-name drift between fixtures and contracts is expected until then and must be resolved before any Playwright file uses them. **`specs/test_artefacts/test-traces.json`** — the trace spine: one entry per test case, each tracing to the acceptance-criterion id(s) it verifies: ```json [ { "id": "TC-1", "text": "register returns 201 on valid input", "traces": ["E1-S1-AC1"] }, { "id": "TC-2", "text": "register rejects duplicate email with 409", "traces": ["E1-S1-AC2"] } ] ``` Every test case must trace to at least one `{story}-AC{n}` id from `specs/stories/story-traces.json`. A test case tracing to no AC tests behavior nobody asked for; an AC with no test case is an untested requirement. A negative test may *also* trace a constraint-obligation id (`OBL-…`, see Step 4.4): ```json { "id": "TC-3", "text": "username under 3 chars rejected with 422", "traces": ["E1-S1-AC2", "OBL-User.username-minLength"] } ``` **`specs/test_artefacts/verification-matrix.json`** — one requirement row per AC, with stable matrix ids, story/AC references, required layers (`unit`, `api`, `e2e`), group, `implementation_paths` (the production files expected to satisfy the AC, empty during pure plan-only work if unknown), and planned checks. This is the shared oracle for `/auto`, generator teammates, and evaluator runtime checks. The executed gate treats evidence older than any declared `implementation_paths` file as stale and blocks. **Trace sidecars** — `/test` must generate these files alongside the plan. Empty arrays are acceptable when no concrete test artifacts exist yet; as implementation and E2E tests are authored, record the executed matrix coverage in: - `specs/test_artefacts/unit-traces.json` - `specs/test_artefacts/integration-traces.json` - `specs/test_artefacts/e2e-traces.json` Each sidecar entry must name the test artifact and its `matrix_id`. ### Step 4.4 — Constraint Obligation Extraction [when design schemas exist] The design schemas encode validation rules (`required`, `minLength`, `pattern`, `enum`, `minimum`, …) the test author may forget. Mine them into machine-checkable obligations — one negative-test obligation per constraint: ```bash # Only when /design has produced the schemas (skip silently in --plan-only, where # they do not exist yet — missing schema paths are skipped, yielding zero obligations). node .claude/scripts/constraints-extract.js \ --schemas specs/design/data-models.schema.json \ --schemas specs/design/api-contracts.schema.json \ --out specs/test_artefacts/constraint-obligations.json \ --index-out specs/test_artefacts/obligation-index.json ``` `constraint-obligations.json` lists each obligation with a stable `OBL--` id and `suggested_cases`. Every obligation must be covered by a negative test case (trace the `OBL-` id from `test-traces.json`). Generate **one representative case per obligation** using the boundary points in `test-design.md` §2 — not one per value. `obligation-index.json` is the upstream index the grounding gate folds in at Step 4.5. ### Step 4.5 — Grounding Gate [HARD BLOCK — when `specs/stories/story-traces.json` exists] Build the AC index (the acceptance criteria are the upstream this layer must cover) and run the deterministic check. When Step 4.4 produced `obligation-index.json`, pass it as a second `--required` index so an un-covered constraint is a `dropped` upstream id — the same hard block as an un-covered AC: ```bash # Flatten every story's acceptance-criterion ids into the upstream index node -e "const fs=require('fs');const s=JSON.parse(fs.readFileSync('specs/stories/story-traces.json'));fs.writeFileSync('specs/test_artefacts/ac-index.json',JSON.stringify(s.flatMap(x=>(x.acs||[]).map(id=>({id})))))" # Include --required specs/test_artefacts/obligation-index.json only when Step 4.4 ran. node .claude/scripts/trace-check.js \ --required specs/test_artefacts/ac-index.json \ --required specs/test_artefacts/obligation-index.json \ --downstream specs/test_artefacts/test-traces.json \ --layer test \ --out specs/reviews/test-grounding.json node .claude/scripts/verification-matrix-gate.js --phase plan ``` `specs/reviews/test-grounding.json` is a **hard gate**: any `net_new` (test case tracing to no AC or obligation) or `dropped` (an AC or constraint obligation with no test case) blocks. Resolve before reporting the plan. (Skip when `story-traces.json` does not exist; omit the obligation index when Step 4.4 did not run.) `specs/reviews/verification-matrix-verdict.json` is also a **HARD BLOCK**: every implementation-ready AC must have a matrix obligation before the plan can be reported. ### Step 4.6 — Acceptance Tests First (AT Layer) [REQUIRED SUB-SKILL: `writing-acceptance-tests-first`] Before implementation begins on each story, write its acceptance test(s) FIRST — a fast, business-readable layer that verifies the AC directly against business logic via a Ports-and-Adapters seam, bypassing the UI/transport stack. This is additive to Step 5's Playwright E2E generation, not a replacement: ATs become the primary per-story verification loop; E2E remains the final full-stack confirmation layer, generated later once source exists. This step runs even in `--plan-only` mode — writing an AT does not require the full running app, only the seam it targets, and an AT that fails because the target function does not exist yet is a legitimate first red state (see the sub-skill's Process step 5). Follow `.claude/skills/writing-acceptance-tests-first/SKILL.md` in full for each story: locate or request the human-provided AT template (`specs/test_artefacts/at-template.*`), identify the Ports-and-Adapters seam (reuse `/seam-finder` output when `specs/brownfield/seams-.md` exists for the story's goal; for a brand-new greenfield module, design the port directly from the story's acceptance criteria), write the AT against that seam with a test-double/fake adapter, run it, and confirm it fails for the right reason before implementation starts. Write acceptance tests to `specs/test_artefacts/acceptance/{story-id}.` (extension matches the project's test stack) and record each in `specs/test_artefacts/at-traces.json`, tracing to the story's AC id(s) the same way `test-traces.json` does: ```json [ { "id": "AT-1", "story": "E1-S1", "text": "registering with a valid email creates an account", "traces": ["E1-S1-AC1"] } ] ``` Every implementation-ready AC should have a corresponding AT before the story is handed to implementation. `at-traces.json` is the deterministic signal downstream review should check for. As of gap G23, this is also mechanically enforced at commit time: `at-first-gate.js` (pre-commit) blocks a story's NEW production source files from being committed unless both the AT file and a `record-at-red.js` receipt exist for that story (see `HARNESS.md` G20/G23). **If `--plan-only`: STOP HERE.** Steps 4–4.6 (plan + cases + fixtures + trace spine + grounding gate + acceptance tests) are the complete deliverable for the planning phase. Do not proceed to Playwright generation — source code does not exist yet. Report the generated artifacts and exit. ### Step 4.7 — Generate Integration Tests (`tests/integration/`) [when source exists] For each story that crosses an external boundary (database, HTTP service, or LLM), generate a deterministic integration test under `tests/integration/` that binds the shipped **boundary-test-doubles kit** (gap G34) instead of reaching real externals. On first use, copy the kit into the project's `tests/` tree from `.claude/templates/boundary-doubles/` (`replay_transport.py`, `fake_llm.py`, `db_fixture.py`, `conftest.py`) and record one HTTP/LLM golden per operation via the kit's record step (run once with `HARNESS_TEST_REPLAY` unset against the real service/model). Rules: - The test runs under `HARNESS_TEST_REPLAY=1` so the app's DB/HTTP/LLM resolve to the doubles — HTTP via `ReplayTransport` at the wrapper's `_call` seam, LLM via `FakeLLMClient` golden responses, DB via the `db_session` transactional-rollback fixture. - No integration test may reach a real external — the `live-externals` gate (G36) blocks a non-localhost URL, a real DB DSN, or a directly-constructed SDK client at commit time. - Record each generated integration test in `specs/test_artefacts/integration-traces.json` with its `matrix_id`. This is additive to Step 5's Playwright E2E (which remains the final full-stack layer). If source does not exist yet (`--plan-only`), skip this step. ### Step 5 — Generate Playwright E2E Tests (`e2e/`) Create `e2e/` at the project root if it does not exist. For each story, generate a Playwright test file named `{story-id}.spec.ts`. In `--e2e-only` mode, generate Playwright specs from `specs/test_artefacts/verification-matrix.json`, not ad hoc story prose. Each E2E test must record its executed row in `specs/test_artefacts/e2e-traces.json` with the corresponding `matrix_id`. Selector and assertion rules are single-sourced in `.claude/skills/evaluate/references/playwright-patterns.md` (the canonical Playwright reference, shared with `/evaluate`). The rules below summarize it — defer to that file if they differ. Rules for Playwright tests: - Use `getByRole`, `getByLabel`, `getByText` — never CSS selectors or XPath. - No `waitForTimeout`. Use `expect(locator).toBeVisible()` with retry. - Each `test()` block maps to exactly one acceptance criterion. The test name must reference the AC. - Import fixtures from `specs/test_artefacts/test-data/`. - Follow Arrange → Act → Assert structure. ### Step 6 — Copy Playwright Config Copy the config template: ``` cp .claude/templates/playwright.config.template.ts playwright.config.ts ``` Fill in the `baseURL` values from `project-manifest.json`. Configure `webServer` entries for each service that needs to be running during tests. ### Step 6.5 — Ship the CI workflow Copy `.claude/templates/github-workflows/e2e.yml` to `.github/workflows/e2e.yml` (skip if the target file already exists — never overwrite a team's edited workflow). This makes every future PR re-run the generated suite in CI instead of only inside harness sessions; the config's `webServer` block self-starts the compose stack on the runner. Requires Docker on the runner (true for `ubuntu-latest`). After copying, adapt the dependency-install step to the project's package layout: the template assumes a root `package.json` and skips `npm install` when none exists — a Python-backend monorepo may need its frontend workspace path installed instead. ### Step 7 — Install Playwright ``` npx playwright install --with-deps chromium ``` ### Step 8 — Verify ``` npx playwright test ``` All tests must pass on the first run against the target environment. A failing test that was written incorrectly is not acceptable — fix the test before reporting results. --- ## Brownfield CR Lane (`--from-cr`) The greenfield lane grounds tests against story acceptance criteria. The brownfield lane grounds them against a **change request** over existing code, and splits the work in two: pin the behavior that must *not* change, and prove the behavior that *must*. Composes existing skills — it does not re-implement them. **Prerequisites:** existing source; `specs/brownfield/code-graph.json` (run `/code-map` or `/brownfield` if absent). The CR is a markdown file, or a GitHub issue fetched first: `gh issue view N --json title,body -q '.title + "\n\n" + .body' > specs/changes/cr-N.md`. Let `` be the issue number or a slug of the CR title. Work under `specs/test_artefacts/cr-/`. ### CR1 — Build the CR acceptance index [HARD BLOCK if empty] ```bash node .claude/scripts/cr-index.js --cr specs/changes/cr-.md --out specs/test_artefacts/cr-/cr-acceptance.json ``` This is the upstream the delta tests must trace to (the brownfield analogue of `ac-index.json`). **If it is empty, STOP** — the CR has no extractable acceptance lines; route to `/clarify` to get testable criteria before writing any tests. ### CR2 — Locate the blast radius Run `/seam-finder ""` to rank the safest cut-points, and `checking-coverage-before-change` on the symbols the CR touches to classify each **COVERED** or **UNCOVERED** (reads `code-graph.json` + coverage). This decides how each symbol gets pinned in CR3. ### CR3 — Regression-pin set (behavior that must stay identical) Write to `specs/test_artefacts/cr-/regression-pins.md`: - **UNCOVERED symbols at a usable seam:** `REQUIRED SUB-SKILL: pinning-down-behavior` — write characterization tests at the seam, then confirm they bite with `node .claude/scripts/mutation-smoke.js --files --test-cmd ""`. A pin you never watched fail proves nothing. - **COVERED symbols:** list the existing oracle tests that must stay byte-identical green across the change. - **UNCOVERED with no usable seam** (seam score < 0.5 / god file): `REQUIRED SUB-SKILL: sprouting-instead-of-editing` — the CR's new behavior lands in a new tested unit instead of editing legacy in place. ### CR4 — Delta test plan (new behavior the CR introduces) Apply the full derivation method in `references/test-design.md` (equivalence partitioning, boundary triples, state transitions, error-path enumeration) to each CR acceptance line. If the CR changes a schema, run `constraints-extract.js` (Step 4.4) and cover every new obligation. Write `delta-test-cases.md` and the trace spine `delta-traces.json` — each delta case tracing to a `CR-AC{n}` id (and any `OBL-` id it covers). ### CR5 — Delta grounding gate [HARD BLOCK] ```bash node .claude/scripts/trace-check.js \ --required specs/test_artefacts/cr-/cr-acceptance.json \ --downstream specs/test_artefacts/cr-/delta-traces.json \ --layer cr \ --out specs/reviews/cr-grounding.json ``` Any `net_new` (a delta test tracing to no CR line — scope creep) or `dropped` (a CR line with no delta test — an unverified requirement) blocks. Resolve before handing off. `/change` then implements the CR test-first against this set: the regression pins are the oracle that must stay green, the delta tests are the new behavior to make pass. --- ## Output | Path | Purpose | |------|---------| | `specs/test_artefacts/cr-/cr-acceptance.json` | (`--from-cr`) CR acceptance index — upstream for delta grounding | | `specs/test_artefacts/cr-/regression-pins.md` | (`--from-cr`) behavior that must stay identical (pins / existing oracles) | | `specs/test_artefacts/cr-/delta-test-cases.md` | (`--from-cr`) new positive/negative/boundary cases for the CR | | `specs/test_artefacts/cr-/delta-traces.json` | (`--from-cr`) delta case → `CR-AC` id(s) trace spine | | `specs/reviews/cr-grounding.json` | (`--from-cr`) deterministic CR-coverage verdict | | `specs/test_artefacts/test-plan.md` | Sprint test plan | | `specs/test_artefacts/test-cases.md` | Full test case inventory mapped to ACs | | `specs/test_artefacts/test-data/` | JSON fixture files per domain entity | | `specs/test_artefacts/test-traces.json` | Trace spine: each test case → AC id(s) and/or `OBL-` id(s) | | `specs/test_artefacts/acceptance/{story-id}.` | Business-readable acceptance test (Ports-and-Adapters seam, test-double adapter), written before implementation | | `specs/test_artefacts/at-traces.json` | AT trace spine: each acceptance test → AC id(s) | | `specs/test_artefacts/verification-matrix.json` | AC → required verification layers and planned checks | | `specs/test_artefacts/unit-traces.json` | Unit test artifacts mapped to `matrix_id` | | `specs/test_artefacts/integration-traces.json` | Integration/API test artifacts mapped to `matrix_id` | | `specs/test_artefacts/e2e-traces.json` | Playwright artifacts mapped to `matrix_id` | | `specs/test_artefacts/constraint-obligations.json` | (when design schemas exist) one negative-test obligation per schema constraint | | `specs/test_artefacts/obligation-index.json` | (when design schemas exist) obligation upstream index for the grounding gate | | `specs/reviews/test-grounding.json` | (when story-traces exists) deterministic AC + obligation coverage verdict | | `specs/reviews/verification-matrix-verdict.json` | Deterministic matrix gate verdict | | `e2e/{story-id}.spec.ts` | Playwright tests per story | | `playwright.config.ts` | Playwright configuration | --- ## Gotchas - **Test cases not mapped to acceptance criteria.** Every test case must cite its AC. Unmapped tests are noise. - **CSS selectors instead of `getByRole`.** CSS selectors break on refactors. Use ARIA-based selectors exclusively. - **Flaky waits.** `waitForTimeout` is banned. Network or animation delays must be handled with `expect(...).toBeVisible()` or `waitForResponse`. - **Missing test data fixtures.** Tests that generate random data inline produce unrepeatable results. Always use fixture files. - **Testing implementation details.** Test behavior through the public interface, not internal state. - **Skipping error paths.** Every documented error path in the AC must have a test. - **Leaving schema constraints untested.** When design schemas exist, every `OBL-` obligation from Step 4.4 must be covered by a negative test — the grounding gate blocks otherwise. Generate one representative case per obligation, not one per value. - **Routing acceptance verification through the full stack when a seam exists.** This is Vaccari's named E2E-as-AT-implementation antipattern: slow, flaky, and a failure means log-spelunking. Step 4.6's ATs should call business logic directly through a Ports-and-Adapters seam with a fake adapter; Step 5's Playwright suite is the final full-stack confirmation layer, not the primary acceptance loop.