# Full Stack Test Gate

> Test a whole application and prove it works, gated by checks that actually run — the project's own unit/integration/regression suites, then a real browser that boots the app, clicks through every feature flow, screenshots each step, and fails on console errors, uncaught exceptions or failed requests; plus pixel-level visual regression and a crawler that reports which features no test touches. Use when the user wants to test an app end to end, write or run unit / integration / regression tests, set up E2E or Playwright testing, drive the UI and take screenshots, verify a change didn't break anything, add a test gate to CI, or find out what is untested. Triggers: "test the app", "end-to-end", "e2e", "regression test", "unit tests", "Playwright", "screenshot the UI", "click through the app", "smoke test", "QA", "visual regression", "test all features", "does it still work".

- Skill: `neuralmedic-de/full-stack-test-gate` (Agent Skill, multi-file: 27 files)
- Install (CLI): `npx skillmds@latest add neuralmedic-de/full-stack-test-gate`
- Raw SKILL.md: https://api.skillmd.com/api/skills/neuralmedic-de/full-stack-test-gate/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Integrations & APIs
- License: MIT
- Author: NeuralMedic-DE (https://skillmd.com/u/neuralmedic-de)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/neuralmedic-de/full-stack-test-gate

---


# Full-stack test gate (unit → e2e, verified)

Prove an application works by **running it**: the project's own test suites, then
a real browser that boots the app, drives every feature, screenshots each step,
diffs the pixels, and names what nothing tested.

## Core principle

**"It works" is a claim; a green gate is evidence — and evidence has a scope.**
The loop is: run the suites → drive the flows → diff the screenshots → crawl for
what no flow touches → fix → re-run.

**Be honest about scope (the rule that keeps this skill correct):** a green run
means *the flows you wrote passed*. It does not mean the app works. Never report
"fully tested" or "all features work". Report **"N flows green, M% of controls
untouched by any flow"** — and the crawler exists precisely to produce that
second number. → `references/01-test-strategy-and-scope.md`

## When to use vs. not

- Use for: testing a web app end to end; writing/running unit, integration or
  regression tests; setting up Playwright; driving the UI and screenshotting it;
  visual regression; verifying a change didn't break anything; adding a test gate
  to CI; finding out what is untested.
- Not for: load/performance testing, security testing, or native mobile apps
  (different tooling). For accessibility, use the sibling `a11y-gate` skill.

## Inputs to gather first

1. **How to start the app** — the command and the URL. Prefer the **production
   build** (`npm run build && npm run preview`); dev servers add overlays and
   dev-only warnings that produce false failures.
2. **The existing test suites** — how unit/integration tests run today. Don't
   guess: `node run-suites.mjs --detect` reads the repo and proposes them.
3. **The critical journeys** — the handful of things that must never break
   (sign in, search, checkout). Ask; don't infer from the routes.
4. **A test account** — a seeded user, never a real one. Credentials come from
   the environment (`${TEST_PASSWORD}`), never the config file.

## Workflow

Load each reference when you reach its step.

1. **Set the strategy and the honest scope.** Decide what belongs in unit vs. e2e
   (if it can be a unit test, it must be). → `references/01-test-strategy-and-scope.md`

2. **Install the harness** and configure it.
   ```bash
   npm i -D @playwright/test playwright pixelmatch pngjs && npx playwright install --with-deps chromium
   cp scripts/e2e.config.example.json scripts/e2e.config.json   # edit baseUrl, webServer, suites, flows
   ```
   Merge `scripts/package.snippet.json` into `package.json` for the npm scripts.

3. **Wire the existing suites** and gate on them — cheapest first, so a broken
   build fails in seconds. → `references/02-test-suites.md`
   ```bash
   node scripts/run-suites.mjs --detect        # propose a suites[] block from the repo
   npm run test:suites                         # -> test-report/suites-report.md
   ```

4. **Write the feature flows** — one journey each, targeted by role + accessible
   name, asserting the user-visible outcome, including the unhappy paths.
   → `references/03-authoring-flows.md`
   ```bash
   npm run test:e2e -- --flow checkout --headed   # watch a single flow run
   npm run test:e2e                               # the gate -> gallery.html + e2e-report.md
   ```

5. **Lock the look in place** with visual regression on the named `shot` steps.
   → `references/04-screenshots-and-visual-regression.md`
   ```bash
   npm run test:visual          # fails on the first run: no baseline yet
   npm run test:visual:update   # review the screenshots, THEN accept them
   ```

6. **Find what you didn't test.** The crawler inventories the app and names the
   routes and controls no flow touches. → `references/05-feature-coverage.md`
   ```bash
   npm run test:coverage            # -> coverage-report.md
   npm run test:coverage -- --suggest   # stub flows for the gaps (assertions are on you)
   ```

7. **Triage red, and treat flaky as broken.** → `references/06-debugging-and-flakes.md`

8. **Gate in CI**, then complete the human sign-off for what automation can't
   reach (real emails, payments, migrations, exploratory). → `references/07-ci-and-sign-off.md`
   ```bash
   npm run test:ci        # Playwright, with retries + trace on failure
   npm run test:report    # open the HTML report
   ```

## Try it on the bundled demo first

The skill ships a working app **and a deliberately broken one**, so you can watch
the gate pass and then watch it fail. A gate you've never seen fail is not a gate.

```bash
node scripts/e2e-drive.mjs --config scripts/e2e.config.demo.json         # 7 flows, exit 0
node scripts/examples/serve-demo.mjs --port 5177 --broken &              # inject a real bug
node scripts/e2e-drive.mjs --config scripts/e2e.config.demo.json         # 4 flows fail, exit 1
```

## What's in this skill

- `scripts/run-suites.mjs` — runs the project's unit/integration/regression suites (any language: a suite is a shell command + an exit code), parses JUnit XML for per-test detail, gates on required failures. `--detect` proposes a config from the repo.
- `scripts/e2e-drive.mjs` — **the main gate.** Boots the app, drives every flow in a real browser, screenshots each step, and watches the console, uncaught exceptions and the network the whole time. Writes `e2e-report.md` + a screenshot `gallery.html`.
- `scripts/visual-diff.mjs` — pixel-diffs the named `shot` screenshots against a committed baseline; `--update` to accept an intended change.
- `scripts/crawl-features.mjs` — crawls the running app and reports which routes/controls **no flow touches**. The honest half of "we tested everything".
- `scripts/e2e.spec.ts` + `e2e.setup.ts` + `playwright.config.ts` — the same flows under `@playwright/test` for CI: retries, traces, video, HTML report.
- `scripts/lib/` — config (+ `${ENV}` interpolation), server boot, the flow driver, the observers, the report writers.
- `scripts/examples/` — the demo app, its `--broken` mode, and a fake unit suite.
- `references/01–07` — strategy & honest scope, test suites, authoring flows, screenshots & visual regression, feature coverage, debugging & flakes, CI & the manual sign-off checklist.

## Definition of done

- [ ] `test:suites` green — every required suite passes.
- [ ] `test:e2e` green — every critical journey runs, **including the unhappy
      paths**, with 0 console errors, 0 uncaught exceptions, 0 failed requests.
- [ ] `test:visual` green against a reviewed baseline (generated in the same
      environment CI uses).
- [ ] `test:coverage` read, not just run — every uncovered route/control is either
      now covered or explicitly, in writing, decided not to be.
- [ ] No flow is ⚠️ flaky. A flaky flow is an unfixed bug.
- [ ] Every bug fixed in this change has a regression test that **fails on the old
      code**.
- [ ] CI runs the gate; the manual sign-off checklist (`07`) is complete.

## Guardrails — avoid these mistakes

- **Don't claim the app works because the gate is green.** It proves your flows
  passed. Say what's untested; the coverage report tells you.
- **A flow with no assertion tests nothing.** Clicking through the UI only proves
  the app didn't crash. State what "it worked" means.
- **Target by role + accessible name, not CSS classes.** A selector that breaks on
  a class rename teaches the team to ignore the suite.
- **`{"do":"wait","ms":500}` is the #1 cause of flakes.** Wait for a condition
  (`waitFor`, `expect*`), not for the clock.
- **Flaky ≠ passing.** Retries exist to *surface* flakes, not to hide them. Never
  fix one by raising a timeout.
- **Never `--update` a baseline to make the build green.** Look at the diff first;
  that is the moment the regression either gets caught or ships.
- **Don't silence your own app's console errors.** `ignoreConsole` is for
  third-party noise. An exception your code throws is a bug, not noise.
- **Test the production build**, and never against production data or a real user
  account.
- **Don't put Playwright/Cypress in `suites`** — the e2e layer is the other
  scripts; you'd run it twice.