Frontend Verify

Frontend Verify

What this replaces

The slow loop is: edit code, start the dev server, open a browser, click each page, watch the console, eyeball the layout, repeat. This skill does that pass programmatically with @playwright/cli running headless, and it does it without dumping every page state into the model context.

The one principle that matters

Reading a full accessibility snapshot or a screenshot into context on every route is the expensive part, not running the browser. So the order of operations is always cheapest signal first:

1. Console errors and failed network requests. Tiny text, catches most real breakage (crashed component, bad fetch, 500 from an API route).
2. Targeted text assertions. Ask the page "is the word Dashboard on screen", not "give me the whole DOM".
3. A snapshot written to disk, read back only for a route that already failed.
4. A screenshot, only as a last resort.

verify-routes.mjs runs steps 1 and 2 across all changed routes in one browser pass, writes the detail to disk, and prints a compact PASS / WARN / FAIL table. You read the table, then open detail files for flagged routes only. Do not read detail for routes that passed.

When to take a screenshot

Almost never. Reach for one only when:

• A route is canvas, WebGL, or PixiJS. The accessibility tree is blind to pixels drawn on a canvas, so text and console checks cannot see the actual render.
• You are chasing a visual or layout regression (overlap, spacing, z-index, styling) that text cannot describe.
• The user explicitly asks to see the page.

For everything else, the console plus a text assertion tells you whether the page works. A screenshot tells you how it looks, which is a different and more expensive question.

Setup check

Confirm the CLI is installed before running anything:

playwright-cli --help

If that fails, the user installs it once with (PowerShell):

npm install -g @playwright/cli@latest
playwright-cli install-browser chrome-for-testing

It runs headless by default, so no window opens during a verify run.

The flow

1. Find what changed

git diff --name-only
git diff --name-only --staged

Keep the files under the frontend (for Next.js that is app/, pages/, components/, src/). Ignore server only, config, and test files unless they back a route you are checking.

2. Derive the affected routes

This is the judgment step. Map changed files to URLs:

• Next.js app router: app/dashboard/page.tsx serves /dashboard. app/page.tsx serves /. Strip route groups in parentheses, so app/(marketing)/pricing/page.tsx serves /pricing. A [slug] segment needs a real value, so pick one that exists (for example /blog/hello-world).
• Next.js pages router: pages/about.tsx serves /about, pages/index.tsx serves /.
• A changed shared component does not map to a route on its own. Find the pages that import it and check those:

  git grep -l "PricingCard" -- "*.tsx" "*.jsx"
  

  Then map those importing files to routes the same way. Walk up until you reach files that are actual routes.

If the route set is unclear, ask the user which routes the change should affect rather than crawling the whole site. Verify only what changed.

3. Make sure the dev server is running

The config points at a base URL like http://localhost:3000. If nothing is serving there, start the dev server (for example npm run dev) in a separate terminal first, or ask the user to. verify-routes.mjs will report a navigation failure if the server is down, which is the signal to start it.

4. Write a config and run

Create a small JSON config (shape below) listing the affected routes, then:

node <skill-path>/scripts/verify-routes.mjs verify.json

The script exits non zero if any route fails, so it slots into a chain that should stop on failure.

5. Read the summary, not the world

The script prints something like:

[PASS] /
[FAIL] /pricing    2 JS console error(s); 1 request failure(s)
[WARN] /play       canvas route: accessibility tree is blind

That table plus report.json is usually all you need. Only open .frontend-verify/<route>/detail.json for a route marked FAIL or WARN. That file has the exact error lines and the failed request log. console.txt and requests.txt sit next to it if you want the raw capture.

6. Drill in only when flagged

For a failing route, after reading its detail.json:

• Need to see the rendered structure: write a snapshot to disk and read that one file, do not stream it inline.

  playwright-cli -s=fe-verify goto http://localhost:3000/pricing
  playwright-cli -s=fe-verify snapshot --filename snap.yml
  

• Need one specific value: use a targeted eval instead of a whole snapshot.

  playwright-cli -s=fe-verify --raw eval "document.querySelector('h1')?.innerText"
  

7. Decide on a screenshot

Apply the rule above. If the route is canvas or PixiJS, or the bug is visual, take one screenshot and look. Otherwise stop, you already know if it works.

8. Report

State, per route, PASS or FAIL and the reason, then a one line verdict. Do not restate passing detail. See the report template at the end.

Config shape

{
  "baseUrl": "http://localhost:3000",
  "session": "fe-verify",
  "rootDir": ".",
  "widths": [375, 1440],
  "viewportHeight": 900,
  "settleMs": 800,
  "apiFilter": "/api/",
  "checkWarnings": false,
  "occlusion": false,
  "axe": false,
  "outDir": ".frontend-verify",
  "mutePath": null,
  "routes": [
    {
      "path": "/",
      "expectText": ["Dashboard"],
      "expectNoText": ["NaN", "undefined"]
    },
    {
      "path": "/pricing",
      "waitForText": "Pro plan"
    },
    {
      "path": "/play",
      "canvas": true
    }
  ]
}

Field notes:

• expectText: substrings that must appear in the page body. Missing one is a FAIL.
• expectNoText: substrings that must not appear. "NaN" and "undefined" are cheap catches for broken data binding.
• waitForText: for async or client rendered routes, poll until this text shows before checking. Use it when content arrives after a fetch.
• canvas: marks a canvas / WebGL / PixiJS route so a clean text pass is reported as WARN, not a false PASS, since the checks cannot see the canvas.
• settleMs: pause after load so client side fetches fire and failed API calls register. Raise it for slow pages.
• apiFilter: regex, only list requests whose URL matches. /api/ keeps the request log focused on your own calls.
• checkWarnings: set true to surface console warnings as WARN. Off by default so warning noise does not bury real failures.
• rootDir: the target repo's root. Used to auto-discover routes (see routes: "auto" below) and to resolve axe-core from that repo's node_modules. Defaults to the current working directory.
• routes: instead of an array, set this to the string "auto" to walk a Next.js app-router tree under rootDir (app/ or src/app/) and derive the route list from page.* files. Route groups like (marketing) are stripped; dynamic segments ([slug]) are skipped and reported as skipped rather than silently dropped, since auto-discovery cannot guess a real param value for them.
• widths: array of viewport widths in px to probe at. Defaults to [1440]. Each defect is tagged with the width it was caught at, so the same element failing at two widths is two findings, not one.
• viewportHeight: height in px used for every resize. Defaults to 900.
• occlusion: set true to also flag interactive controls whose centre point is covered by a different element. Off by default — sticky headers and custom dropdowns are the expected false-positive source, so treat it as unproven and review its findings by eye.
• axe: set true to also run axe-core's WCAG ruleset (needs axe-core installed in the target repo — resolved from rootDir's node_modules). Off by default: it costs about 70 seconds per route on Windows, because axe.min.js (560KB) has to reach the page as ~110 chunked eval calls per route to stay under the cmd.exe command-line limit. When off, or when axe-core cannot be resolved, the run prints one line and continues — it never fails the run.
• mutePath: path to the mute file. Defaults to <outDir>/muted.json.

Interaction (optional, off by default)

By default the scanner only ever measures each route's first paint. Tabs, accordions, dialogs, filled forms and validation-error states are never rendered, so the rules never see them. Turn on exploration and it reaches those states, then runs the same seven rules there.

"explore": {
  "enabled": true,
  "budgetMs": 60000,
  "invalidPass": true,
  "mutate": ["#new-policy-form"],
  "skip": [".danger-zone"]
},
"flows": {
  "/settings": [
    { "click": "#billing-tab" },
    { "fill": { "#seats": "25" } },
    { "click": "#review", "scan": true }
  ]
}

Nothing changes unless you opt in. With explore absent or enabled: false, output is byte-identical to before this feature existed, down to the absence of the state key.

It will not change your data. Auto-exploration only touches controls it can positively identify as safe: [role=tab], summary, [aria-expanded], [aria-haspopup], and typed form fields. Everything else is treated as mutating and skipped, including any button[type=button], because a type=button can call fetch('/api/delete') and nothing in the DOM distinguishes that from a tab switch. Submits, destructive labels, and links that navigate away are always skipped. Forms are filled but never submitted.

To act on something classified mutating, name its exact selector in mutate. skip wins over mutate.

• budgetMs — total exploration time per route, split evenly across widths. Default 60000.
• invalidPass — default true: fill each form with invalid values first (to reach error states), then valid ones. Set false if validation is server-side and invisible without a submit.
• flows — hand-written steps for depth auto-exploration will not attempt, such as step 3 of a wizard. Flows run whether or not explore.enabled is set, and are not budget-limited, because you chose the steps yourself.

Each finding gains the state it was found in. A defect present in several states is reported once, listing the others, so turning this on cannot flood the report.

Report page

verify-routes.mjs writes JSON; it does not render anything on its own. To see the findings as evidence cards (crop image first and large, then the rule, then one short line) instead of reading report.json by hand, run:

node <skill-path>/scripts/report-server.mjs <outDir> [<outDir> ...]

Then open http://localhost:7788. Pass one outDir per repo to see several projects on the same page. "new since last run only" is checked by default, so a recurring check does not get buried under everything it already found last time. Copy fix prompt copies a ready-to-paste instruction for fixing that one finding; Mute hides a finding (it reappears if the underlying page changes, since the fingerprint used to remember a mute excludes the message text on purpose).

Auth protected routes

Log in once and save the browser state, then point the config at it:

playwright-cli -s=fe-verify open
playwright-cli -s=fe-verify goto http://localhost:3000/login
# drive the login with find / form_input / eval, then:
playwright-cli -s=fe-verify state-save auth.json

Add "stateFile": "auth.json" to the config. The script loads it before visiting routes, so protected pages render as a logged in user.

Token rules

Do:

• Run verify-routes.mjs once over the changed routes and read the summary.
• Open detail files only for flagged routes.
• Use --filter on requests and targeted eval to pull single values.
• Write snapshots and screenshots to disk; read a file back only when needed.

Do not:

• Crawl or verify routes the change did not touch.
• Read a full snapshot or screenshot into context just to confirm a page loaded.
• Screenshot a route the console and text checks already cleared.
• Re-snapshot a route on every small edit when the console is already clean.

Caveats

• Canvas, WebGL, and PixiJS render to pixels the accessibility tree cannot read. Verify those by evaluating app state on window (for example a game store or a ready flag) or by taking one screenshot.
• Web components using shadow DOM can hide content from the snapshot. Standard Next.js, React, and Tailwind are not affected. If a Lit or web component route reads empty, fall back to a screenshot.
• playwright-cli goto exits 0 even when navigation fails (connection refused, DNS error). The script already detects this from the command output, so trust its navigation-failed result over a raw exit code if you run goto yourself.

Report template

Frontend verify: <branch or change summary>

[PASS] /                ok
[FAIL] /pricing         2 JS console errors, 1 failed /api/plans (500)
[WARN] /play            canvas route, screenshot checked: renders correctly

Verdict: 1 route broken. /pricing throws in PricingCard and its plans
fetch returns 500. Fix before shipping.

Files

• scripts/verify-routes.mjs: the verification driver. Reads a JSON config, runs one headless pass, writes detail to disk, prints the summary, exits non zero on any failure.
• references/playwright-cli-cheatsheet.md: the verification focused command list and the output-format gotchas. Read it before driving playwright-cli by hand.