UI Sweep

ui-sweep — full UI interaction sweep

Before SOP screenshots, systematically click through the interface to catch the silent potholes — buttons that do nothing when clicked, buttons that throw errors — before you shoot the SOP or sign off a regression pass. Faster than eyeballing it by hand, and with deterministic coverage guarantees that random monkey testing doesn't give you. The driver is vercel-labs/agent-browser (★40.5k, Apache-2.0, a Rust CLI built for AI agents: a11y snapshots with element references, click occlusion detection built in, console/errors capture, explicit confirm/prompt handling, state save/load for login-state injection), plus a self-built ~350-line traversal orchestrator (scripts/sweep.mjs, which takes ownership of the agent-browser session lifecycle and runs a per-click domain check — see "The domain boundary has two layers" below). Verified in a real run against an internal SPA project on 2026-08-13: 10 screens, 218 ledger records (205 actual clicks), 0 page exceptions, 0 console errors, and it caught 1 real defect (an "Add"-type button that did nothing at all on empty input — no hint, no change).

Requirements

• npm i -g agent-browser — bundles Chrome for Testing, no Playwright dependency (Apache-2.0). This skill was written against 0.27.0 (headless Chrome for Testing 152); behavior on other versions may differ.
• Node.js (to run scripts/sweep.mjs and scripts/export-state.mjs); scripts/export-state.mjs additionally depends on playwright-core (declared in scripts/package.json, one npm i covers it).

The six-step flow

1. Check dependencies: confirm agent-browser --version runs; confirm the target is a site you own or are authorized to test (see the safety boundary below).
2. Inject login state: run scripts/export-state.mjs --cdp <CDP address> --out <state.json> --origin <origin>... to export cookies plus each origin's localStorage from an already-logged-in real browser (connected over CDP), producing a {cookies, origins} structure. Put that state.json path into the STATE_FILE field of sweep.config.mjs — don't run agent-browser state load yourself. The engine loads it automatically inside the restricted session it builds (see step 4, "Run the engine"; loading it manually would spin up a session before the engine takes ownership of the session lifecycle, which makes the engine's --allowed-domains a no-op for that session — verified against 0.27.0). Never touch plaintext passwords (the whole flow never types a password — it only relays state from an already-logged-in browser); delete the login-state file the moment you're done with it, and never commit it to git.
3. Build the screen list (the one part that needs per-project customization): first poke around manually, or agent-browser open <ROOT> + snapshot -i, to learn the structure of the first screen, then write each "screen" into the project's own sweep.config.mjs. A "screen" is a deterministic recovery path from ROOT ({id, path: [{css}|{text}], settleMs?} — path is the sequence of elements to click, in order, to reach that screen from the root). Every distinct state of an SPA's overlays/drawers/tabs counts as its own separate screen — don't skip any. When orphan-feature reconciliation is needed: in this same step, generate INVENTORY per the "Orphan-feature reconciliation (optional)" section below and put it into sweep.config.mjs (skip it and that feature stays entirely off, silently).
4. Run the engine: node scripts/sweep.mjs <path>/sweep.config.mjs. Run --check-config first to confirm the config is valid (missing ROOT/SCREENS fails fast instead of silently running an empty sweep; the --check-config output also prints the ALLOWED_DOMAINS/STATE_FILE values the engine will actually use, so you can double-check them before pointing the sweep at a real site); run the full sweep in default mode (without --strict — see the trade-off note below). The engine takes ownership of the agent-browser session lifecycle: on startup it first closes its own dedicated isolated session (--session ui-sweep — it never touches any other session you have running), then rebuilds it fresh with --allowed-domains <ALLOWED_DOMAINS>. This step is required because --allowed-domains only takes effect at the moment the browser process launches — an already-running session doesn't pick up a new value on later commands (verified against 0.27.0). When sweep.config.mjs sets STATE_FILE, the engine loads it itself right after this fresh session is up. When sweep.config.mjs doesn't set ALLOWED_DOMAINS, the engine derives it from ROOT's hostname; to allow multiple sub-/upstream domains, set ALLOWED_DOMAINS = 'a.example.com,b.example.com' (comma-separated) explicitly in the config. See "The domain boundary has two layers" below for what --allowed-domains actually does and doesn't cover — it doesn't stop click-triggered navigation; the reliable boundary is the engine's own per-click domain check.
5. Read the results: the engine buckets every click into one of eight categories — ok-changed (fingerprint changed, including checkbox checked state), dead (no reaction at all), dialog-dismissed (a dialog was dismissed), click-error (the click command itself errored), page-error (the page threw an uncaught exception), skipped-denylist (matched the denylist and was never clicked), miss-not-found (couldn't locate the target by role+name, and the by-index fallback also came up empty), left-domain (the click navigated outside ALLOWED_DOMAINS; the engine already forced a recovery — see "The domain boundary has two layers" below). There's also note-relocated-by-index — this is an annotation, not a separate category: when a target can't be located by role+name, the engine falls back to "the same position in the same-role sequence", logs this annotation (with the name it actually landed on), and the click's real outcome still lands in one of the eight categories above — it isn't counted separately. False-positive triage method (distilled from the real run — apply these filters before deciding whether real-browser verification is needed):
   • For click-error, check the dialog ledger first — a synchronous window.prompt/confirm blocks the click command until it times out, and this kind of "error" is often the button working fine, just blocked by a synchronous dialog; if the ledger shows the prompt content and a dismissal record, that rules it out.
   • For dead, think "same-screen state accumulation" first: the engine doesn't restore the baseline after every single click by default (see the --strict note), so state left over from an earlier click on the same screen — an expanded panel, a hint already shown — can mask the effect of the current click and produce a false dead. Re-run the suspicious dead entries with --strict before drawing a conclusion.
   • For miss-not-found, think "lost target" first: it's usually an earlier click that changed the panel state (the target got hidden/scrolled out/the layout reflowed), or a language/theme switch that shifted the element's name, so both the direct role+name match and the by-index fallback come up empty. It isn't necessarily a product defect, but if the same element goes missing consistently across runs, it's worth confirming by hand whether it was really removed, or its role/name genuinely changed (update the screen list in sweep.config.mjs accordingly).
   • left-domain doesn't need "triage" — it's a safety-boundary event, not something for product-defect judgment. Every left-domain entry in the ledger has already been recovered by the engine on the spot, so it doesn't affect later clicks. If one element consistently produces left-domain (e.g. an in-product link to a legitimate upstream domain), that's expected behavior, not a defect — just add that domain to ALLOWED_DOMAINS so it participates in the sweep normally.
   • Navigation buttons for the current state (already on that page) and header/page copies hidden behind an overlay legitimately show as dead — that's not a defect.
   • Under headless, browser APIs like fullscreen or native date pickers can be no-ops — that's an environment limitation, not a product bug.
   • A real defect (a dead/click-error/page-error judged to be an actual product problem) must be verified one by one in a real browser before it's called a defect — never conclude from the ledger alone.
6. Produce the report: follow the skeleton in references/report-template.md — six sections: totals, real defects (verified), false-positive triage (with the exclusion reasoning for each), observations (not indicted, but worth flagging), orphan-feature reconciliation (see "Orphan-feature reconciliation (optional)" below; drop this section when INVENTORY isn't configured), and coverage gaps (an honest accounting of what wasn't clicked and why). The orphan-feature reconciliation section must name every field of the orphan-audit ledger record, one report location per field: unreachable (the suspected-orphan list, triaged entry by entry), broken (entries that don't reach, each with its status code), exempt_hits (entries matched by exempt, count only), inventory_hit_count/inventory_auditable_count (the coverage-rate numerator/denominator — the denominator already excludes exempt, which is inherently unreachable and shouldn't count against coverage; seen_count is a separate field for the total number of observed entries, including ones not in INVENTORY — diagnostic only, not the coverage numerator), and coverage_note (the coverage-gap explanation — it must be carried into the report's "Reconciliation coverage gaps" section verbatim, never dropped or paraphrased down to a summary; when network collection failed or ran zero times, this field carries a warning, and the report must surface that warning as-is, not quietly suppress it because it reads as bad news).

The --strict mode trade-off

Default (without --strict): within a screen, the baseline is restored only after a click that actually changed state; a run of consecutive "no reaction" clicks doesn't re-restore each time — this runs fast (roughly half the time of --strict), at the cost that same-screen state can accumulate and mask a later click's real effect, producing the dead false positives mentioned above.

--strict: unconditionally restores the baseline (re-runs restore(screen)) after every single click before moving to the next one — fixes the state-accumulation false positive, at the cost of running roughly 2x slower for a full sweep. Recommendation: run the first full-site pass in default mode, then re-run the suspicious entries in the dead list with --strict to verify them — you don't need to force --strict for the whole site.

Orphan-feature reconciliation (optional)

The flow above is a traversal view: it clicks through whatever elements exist in the UI and asks "did clicking this do nothing?" It's black-box, so it structurally can't catch "the feature is implemented but there's no UI entry point for it at all" — because something with no entry point never shows up in the a11y snapshot. This section adds a reconciliation view: a code-side inventory (INVENTORY) ∩ what the traversal actually reached (network requests) → take the set difference; the two views are complementary. When INVENTORY isn't configured, this whole feature is silently off — behavior is identical to not having this section at all, zero side effects.

Three verdicts

Entries whose coverage was incomplete (a screen hit restore-failed / miss-not-found / etc. and didn't finish) get tagged note-partial-coverage instead — incomplete coverage must never be called unreachable; this is the same honest-accounting principle as the report template's "Coverage gaps" section, applied here.

How to generate INVENTORY

The engine (scripts/sweep.mjs) doesn't know anything about any particular tech stack — the inventory has to be generated on the spot by Claude, based on what the project actually uses, and put into the INVENTORY field of sweep.config.mjs:

export const INVENTORY = {
  apis: ['GET /api/projects', 'POST /api/projects/:id/archive'],  // method + path pattern, :id stands for a variable segment
  routes: ['/dashboard', '/projects/:id'],                        // frontend route patterns
  exempt: [/^POST \/api\/webhooks/, /^GET \/health$/],            // expected to have no UI entry point, excluded from unreachable
}

Extraction technique by common stack (search for route/endpoint registration code, not business logic):

• Go chi/gin: search for route registration calls like r.Get(/r.Post( (chi) or router.GET(/router.POST( (gin); the path literal is an API inventory entry, and any :id/{id} segment gets normalized to :id.
• Express: search for router.<method>(path, ...) calls like router.get(/app.post(; take the path literal the same way.
• FastAPI: search for decorators like @app.get(/@router.post(; the path is a decorator argument.
• React Router: search for <Route path="..."> or the path fields inside createBrowserRouter([{ path: "..." }]); assemble into the routes inventory.
• Vue Router: search for the path field on each object in the routes array.

After generating it, you must spot-check it against real files: randomly sample 3–5 entries from INVENTORY, open the corresponding source file, and confirm the path string actually exists and the method is actually right — never list entries from memory or make them up; if the inventory itself is fabricated, every downstream set-difference conclusion is invalid too.

Four-way triage guide (mirrors the dead-click false-positive triage)

Before treating unreachable/broken-entry entries as product defects, filter them through these first:

1. Permission-related — an admin/elevated entry point will necessarily show unreachable when the traversal runs under a lower-privilege login (the button literally isn't visible without that login state) → re-run with a higher-privilege login state, or add the entry to exempt.
2. Data-state-related — an entry point that only appears given specific data state (e.g. no "Edit" button when a list is empty) → seed the required data before the traversal, or mark it in the report as a "conditional entry point" rather than an orphan.
3. Denylist-related — destructive buttons are only logged this run, never clicked (skipped-denylist), so their corresponding API naturally shows zero triggers — that's not "no entry point", it's "not clicked this run" → cross-reference the specific element names logged under skipped-denylist (the " items denylisted-unclicked ()" line in the report template's "Coverage gaps" section) against the unreachable list; rule out anything that could only be triggered by one of those elements, then move on to the next filter for what's left.
4. Coverage-related — a screen that didn't finish due to restore-failed / miss-not-found / etc. → entries tied to that screen must not be called unreachable, only tagged note-partial-coverage; recommend a follow-up run with full coverage before re-triaging.

The three conditions for a real orphan (all must hold)

1. The feature genuinely exists in the code (the INVENTORY entry's underlying route/endpoint code exists, already spot-checked);
2. A full-text search of the frontend source finds no code anywhere that calls this endpoint/route (not "the traversal didn't reach it" — "nothing in the code ever references it");
3. Zero triggers across the whole traversal run, and the four triage filters above have already been ruled out.

Only when all three hold can you write "real orphan" into the report's conclusion; satisfying the "zero triggers" condition alone only makes it an unreachable candidate — it isn't enough to call it.

The safety boundary is unchanged: reconciliation only reads the network requests log to normalize and diff — it adds no write operations of any kind, and doesn't change the traversal's own denylist or confirm/prompt dismissal rules.

Safety boundary (hard rules)

• Destructive buttons are logged, never clicked: the engine has a built-in default denylist — revoke, dissolve, delete, remove, discard, drop, log out / logout / sign out, archive, clear, reset, send, save, upload, as regexes (most also matched in Chinese: 吊销/解散/删除/退出/归档/清除/重置/发送/保存/上传; remove/discard/drop are English-only additions layered on top of the existing 删除/清除 Chinese coverage — added 2026-08-13: a real English-UI <button>Delete item</button> was previously getting clicked for real, POST and all, because the old regex only had the Chinese 删除, not the English word "delete"). Matches are only logged to the ledger (skipped-denylist), never clicked. A project can append to the denylist via DENY_EXTRA (a regex) in sweep.config.mjs, but there is no configuration path that can trim or override the default denylist — the safety floor cannot be weakened by config. Site-specific noise buttons (e.g. some non-destructive button you'd rather skip every run) don't belong in the default denylist; add them via your own project's DENY_EXTRA.
• confirm/prompt dialogs are always dismissed: after every click the ledger checks dialog status, and any confirm/prompt/alert detected is unconditionally dismissed, so no real write is ever produced.
• The domain boundary has two layers (verified against 0.27.0; this is an honest two-layer statement, not a single-layer "locks it down" promise):
  1. --allowed-domains <ALLOWED_DOMAINS> only constrains agent-browser's explicit navigation (commands like open/back/forward that specify a URL directly). It does not stop click-triggered navigation — the default behavior of <a href>, location.href, and similar same-window navigation triggered by a click are not covered by this flag; in testing, clicking these actually navigated away. This flag also only takes effect at the moment the browser process launches: preloading state (running state load before the sweep) or reusing an already-running session both make it a no-op for that session, because later commands changing the value don't make an already-launched browser process re-read its config.
  2. The reliable boundary is the engine's own responsibility: after every click, the engine parses the hostname of urlAfter, and if it isn't in the ALLOWED_DOMAINS set, logs left-domain and forces a recovery on the spot — it doesn't wait for the next loop iteration. This layer doesn't depend on agent-browser's flag semantics; it's the domain-boundary promise this skill can actually keep. ALLOWED_DOMAINS defaults to the hostname derived from sweep.config.mjs's ROOT; override it explicitly (comma-separated) in the config when you need to allow more than one domain. To make --allowed-domains at least effective for the "explicit navigation" layer, the engine takes ownership of the session lifecycle itself (see "Run the engine" above): on startup it closes its own dedicated isolated session (--session ui-sweep) and rebuilds it with this flag — no manual steps required from you.
• Only for sites you own or are authorized to test: never run this skill against a third-party site without authorization.
• Delete the login-state file when you're done with it: the state file produced by export-state.mjs holds cookies/localStorage — treat it as a sensitive credential, delete it as soon as you're done, never commit it to git, never upload it; sweep-out/ (screenshots and ledger) is likewise excluded from git — the engine writes a sweep-out/.gitignore (content *) only when the directory has none; an existing one is left untouched — verify yourself that it actually ignores the screenshots and the ledger.

Known limitations (stated honestly)

• Checkbox toggling is only verified at the fingerprint level: the fingerprint includes a count of input:checked/[aria-checked="true"], which catches the signal that "some checked state changed", but doesn't re-verify the semantics of any individual checkbox (what being checked actually means) — manual re-verification is still recommended for cases you're unsure about.
• Some browser APIs are no-ops under headless: fullscreen APIs, native date/color pickers, and similar can fail to produce real effects under headless Chrome — a dead result for these doesn't indicate a product defect.
• Canvas/custom-drawn UI is out of scope: a11y snapshots only see elements the accessibility tree perceives; interactions on a canvas or purely custom-drawn (non-standard-DOM) controls never show up in the traversal plan. This skill can't cover them — they need dedicated verification.

Sources

• Real-run data (source: the real-run ledger, ledger.jsonl): a run against an internal SPA project on 2026-08-13, 10 screens, 218 ledger records (205 actual clicks — 218 minus 11 skipped-denylist minus 2 miss-not-found, neither of which ever issues a click) — ok-changed 106 / dead 97 / dialog-dismissed 0 / click-error 2 / page-error 0 / skipped-denylist 11 / miss-not-found 2; 2 of those hit note-relocated-by-index (relocated-by-index, an annotation rather than a separate category, already counted under the categories above). The engine version used for that run predates the left-domain category (added this round — see "The domain boundary has two layers"), so this dataset has no samples of it. 0 page exceptions, 0 console errors, 1 real defect caught (an "Add"-type button that did nothing on empty input).
• Driver: vercel-labs/agent-browser, tested against 0.27.0 (headless Chrome for Testing 152).
• Research basis: internal tooling research (GitHub candidate comparison — agent-browser vs. playwright-mcp/chrome-devtools-mcp (MCP form; token/latency cost doesn't pencil out for a full sweep) vs. browser-use (non-deterministic) vs. crawlee (a URL crawler; the SPA in-screen state logic still has to be hand-written) vs. gremlins.js (random monkey testing, unmaintained)).