# A11Y Test

> Use when you need to run real accessibility tests — Playwright keyboard interactions, axe-core scanning, visual regression, and WCAG 2.2 compliance checks. The measurement layer that feeds evidence into a11y-critic reviews.

- Skill: `zivtech/a11y-test` (Agent Skill, multi-file: 8 files)
- Install (CLI): `npx skillmds@latest add zivtech/a11y-test`
- Raw SKILL.md: https://api.skillmd.com/api/skills/zivtech/a11y-test/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Web & Frontend
- License: Apache-2.0
- Author: zivtech (https://skillmd.com/u/zivtech)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/zivtech/a11y-test

---


# Accessibility Testing Skill

## Browser Tooling Routing (read first)

Pick the right execution mode from the routing table before running anything (the table is the source of truth — don't trust remembered mode counts):

| Task | Tool | Why |
|---|---|---|
| Codified CI keyboard tests, visual regression, axe-core scans, WCAG compliance suites | `npx playwright test` with `.spec.js` files | Real keyboard events, CI-runnable, version-controlled, reproducible. Primary path — all mandatory rules below still apply. |
| Baseline sweep across a list of URLs — machine-readable axe-core evidence per page, no `.spec.js` authoring, not CI-embedded | [`references/baseline-url-scan.mjs`](references/baseline-url-scan.mjs) (in-repo reference script; peer deps `playwright` + `@axe-core/playwright`) | Sequential per-page axe scan + summary JSON for baseline/regression evidence across many pages in one run. `--census` adds DOM-census heuristics (empty paragraphs, autocomplete-absence, duplicate ids); `--alt-snapshot` writes a diffable per-page alt-text map. Detector output, not a conformance verdict — axe-detectable subset (plus heuristics) only. See "Baseline URL-list scan" below. |
| Interactive agent-driven reconnaissance: snapshot ARIA structure, navigate a SPA to reach the page under test, verify a fix in place, capture annotated screenshots, probe a disclosure/menu/modal without writing a test file | `agent-browser` CLI (snapshot+ref pattern, persistent CDP daemon, real keyboard events) | One shell call per action, no test-file overhead, returns `@e1`-style refs that map directly to actions. See "Interactive reconnaissance with agent-browser" below. |
| Generate a test script from a prose spec ("test that this modal traps focus and Escape closes it") | `/webwright:run` or `/webwright:craft` (Claude Code plugin) | LLM generates complete Python Playwright script. Review before trusting. Also captures `aria_snapshot()` for deep ARIA tree inspection. See "Test script generation with Webwright" below. |
| Goal-driven journey audit of a live URL — "can a keyboard-only or screen-reader user complete this task?" — with evidence artifacts | `keyboard-a11y-tester` (external clone, pinned release `0.5.0`; deterministic runner + agent-driven serve/step loop) | URL + goal in, evidence-linked WCAG findings out — no test file needed. Emulated screen-reader announcements, live-region capture, and focus-indicator measurement at the page/journey level that no other mode provides. See "Goal-driven journey audits with keyboard-a11y-tester" below. |
| Component/unit-level screen-reader assertions — accessible names, reading order, live-region announcements — in the project's own test suite (Vitest/Jest+jsdom, Storybook play functions, or a browser page), no URL or journey needed | `@guidepup/virtual-screen-reader` (npm devDependency, exact-pinned `0.32.1`) | Per-component, per-PR spoken-output evidence in milliseconds — the implement→test layer keyboard-a11y-tester can't reach (it needs a deployed URL). Synthetic interactions: never keyboard-operability evidence. See "Component screen-reader assertions with virtual-screen-reader" below. |
| Visual inspection, DOM queries from a conversational session | `agent-browser screenshot` / `agent-browser screenshot --annotate` / `agent-browser snapshot --max-output 8000` | Same daemon, no test runner needed. |
| A person confirms a fix at the fixed stage (the closure record needs its `attestation` block), or walks an ICT baseline row no machine mode covers | [`references/human-verification-walkthrough.md`](references/human-verification-walkthrough.md) | The human tier: a planned-operation-driven walk that records `before` / `action` / `expected` / `observed` per operation in a shape the five admissibility rules can read, plus an attended-media shape for the alternative-content rows. Never a free walk; the one place a single PASS is the unreliable result. See "Human verification walk-through" below. |
| Anything requiring real keyboard event delivery through an MCP wrapper | **DO NOT USE Playwright MCP.** Its `browser_press_key` calls are silently dropped for most interactive widgets. Use `npx playwright test` or `agent-browser` instead. |

**Decision flowchart:**
```
Do you have a prose description of what to test, but no test script yet?
  YES → /webwright:run (one-shot) or /webwright:craft (reusable parameterized tool)
  NO, you need to run an existing test → npx playwright test
  NO, you have a list of URLs and need machine-readable axe evidence across all of them, no test file → references/baseline-url-scan.mjs (or pa11y-ci --sitemap for sitemap-wide sweeps)
  NO, you need to audit a live URL against a user goal (journey, announcements, focus indicators) → keyboard-a11y-tester
  NO, you need to assert component announcements, names, or reading order in unit tests (no URL yet) → virtual-screen-reader
  NO, you need to explore interactively → agent-browser
  NO, a person must confirm a fix on the pinned version, or walk a baseline row no mode covers → references/human-verification-walkthrough.md
```

**CDP keyboard event delivery for `agent-browser` has been verified end-to-end** on both a vanilla JS disclosure widget (WAI-ARIA APG disclosure-faq example: `focus → press Enter → aria-expanded: false → true`) and a React state-driven modal (react.dev DocSearch: `Meta+K` → React global keydown listener → state-mounted searchbox). The MCP keyboard delivery bug does not apply to `agent-browser` because it calls CDP `Input.dispatchKeyEvent` directly rather than through an MCP wrapper.

## Verification evidence contract

**Evidence type must match the failing condition.** A screenshot is never evidence for an interaction-class fix (keyboard operability, focus behavior, or a status-message announcement) — it shows what a sighted mouse user sees, not what a keyboard or screen-reader user experiences. When a fix's evidence doesn't match its defect class, the fix ships labeled **partial**, naming which defect classes still lack matching evidence.

| Defect class | Evidence REQUIRED before "verified" | Mode |
|---|---|---|
| `keyboard-operability` — Keyboard operability (reachable, operable with Tab/Enter/Space/Escape/arrows) | Real-keyboard Playwright transcript — actual `page.keyboard.press()` calls, never ARIA-attribute inspection alone | `npx playwright test` |
| `focus-order-indicator` — Focus order & focus-visible sufficiency | Journey-level focus trace evidence | `keyboard-a11y-tester` |
| `name-role-state` — Accessible name/role/state; status-message announcements | Assertion output against actual computed screen-reader output | `virtual-screen-reader` |
| `machine-detectable` — Machine-detectable semantics, contrast, alt-presence (a rule fires or stops firing) | Re-scan of the touched page(s) after the fix | `baseline-url-scan.mjs` (axe-core violations; `--census`/`--alt-snapshot` for the heuristic classes) |
| `visual-only` — Visual-only classes (layout, spacing, color/swatch correctness) | Screenshot comparison | screenshots (`agent-browser screenshot` / Playwright screenshot) |

This table is what `a11y-critic` Phase 0 checks a remediation's attached evidence against, and what `bug-reporting`'s "Verification evidence" field cites.

### Detector-lane authority boundary

A detector PASS means only "no detection fired for this route, state, viewport, config, and version" — never a WCAG, Section 508, keyboard, or assistive-technology verdict. Cross-tool agreement on the same target raises triage priority; it never confirms a defect by itself, and an absence of detection is not evidence of conformance.

**An infrastructure limit must never emit a canonical result.** A step-cap watchdog, a timeout, or a crashed collector is an *abort*, not a PASS/FAIL/BLOCKED outcome — record it as what it is (aborted, incomplete, environment-limited) and keep it out of the pass/fail denominator until it is resolved.

**Mandatory cross-check rule:** whenever a run's non-conclusive rate (`BLOCKED`, `cantTell`, or equivalent) approaches saturation for a batch — most of the sampled set landing in a single non-conclusive bucket rather than spread across pass/fail — treat that as a signal about the collector, not about the product, and cross-check the batch against an independent evidence lane (a different tool, a driven session, or manual sampling) before the numbers reach a client-facing report. A near-saturated non-conclusive rate that ships unchecked reads as "almost entirely untestable," which may simply be an ordinary pass/fail distribution obscured by a collector fault.

Related discipline, restated for this boundary: never promote scanner output straight to a WCAG or Section 508 verdict; never treat count-parity between two runs as completeness; never collapse `cantTell` / informational / skipped / blocked / untested into pass or fail — each stays a distinguishable, visible state (see the coverage-ledger vocabulary in `acr-reporting`'s untested gate for the report-side version of the same rule).

### Evidence retention (append-only)

Never overwrite an evidence run. Failed, intermediate, and superseded captures are retained beside the final result under names that state *why* they are not final (for example `-raw-live-capture`, `-script-error`, `-modifier-mismatch`, `-pre-final-adjudication`). The same discipline extends to generated deliverables: every non-final revision is kept beside the final one with an append-only supersession log, and each non-final revision is explicitly marked not client-facing and not a conformance, certification, publication, or acceptance artifact.

Retention is not bookkeeping for its own sake — it is what makes silent errors findable. A numeric error in an otherwise structurally valid generated deliverable — a formula range that under-counts, a mapping that drops rows — passes schema validation and surfaces only when a later revision can be diffed against the one that was wrong. Overwrite the run and that diff is gone.

`references/hash-evidence.mjs` makes the rule checkable: it writes an append-only `checksums.json` manifest beside an evidence tree and `--verify` lists every modified, missing, or unlisted file, exiting non-zero on drift (modified or missing; unlisted too under `--strict`) — it makes silent edits findable, not impossible.

## Retest classification

Two clauses that govern when a retest result is trustworthy.

**n = 1 is variance, not a finding.** A single failed reproduction attempt is inconclusive, not a FAIL. A FAIL requires the same miss reproduced across two independent sessions (a different run, same conditions). This mirrors the routing rule already in force elsewhere in this bundle for model-benchmark evaluation: a single-lane result that flips under byte-identical conditions is treated as variance until adjudicated by a second, independent pass — never reported as a conclusion on its own.

**A version or content-marker delta forces a fresh retest.** Frozen evidence has an expiry condition tied to the product, not the evidence: the moment the product's version or a tracked content marker changes, every baseline captured before that change stops being admissible as a claim about *current* conformance — it remains valid history and nothing more. Two rules follow:
- Capture the version or content marker as a field on the evidence artifact itself, so a stale-baseline check is mechanical rather than remembered.
- On a detected delta, a fresh retest is mandatory for any row whose claim is about current conformance. "We tested this in a previous cycle" is not, by itself, an outcome — a frozen baseline may never silently stand in for current evidence.

### Human verification walk-through

The retest campaign has a human tier, and it is not a free walk. When a closure record needs its `attestation` block (the fixed-stage confirmation `acr-reporting` admits an improved term on), or when a declared-508 engagement reaches one of the crosswalk's 13 `not-covered` rows, a named person walks entries from the campaign's planned operation set — never off it — and records, per operation, where they started and how they got there, the action, the expected result, and the observed result, in the package shape [`references/human-verification-walkthrough.md`](references/human-verification-walkthrough.md) specifies (an operation shape, and an attended-media shape for the alternative-content rows). The record is appended to the sample's evidence artifact under the append-only rule, and the closure's attestation block cites it.

**The n = 1 rule reverses at the fixed stage.** A single failed reproduction is variance at diagnosis because the expensive error there is a flaky miss reported as a regression. At the fixed stage the expensive error runs the other way — one human PASS becomes a `supports` in a published document, and the person confirming a fix knows what they expect to see. A single human `FAIL` sends the item back to remediation on its own; a human `PASS` that will back a fixed-stage `supports` needs a second package by a different person, or the same person in a separate session on a later day, with at least one of the two not by the fix's author (the closure contract's `second_confirmation`). Only the different-person branch controls the expectation the rule names; the same-person branch controls session and environment variance and is the weaker one — prefer a different person. A walked `BLOCKED` or `FAIL` never attests: the closure stays `draft_not_attested`, the walk is cited, and for `BLOCKED` the instrument that would decide it is named. Photosensitivity-class content (2.3.1 / 2.3.2) is never walked by extended attendance — a capped first look, or a declined one, goes straight to `BLOCKED` with the analyzer named; the reference carries the safety clause.

### Campaign completeness contract

A retest campaign is not complete when the runner exits — it is complete when **zero** planned operations remain unresolved. Treat "the suite ran" and "every planned operation has a result" as different claims until proven equal:

- State an explicit zero-unresolved contract as an exit condition: enumerate the planned operation set before the run starts, and the run does not close out until every entry in that set carries a disposition (pass, fail, or one of the non-pass values above — never silently dropped).
- Provide a recovery path that re-drives specifically the unresolved operations, not the whole batch, when a run exits early.
- Support a resumption contract: an interrupted campaign continues from its unresolved set on the next run rather than restarting from zero.

This is a contract for the evidence a retest run must produce, not a specification for a particular runner implementation — see `docs/a11y-evaluation-report-contract.md` for the report-level half of the same completeness rule.

### Operation-evidence admissibility

Retest evidence is admissible for the operation it claims only when it survives five rules. These govern the *evidence package* for a single operation — a retest hitting a specific target, a keyboard trace, a passive DOM/AX observation — not the accessibility of the target itself. The `evals/suites/a11y-test-operation-evidence` lane exercises each with a clean control.

- **A bounded diagnostic is not a conclusion.** A `focus_stagnation_observed`-class note — focus not advancing on a keyboard probe — is a bounded collector observation, not a WCAG 2.1.2 keyboard-trap finding. Promoting it to a trap conclusion requires a separate trace that attempts the documented exit (press Escape or the exit keys and show focus cannot leave). Absent that trace the operation stays `BLOCKED` where the stagnation observation is admitted evidence about that operation (a collector block, not a conformance outcome); stagnation alone is neither a trap nor a conformance failure.
- **Setup and action must be continuous.** An action's evidence is admissible only if its starting (`before`) identity equals the terminal identity of the setup that immediately preceded it in the *same session*. A setup in one session and an action from a different starting locus in another do not compose into evidence about the planned operation.
- **Conditional states are natural-only.** A state that appears only under a condition (an empty result, an error) stays `UNTESTED` until it occurs naturally under an approved input. Inducing it synthetically — editing a response, forcing the state — does not clear coverage; it shows the message renders, not that the state is reachable in use.
- **Passive observations are bound, never standalone.** A DOM/AX snapshot (roles and states present in the rendered tree) is admissible only as support bound to the causing action and on a source allowlist. By itself it is never evidence of keyboard-reachability or of announcement — those require the causing key press and its observed result.
- **No silent ancestor remapping.** When the exact target is not on the focus path, evidence recorded against a nearest reachable ancestor is admissible only through a reviewed, separately frozen owner/descendant mapping (the composite's documented owner and navigation model). A silent nearest-ancestor substitution is not evidence about the target.

**Human-sourced packages are scored by the same five rules.** A person's walk-through package ([`references/human-verification-walkthrough.md`](references/human-verification-walkthrough.md)) carries the fields the rules read — `before.reached_by` and `before.locus` in one `session` for continuity, `observed_via` for passive binding, `target_reached` for ancestor remapping, the exit attempt in `action` before any trap conclusion, approved inputs only for conditional states. A package missing those fields fails the rule whose predicate it cannot show: "I checked it, it's fine" has no starting locus and no action for its observation to bind to, so it breaks `setup_action_continuity` and `passive_observation_binding` as written; the tidier package with every field filled and `observed: "as expected"` breaks rule 4 alone, because no observed result of the action is bound to it — no sixth rule is needed for either, and the operation stays where its admitted evidence left it. The attended-media shape (audio description, captions, transcripts, and any observation about content rather than an operation) has no locus and no action, so rules 2 and 5 do not apply to it and the reference marks them so; `played` — the stretches actually played, with the alternative selected — is the binding field, and rule 4 is what catches a menu listing or a track label offered as evidence of the alternative. For human evidence rules 2 and 4 are structured self-report rather than a verified identity or a source allowlist; the record says so, and the second confirmation is the control.

### Structured disposition block

Every admissibility review closes with one fenced yaml block a rule-based scorer checks mechanically (the `score_acr.py` precedent). Stable ids for the five rules above: `bounded_diagnostic_not_promoted`, `setup_action_continuity`, `natural_only_conditional_state`, `passive_observation_binding`, `ancestor_remapping_review`.

```yaml
admissibility: ACCEPT | REJECT                                     # ACCEPT exactly when rules_violated is empty
dispositions: {<operation id>: PASS | FAIL | UNTESTED | BLOCKED}   # every operation in the package
rules_violated: {<operation id>: [<rule id>, ...]}                 # only operations whose evidence breaks a rule
claim_boundary: "<per operation: what the admitted evidence establishes, and what it leaves undecided about the target>"
```

The four disposition values are this block's closed set. `PASS` and `FAIL` mean admitted evidence decides the operation's own predicate; `BLOCKED` means an admitted bounded observation *about that operation* stands without the trace or instrument reading that would decide it — a collector's `focus_stagnation_observed` note with no exit-path trace, or a person's "I watched the animation and cannot count flashes per second" with no analyzer reading; the definition was written around collector output and applies unchanged to an attended human observation, whose `claim_boundary` names the missing instrument; `UNTESTED` means no admitted observation bears on it. Rejected evidence never moves an operation: a recorded prior state stays, and an operation with none takes only what its admitted evidence supports. A non-conclusive run state outside the four (`cantTell`, `skipped`) is `UNTESTED` here and named in `claim_boundary`, never folded into `BLOCKED`. `admissibility` and `rules_violated` score the evidence package, not the target's accessibility; `dispositions` carries forward only what admitted evidence establishes about each operation.

### PASS partition: rule-tier vs chain-tier

Not every PASS carries the same evidence. A **rule-tier** pass verified the operation-specific predicate the row is about — this exact control, this exact expected result. A **chain-tier** pass rode a generic keyboard-chain success: the page was navigable and nothing obviously broke, with no operation-specific rule behind it. Counting the two together overstates coverage.

Partition passes into the two tiers and make the ratio visible in the evidence artifact itself, not in prose. A completeness claim reporting a single PASS count — without showing how many rested on an operation-specific rule versus a generic chain — has not established what it claims. This is a self-check on our own output quality, not a statement about the product: a chain-tier-heavy PASS set is a signal to go back and add operation-specific predicates, never a reason to report a high pass rate.
### Evidence consumption (context discipline)

**Evidence artifacts are handles, not payloads.** Reference `trace.json`, `findings.json`, axe-core results, and similar artifacts by path — never paste them wholesale into context. Pull only the fields under adjudication; jq recipes for the common extractions live in [references/evidence-extraction.md](references/evidence-extraction.md).

Screenshots are cited by path in findings and reports. Actually viewing one — reading its image content into context — is reserved for visual-class adjudication (the last row of the evidence table above); every other defect class is decided from extracted fields, not pixels.

Long-running commands (scans, test runs, crawls) redirect stdout to a file; read back only the summary or failing subset, never the full log — then pull what you need with the extraction recipes ([references/evidence-extraction.md](references/evidence-extraction.md)).

`agent-browser` calls in conversational sessions carry `--max-output <chars>` to cap output per call — see "Interactive reconnaissance with agent-browser" below.

## Interactive reconnaissance with agent-browser

For ad-hoc a11y probing inside a conversational session — before writing a `.spec.js` file, when verifying a single fix, or when exploring the ARIA structure of an unfamiliar component — use `agent-browser`. The snapshot+ref pattern eliminates locator hunting:

```bash
agent-browser open https://example.com/component-under-test
agent-browser snapshot -i --max-output 8000        # Returns interactive elements with refs: [ref=e1], [ref=e2]...
agent-browser focus @e1                            # Focus by ref
agent-browser press Enter                          # Real CDP keyboard event
agent-browser get attr @e1 aria-expanded            # Verify state mutation
agent-browser screenshot --annotate                # Numbered overlays mapping to refs (useful for multimodal review)
agent-browser close
```

Key flags: `--profile Default` (reuse the user's Chrome login state for authenticated sites), `--session <name>` (isolated browser per parallel agent), `--json` (parseable output for programmatic checks), `--max-output <chars>` (truncate output to N characters — cap output-heavy commands like `snapshot -i`; a capped snapshot can silently drop element refs, so raise the cap rather than probing an inventory you know is truncated), `--allowed-domains` (safety).

**Keyboard-driving discipline (applies to all interactive modes, this one included):** never send a pre-counted sequence of Tabs. Snapshot/observe, then act on what is actually focused — "Tab until the focused control is named X" is right; "Tab 6 times" is wrong. Confirm success by state change (attribute flip, URL change, announcement), not assumption.

**When to escalate to `npx playwright test`**: when the verification needs to live in CI, run across PR builds, or exercise the 12 APG widget pattern templates below. Reconnaissance with `agent-browser` is for interactive probing; codified regression still belongs in `.spec.js` files.

## Batched one-pass per-page audit (default for rendered-context adjudication)

When you have to adjudicate more than one scanner occurrence on a page — WCAG-EM audit sampling, a scanner gap-review lane, or re-testing a batch of findings — **work the page, not the rule**. A human reviewer loads a page once and reads its headings, control names, images, tables and landmarks in a single pass, then disposes of every finding on that page from that one look. Probing one element per rule (open page, check the empty button; re-open, check the missing alt; re-open, check the layout table…) is the slow anti-pattern this replaces: it re-navigates the same page many times and scatters the evidence.

The default instead: **one DOM + accessibility-tree capture per bound page/state, covering every rule class at once, then adjudicate all of that page's occurrences from the single capture.** [`references/page-audit.mjs`](references/page-audit.mjs) is the reusable auditor — `auditPage()` runs in the page context and returns evidence for five scanner-rule classes in one call (`region_missing`, `heading_empty`, `button_empty`/empty-name controls, `alt_missing`, `table_layout`). Run it through whichever browser mode you are already in (Playwright `page.evaluate(AUDIT_PAGE_SRC)`, `agent-browser eval`, or the in-session JS tool).

Method:

1. **Group the page's occurrences first.** Map the raw scanner occurrences you must adjudicate to their pages/states (each SPA route or query state is a distinct page). An N-state app is N captures, not N × (rules) probes.
2. **One capture per page/state.** Run the auditor once the page is loaded and settled. Persist the returned report as the page's evidence artifact (hash it for the receipt).
3. **Adjudicate every rule for that page from the one report.** The report already carries the empty-name controls, missing-alt images, layout tables, missing landmarks and empty headings together — decide each occurrence's disposition against it without re-navigating.

Discipline (the same rules that govern any evidence here):

- **Detector, not verdict.** The auditor reports the facts a rule keys on; it assigns no WCAG SC, severity, or pass/fail. You do that, per occurrence, in rendered context.
- **Cross-check the computed name.** Its `accName` walk applies ARIA precedence (an empty `aria-labelledby` target yields an empty name), but confirm an empty-name finding against the browser's OWN computed accessible name (Playwright `page.accessibility.snapshot()`, `agent-browser snapshot -i`, or the DevTools Accessibility pane) before mapping it to a criterion — that read decides 4.1.2 (empty name) vs 2.4.6 (present but non-meaningful).
- **Positional locators drift.** A retained XPath from an earlier scan may not resolve in the current build (SPA re-render). Confirm at the rule/element level with a stable selector and disclose the drift; never launder a fresh element into a byte-identical reproduction of an old occurrence, and never force a DISMISS when the exact target could not be relocated.
- **Overlapping views are not additive.** Clusters, raw occurrences, execution groups, criterion cells and ICT rows are different lenses on the same page; do not sum them, and a local observation never proves the whole page passes.

## Attribute-removal differential diagnosis

A finding usually names a **cause**: "the `aria-label` is overriding better visible text", "`tabindex="0"` on the wrapper is why tab order is wrong". A snapshot proves the attribute exists. It does not prove the attribute is *why*. Differential diagnosis closes that gap: remove the suspect attribute, re-observe under identical conditions, and see whether the experience actually improves.

Technique credited to SSA's [YANKI](https://www.ssa.gov/accessibility/yanki/yanki.html) bookmarklet (Accessible Solutions Branch — same team as ANDI), which does exactly this for human testers. The bookmarklet itself is not routed here: in a scripted lane the removal is one line, and scripting it means the before/after pair is *capturable as evidence* rather than something you saw once.

```js
// Playwright. Baseline first — never mutate before you have captured the baseline.
const before = await page.accessibility.snapshot();

await page.evaluate(() =>
  document.querySelectorAll('[aria-label]').forEach(el => el.removeAttribute('aria-label'))
);

const after = await page.accessibility.snapshot();
// The diff is the evidence. Attach both halves, not the conclusion.
```

What each removal tests, and what a positive result means:

| Remove | If the result improves, the finding is |
|---|---|
| `aria-label` / `aria-labelledby` | the accessible name is overriding better visible text (2.5.3, 4.1.2) |
| `role` | an applied role is fighting the element's native semantics |
| `tabindex` > 0 | the author-imposed order is why focus order is illogical (2.4.3) |
| `tabindex="-1"` on an interactive control | the control is being held out of the tab order (2.1.1) |
| `tabindex="0"` on a non-interactive element | tab-order noise, not a real stop |
| `aria-hidden` | content is hidden from assistive technology that should not be |

Rules:

- **It is a diagnostic, never a fix and never a conformance result.** "Removing X made it better" is evidence about a cause. The remediation is still a design decision, and stripping the attribute in production is almost never it.
- **Baseline before you mutate, and re-observe under identical conditions** — same viewport, same route, same focus starting point. A differential taken across two different states measures nothing.
- **Capture both halves.** A before/after pair is admissible under the verification evidence contract; "I removed it and it looked better" is not.
- **Run it in a scratch tab, never a page the user is working in.** The DOM mutation is real.
- **A worse result is also a result.** If removal degrades the experience, the hypothesis is falsified — record that, do not quietly re-run with a different attribute until something confirms the finding you already wrote.

**Not yet verified:** the `agent-browser` equivalent of the `page.evaluate` step. The Playwright form above is the verified route; anyone wiring the agent-browser lane should confirm its own subcommand against `--help` first rather than assuming one exists.

## Test script generation with Webwright

**When to use:** You have a prose a11y requirement (from the planner or a ticket) and need a runnable test script, without hand-writing it.

**What it produces:** A Python Playwright script with navigation, keyboard interactions, ARIA state assertions, and screenshots. Webwright generates `sync_playwright` scripts by default — if you need async for an existing test harness, specify in the prompt.

**Language mismatch warning:** Webwright generates Python. Existing CI is Node.js/.spec.js. Generated scripts are starting points — for CI, port logic to .spec.js using the APG templates below, or run Python directly if a Python test runner is available.

**Example `/webwright:run`** (actual prompt that produced a passing dialog focus trap test in benchmark):
```
/webwright:run Navigate to https://www.w3.org/WAI/ARIA/apg/patterns/dialog-modal/examples/dialog/.
Open the modal by clicking the trigger button.
Verify focus moves into the modal.
Tab through all focusable elements and verify focus wraps (focus trap).
Press Escape and verify the modal closes and focus returns to the trigger.
```

**Quality gate:** The operator must review generated scripts before trusting results. Check that:
- Keyboard events use `page.keyboard.press()` or `locator.press()`, NOT synthetic `dispatchEvent`
- Assertions verify state changes (before/after), not just attribute presence
- No `time.sleep()` > 5 seconds or hardcoded waits that mask timing issues

**ARIA snapshot capability:** Webwright's Playwright environment captures `page.locator("body").aria_snapshot()` — the full accessibility tree with roles, states, and relationships as structured YAML. Richer than `agent-browser snapshot -i --max-output 8000` for structural analysis (captures all 4 tab→panel relationships via aria-controls/aria-labelledby cross-references, vs. agent-browser which shows only interactive element refs).

**Limitations:**
- No built-in axe-core — the LLM must write injection code (it does this correctly; see benchmark task 3c)
- May miss a11y-specific patterns unless the prompt is specific about what to check
- Python scripts don't run in JS CI without a Python runner
- Requires Claude Code plugin install — not available in Codex CLI
- Do not run simultaneously with agent-browser — both launch Chrome instances that may conflict on ports

**Benchmark results (2026-05-26):** 25/25 across 5 WAI-ARIA APG tasks (dialog focus trap, tabs ARIA state, axe-core injection, menu keyboard navigation, ARIA tree inspection). All scripts used real `page.keyboard.press()` calls. Full results in `evals/suites/webwright-benchmark/`.

### Installation

**Prerequisites:** Python 3.10+, Playwright Python (`pip install playwright && playwright install chromium`)

**Two-step install:**
1. `/plugin marketplace add microsoft/Webwright`
2. `/plugin install webwright@webwright`

**If marketplace fails:** `git clone https://github.com/microsoft/Webwright && /plugin install ./Webwright`

**Platform note:** Claude Code plugin only. Not available in Codex CLI. From Codex, the usable browser automation options are `agent-browser` and `keyboard-a11y-tester` (both plain CLIs). Generated `.py` scripts can be executed from Codex via `python3 script.py`.

## Goal-driven journey audits with keyboard-a11y-tester

**When to use:** you have a live URL and a task in plain words ("can a keyboard-only or screen-reader user complete X?") and need evidence-linked WCAG findings without writing a test file — discovery audits, before/after patch evidence, whole-journey reviews. **When NOT to use:** widget CI regression (→ `.spec.js` + the APG templates), quick probing or authenticated Chrome-profile flows (→ `agent-browser`), rule scans (→ axe-core, §4).

**What it is:** [ezufelt/keyboard-a11y-tester](https://github.com/ezufelt/keyboard-a11y-tester) — an external tool, adopted at release `0.5.0` (commit `7e852a7`, MIT; originally adopted at `97eb13e`, bumped 2026-07-11 after upstream merged our PR #7 and began tagging releases — re-verify on every upgrade). Two layers: a deterministic Playwright/CDP runner (real keyboard events only, never `.click()`; machine-decidable WCAG checks; dual-signal focus-indicator measurement) and an emulated screen-reader persona (`@guidepup/virtual-screen-reader`: announcement capture, live-region monitoring, reading-order census). Runs both W3C personas (keyboard "Ade", screen-reader "Lakshmi") in one pass. As of 0.5.0 it also detects broken ARIA ID references and keyboard-focusable controls missing from the accessibility tree, includes our 3.3.2 UA-default-name check, and supports authenticated runs via `--storage-state <playwright-storageState.json>` (agent-browser remains the route when you want to reuse the user's real Chrome profile instead of exporting state). Cross-validated against this repo's 33 critic fixtures on 2026-07-10 — agreement record: `evals/results/keyboard-a11y-tester/README.md`.

### Install (clone path — verified; Node ≥ 20)

```bash
git clone https://github.com/ezufelt/keyboard-a11y-tester && cd keyboard-a11y-tester
git checkout 0.5.0                    # adopted pin (tagged release)
npm install && node scripts/setup-check.mjs   # npx playwright install chromium only if browser_available=false
```

The upstream Claude Code plugin flow (`/plugin marketplace add ezufelt/keyboard-a11y-tester`) exists but is unverified here; the clone path works from both Claude Code and Codex.

### Run

```bash
# Batch blind Tab-crawl (unattended; never presses Enter/Space) — per viewport:
node scripts/runner.mjs --url https://site --viewport desktop --max-steps 40 --out <dir>

# Driven session (the value center — the agent decides every keystroke):
node scripts/runner.mjs serve --url https://site --goal "find and submit the contact form" \
     --viewport desktop --port 9333          # default port is 9333 (9400 in upstream examples is just an example)
# prints: READY <session-dir>
node scripts/runner.mjs observe <session-dir>
node scripts/runner.mjs step <session-dir> --press Tab      # one key → AX name/role/state, focus style, sr_announcement
node scripts/runner.mjs step <session-dir> --type "hello@example.com"
node scripts/runner.mjs finish <session-dir> && node scripts/runner.mjs stop <session-dir>
```

**Core discipline: observe → decide → act** (see the keyboard-driving rule in the agent-browser section — it originated here). Read `sr_announcement.live_announcements` after any action that visibly changes the page: an entry proves the update reaches a screen reader; its absence after a visible change is 4.1.3 failure evidence.

### Artifacts (temp dir, or `--out`)

- `trace.json` — per step: keystroke, selector, CDP AX name/role/state, computed focus style, `focus_moved`, screenshot ref, `sr_announcement`
- `deterministic-findings.json` — `{wcag, persona, conformance_level, confidence, severity, url, locations, persona_impact, evidence[]}`
- `screen-reader-census.json` — whole-page reading order (spoken phrase, role, selector) + declared live regions
- `screenshots/step_NNNN.png` — focused-region crops

These are measured test evidence for a11y-critic reviews (formal Phase 0 tier wiring lands with assessment Phase 3).

### Calibration rules (measured on our 33 fixtures, 2026-07-10)

1. **Batch-mode 4.1.3 "silent live region" findings are never failure evidence.** A blind crawl never operates anything, so correctly-wired regions look silent (confidence 0.35–0.4 vs 0.7+ elsewhere). They are prompts to run a driven session and judge from `live_announcements`.
2. **UA-intrinsic names mask missing labels.** An unlabeled `<input type=file>` reports AX name "Choose File", so the unnamed-control check stays quiet. Label association still needs axe/static/judgment review.
3. **Component-scale pages ≠ full pages.** Skip-link (2.4.1) and landmark findings assume a whole page; on component targets treat them as granularity artifacts.
4. **AA vs AAA honesty.** 2.4.13 focus-appearance findings are AAA-informative by design — never report them as 2.4.7 failures. The AAA pixel measurement is also rendering-environment-sensitive (macOS locally can emit AAA-informative findings that Linux CI does not, observed at both `97eb13e` and `0.5.0`) — one more reason never to gate on it.
5. **Emulated SR ≠ real AT.** Findings are spec-compliant-announcement evidence; the §6 manual NVDA/VoiceOver protocol still applies before shipping.
6. **`conformance_level` is the check's gate, not the SC's WCAG level** *(code-read at `0.5.0`, not fixture-measured — [upstream #27](https://github.com/ezufelt/keyboard-a11y-tester/issues/27), filed 2026-08-04)*. Only the 2.4.13 check emits `AAA`; every other finding falls through `level || 'AA'` to `"AA"`, mislabeling the nine Level A SCs the checks cover (1.1.1

…(truncated)