Uxaudit

UX Regression Testing For Apps

A skill for running UX regression testing on a running app through two reader-facing buckets — UX issues and UI risk signals. The dashboard is meant to be read through those two top-level buckets first; each check inside the ux band is also tagged with a subtype (understand / decide / act / recover) so reviewers can drill into where in the user's flow the failure happens. Every judgment-call check is anchored in published, publicly-verifiable research (NN/g, WCAG 2.2, ISO 9241-11/110, Krug, Norman, Cooper, Christensen, peer-reviewed studies). Self-evident anti-patterns (lorem ipsum, "Coming soon" stubs) ship without an academic anchor because they aren't judgment calls. The full allow-list and the "don't invent" rule live in references/knowledge/sources.md. Architecture, subagent isolation, and the L1→L4 evaluation model live in references/concepts/architecture.md.

What this is

Most testing tells you whether code works. uxaudit tells you whether the result is usable and wants-to-be-used.

Two layers run side by side:

• Catalog — universal craft floor: AI slop fingerprints, WCAG, Nielsen heuristics, generic taglines. Project-agnostic, mostly stable.
• Scenario — whether THIS app's actual user journeys complete with continuity. Discovered from the project's own files; differs per project.

The catalog covers the floor of craft. The scenario layer covers journey continuity. Neither pretends to predict whether the product is the right product to build — that's a human question.

The dashboard presents findings in two reader-facing buckets:

• UX issues — problems that make the user struggle to understand, decide, or complete the task
• UI risk signals — templated, sloppy, or risky interface patterns that are likely to degrade the experience

Each check is also tagged with a category (derived from its directory layout — accessibility, ai-slop, core-experience, desirability, usability) and, when its band is ux, a subtype (understand / decide / act / recover). Reviewers should read the output through the two top-level buckets first; category and subtype are secondary axes the dashboard exposes for drill-down.

How the dashboard reads (band first, then UX subtype)

The reader-facing hierarchy is:

1. Band (top level): ux vs ui-risk
2. UX subtype (only inside the ux band): understand → decide → act → recover
3. Severity (only inside the ui-risk band): critical → major → minor

The four UX subtypes are the canonical questions a reviewer cares about:

An older rationale doc, references/concepts/ux-layers.md, frames the same territory using the Usability / Desirability / Retention vocabulary. That naming is kept only as conceptual prose — the check catalog itself has no layer field, only band + subtype. A "Retention" layer is explicitly out of scope: a single session cannot measure whether users come back.

Argument semantics

Frontmatter argument-hint lists the surface flags. Their meanings:

The iteration metadata records the actual scenario contract mode used:

• scenario_mode
• scenario_source_iteration
• comparable_to_previous

This is what lets History mode distinguish "same measuring stick, new fix attempt" from "the measuring stick itself changed."

Native macOS/iOS/Android apps and CLIs are out of scope: uxaudit is screenshot- and DOM-driven, so a target without a rendered DOM cannot be audited.

Alternate entry points the orchestrator may be asked to run directly (not slash invocations — these are post-audit views over an existing workspace):

python $UXAUDIT_DIR/scripts/generate_dashboard.py <workspace> --timeline
python $UXAUDIT_DIR/scripts/generate_dashboard.py <ws-a> <ws-b> --compare
python $UXAUDIT_DIR/scripts/generate_dashboard.py --library

$UXAUDIT_DIR is exported by orchestrator playbook Step 0 in plugin / --plugin-dir / standalone-symlink / dev-checkout modes — the resolution chain in references/playbook/00-env-setup.md walks $CLAUDE_SKILL_DIR → $CLAUDE_PLUGIN_ROOT/skills/uxaudit → $HOME/.claude/skills/uxaudit → common dev paths and verifies each candidate by looking for SKILL.md. The dashboard auto-detects mode (audit / history / compare / library) from inputs. See references/modes/library.md for the central registry, archiving, and .uxauditignore.

Viewport presets

Named presets that claim to be "mobile" (mobile-sm, mobile, tablet, tablet-pro) set isMobile: true in Playwright — without this the browser ignores <meta viewport> and lays out at 980 CSS px even inside a 390-wide context, so media queries like @media (max-width: 640px) silently never fire. Both presets and custom WxH go through this logic.

For any device that isn't in the preset list, use --viewport WIDTHxHEIGHT directly (e.g. --viewport 430x932 for iPhone Pro Max, --viewport 744x1133 for iPad Mini). Custom sizes use DSF=1 and a desktop UA — they're a Responsive-Mode equivalent, not a device simulator.

4K monitors are intentionally NOT a preset: modern OS scaling (macOS Retina, Windows 150 %+) renders 4K displays at ≤ 1920 × 1080 CSS px, so a physical-4K preset would duplicate desktop-lg.

Workflow (orchestrating agent)

The orchestrator follows the literal step-by-step in references/playbook/index.md (which routes to the per-phase files 00-env-setup.md … 06-aggregate-report.md). Read the index before running. The high-level shape:

Critical isolation rule: L3/L4 Judge dispatches in Step 11 are physically sandboxed via their agent frontmatter tool allowlists — uxaudit-l3-judge has Read, Write only, uxaudit-l4-judge has Read, Write, Glob only. They physically cannot read project-context.json, the spec, the compiled journey-scripts/, or any project file because they have no Bash, no WebFetch, no Grep. The L4 split (Step 9 Compiler + Step 10 capture + Step 11 evidence-only Judge) is what makes "pixel-only at judgment time" mechanically enforceable. The Reconciler in Step 14 is the only place where intent and observation meet. See references/concepts/architecture.md for the rationale and references/subagents/judge-output-format.md for the canonical Judge output contract (kept as a reference document for check authors — the new uxaudit-l3-judge / uxaudit-l4-judge agents embed the contract inline in their system prompts).

/uxaudit is autonomous from L1 through L4. Step 12 blocks the pipeline until every Judge has actually run; Step 13 blocks if any verdict is malformed.

L4's pixel-only guarantee at judgment time is enforced by the capture/judge split:

• Step 9 (Compiler) is the only role with selector context — it reads HTML and writes JSON action scripts. It commits no judgment.
• Step 10 (capture_journey.mjs) executes those scripts and writes screenshots + steps.json. It commits no judgment.
• Step 11 (L4 Judge) reads only the screenshots, the action trace, and the compressed evaluation brief. It must NOT open a browser, run curl, or read journey-scripts/. Verdicts come from what is visible in the PNGs, with steps.json used only to disambiguate ordering and to know whether a screenshot represents a successful step or a captured failure state.

Adding or modifying checks

Reading order when authoring a new check:

1. references/authoring/check-authoring.md — contract: when to add a check, the 10 manifest fields, classification (band/subtype) and method (L1–L4) choice, evidence-file policy, common mistakes, end-to-end checklist
2. schemas/check-manifest.schema.json — formal JSON Schema enforced by the pre-flight validator
3. references/authoring/adding-checks.md — L1 / L2 / L3 / L4 worked walkthroughs with the actual check_lib / _target.mjs API
4. scripts/validate_checks.py + scripts/test_checks.py — run after every edit

python scripts/validate_checks.py            # pre-flight: validate all check.json
python scripts/test_checks.py                # regression: L1 fixtures
python scripts/validate_results.py <iter>    # post-flight: every result.json

Three safeguards work together to prevent silent quality drift as the catalog grows:

For L3/L4 specifically, every Judge subagent must produce result.json conforming to references/subagents/judge-output-format.md (the canonical contract for check authors). The uxaudit-l3-judge and uxaudit-l4-judge agent definitions embed the contract inline in their system prompts — authors of new L3/L4 checks read the reference document to understand the contract, then write a check-specific prompt.md that only enumerates the finding-type tags and the per-axis rubric. Keep evidence assets minimal: L3 usually attaches zero or one supporting screenshot/crop when it materially helps the reviewer; L4 attaches ordered journey waypoints.

What this skill DOES NOT do

• Predict retention — only real users over time can show this
• Replace human judgment — checks surface signals; humans decide what matters
• Test correctness — that's the Evaluator's job (Playwright + assertions). This skill assumes the app works.
• Test market fit — the spec being built may be the wrong spec entirely. Human question.

See also

Runtime references — the orchestrator loads these on-demand during a run:

Subagent definitions — Claude Code loads these automatically when the orchestrator dispatches subagent_type="uxaudit-<role>". The orchestrator does NOT re-inject them into prompt bodies:

Subagent reference documents — read by check authors or for understanding contracts; NOT injected at dispatch time:

Contributor references — human-only documentation, NOT loaded at runtime. Do not cite from subagent briefs or playbook phase files:

Schemas + scripts — machine artifacts, not prose: