# Web Testing Visual Regression
> Visual regression testing — screenshot baselines, determinism, diff review, baseline custody and CI wiring. Load when adding, reviewing or debugging tests that compare a rendered subject against an approved image.
- Skill: `agents-inc/web-testing-visual-regression` (Agent Skill, multi-file: 7 files)
- Install (CLI): `npx skillmds@latest add agents-inc/web-testing-visual-regression`
- Raw SKILL.md: https://api.skillmd.com/api/skills/agents-inc/web-testing-visual-regression/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: agents-inc (https://skillmd.com/u/agents-inc)
- Updated: 2026-09-10
- Page: https://skillmd.com/skills/agents-inc/web-testing-visual-regression
---
# Visual Regression Testing
> **Quick Guide:** A visual test asserts that a subject still looks the way a human approved it. The comparison is trivial; everything hard surrounds it — making the render deterministic, scoping the capture to the subject, generating baselines in the environment that will diff them, and looking at every diff before accepting a new one. A baseline accepted without being looked at turns the suite into a machine that asserts the bug.
**Detailed Resources:**
- [examples/core.md](examples/core.md) — comparison configuration, the baseline matrix, masking, subject scoping
- [examples/determinism.md](examples/determinism.md) — the stabilizing stylesheet, fonts, frozen clocks, seeded randomness, fixture data, environment parity
- [examples/catalog-driven.md](examples/catalog-driven.md) — a component-state catalog as the visual corpus, modes, interaction-produced states, change detection
- [examples/ci.md](examples/ci.md) — pipeline wiring, diff artifacts, the baseline-refresh workflow, cost control
- [reference.md](reference.md) — comparison-option semantics, update modes, the pre-commit checklist, troubleshooting table
---
## Which path applies
The two families differ in where the baseline lives, who accepts a change, and who owns the render environment. Pick before writing anything; retrofitting is a migration rather than a flag.
- **Baselines committed to the repository, compared by a harness you run** — review happens in code
review, and the render environment is yours to pin. Free per run, expensive in environment work.
[examples/core.md](examples/core.md) and [examples/determinism.md](examples/determinism.md).
- **Snapshots uploaded to a change-detection service** — review happens in a hosted UI with a
recorded accept per change, and one controlled renderer removes dev-vs-CI drift. Billed per
snapshot, near-zero environment work. [examples/catalog-driven.md](examples/catalog-driven.md).
Running both over the _same_ subjects is the one combination to avoid — two baselines, two review flows, two chances to accept the wrong one. Splitting them by subject (a state catalog in the service, a handful of critical full-page checks committed) is common and works.
---
## Before writing a visual test
**Look at the diff before accepting any baseline.** The approved image is the entire assertion, so accepting one unseen permanently encodes whatever was on screen, regression included, and the suite can never report it again.
**Make the render deterministic before capturing** — fonts loaded, motion stopped, clock frozen, randomness seeded, data fixed. A pixel that varies between runs is being asserted as though it were part of the design.
**Generate and compare baselines in the same environment.** Font rasterization and subpixel rendering differ between operating systems and container images, so a baseline made anywhere else shows a full-frame difference on every subject from the first run.
**Capture the subject, not the page around it.** An element or a clipped band fails only when it changes, which is what makes the failure name its own cause.
**Mask a region that legitimately varies rather than widening the tolerance.** A tolerance wide enough to absorb a live timestamp is wide enough to absorb a collapsed column.
---
**Auto-detection:** visual regression, visual testing, screenshot testing, screenshot baseline, baseline image, golden image, pixel diff, image diff, expected/actual/diff, maxDiffPixels, maxDiffPixelRatio, diffThreshold, update snapshots, accept baseline, snapshot review, UI diff, stabilizing stylesheet
**Applies to:**
- Catching unintended visual change in components, pages and design-system primitives
- Guarding a shared component library where one CSS change fans out across consumers
- Covering theme and viewport matrices without duplicating a test per cell
- Guarding what no assertion expresses — spacing, overflow, z-order, focus rings, truncation
- Making a render deterministic enough to be worth comparing
- Deciding what a diff means, and who is allowed to accept it
- Wiring the visual job so its failures are actionable and its cost is bounded
**Handled elsewhere:**
- Authoring the component-state catalog a visual corpus is built from — this skill decides which
states are worth capturing, not how a state is declared
- Driving a browser to reach a state — navigation, locators, network interception; the capture
happens once the state is on screen
- Asserting text, roles, values or behaviour — those fail with a sentence, where a pixel diff fails
with homework, so reach for them whenever they can express the requirement
- Pipeline syntax, runners, caching and secrets — this skill states what the visual job must do,
and the provider decides how it is spelled
---
## Philosophy
Every other kind of test states its intent in code: `expect(total).toBe(42)` says what correct means. A visual test states its intent in a **file** — an approved image somebody looked at once. Three consequences follow.
**The baseline is a review artifact, not a build artifact.** It records the judgement of a human who decided the UI was right. Regenerating it without looking destroys the only thing the test knows.
**Nondeterminism is a false requirement, not flake.** A pixel that varies between runs is being asserted as part of the design. Pin it or mask it; never widen the tolerance until it stops complaining.
**The diff is the output.** A suite that fails without producing a viewable expected/actual/diff triplet is unactionable, and unactionable suites get disabled.
**What visual tests are good at** is whole-appearance properties no reasonable assertion expresses — a shadow that vanished, a 3px shift that broke alignment, a font that failed to load, a dark-theme token that resolved to white-on-white, a container that stopped clipping overflow.
**What they are bad at is anything you could name.** If `toBeVisible()`, a text assertion or an accessibility-tree assertion would express it, write that instead.
---
> **The snippets below are portable shapes, not runnable code.** `matchScreenshot(subject, name, options)` stands for your harness's screenshot assertion, and `freezeClockAt`, `goto`, `byTestId`, `byRole` and the bare `page` stand for its clock, navigation and locator controls. Platform APIs such as `document.fonts.ready` are named as themselves. The option names are the ones the common pixel comparators share; confirm every spelling against your harness's own documentation. [examples/determinism.md](examples/determinism.md) works the same patterns through fuller call shapes, which are closer to what you will actually type.
## Core patterns
### Pattern 1: Configuration-Driven Comparison
Tolerances, path layout and the baseline matrix belong in configuration. Per-assertion options are for what genuinely varies by subject — the mask list, the clip box.
```typescript
// visual.config.ts
const PIXEL_THRESHOLD = 0.2; // per-pixel colour distance, 0-1
const MAX_DIFF_PIXEL_RATIO = 0.01; // share of the image allowed to differ
export default {
screenshots: {
threshold: PIXEL_THRESHOLD,
maxDiffPixelRatio: MAX_DIFF_PIXEL_RATIO,
stylePath: "./visual/stabilize.css", // injected into every capture
},
};
```
The two knobs answer different questions and are not interchangeable. `threshold` decides whether a _single pixel_ counts as different, which is what anti-aliasing needs. `maxDiffPixels` / `maxDiffPixelRatio` decide how many differing pixels are tolerated, and nothing needs those above about 1% except a subject that should have been masked.
Full code: [examples/core.md](examples/core.md)
### Pattern 2: The Matrix Comes From Configuration
One test, N approved images. The configuration cell's name enters the baseline path, so each cell is approved independently.
```typescript
projects: [
{ name: "desktop-light", use: { viewport: DESKTOP, colorScheme: "light" } },
{ name: "desktop-dark", use: { viewport: DESKTOP, colorScheme: "dark" } },
{ name: "mobile-light", use: { viewport: MOBILE, colorScheme: "light" } },
{
name: "second-browser",
use: { browser: "firefox" },
ignoreSnapshots: true,
},
];
```
Duplicating a test per theme means every subject is written twice and the one nobody remembers to add silently stops covering anything. `ignoreSnapshots` keeps a browser in the functional run without a second baseline set to maintain.
Full code: [examples/core.md](examples/core.md)
### Pattern 3: Determinism Before Assertion
Six sources of nondeterminism account for nearly every flaky visual test, and each has a fix cheaper than the flake.
| Source | Symptom in the diff | Fix |
| -------------------- | ----------------------------------------------- | ---------------------------------------------------------------- |
| Web fonts | Whole text blocks shift; fallback metrics | Self-host and preload, then await `document.fonts.ready` |
| Animation/transition | Random intermediate frames | Disable CSS motion at capture; stop JS-driven motion too |
| Time | "2 minutes ago", clocks, date-stamped rows | Freeze the clock before navigating — freeze, do not fast-forward |
| Randomness | Shuffled lists, random ids, placeholder avatars | Seed the generator before any application script runs |
| Data | Row order and counts vary per run | Serve fixed fixtures with stable ordering and identifiers |
| Scrollbars / DPR | ~15px width shift, blurry text at 2× | Hide scrollbars in the stabilizing stylesheet; pin scale |
```typescript
await freezeClockAt(new Date("2026-01-15T12:00:00Z")); // before navigation
await goto(DASHBOARD_URL);
await page.evaluate(() => document.fonts.ready); // a real signal, not a guessed delay
await matchScreenshot(summaryRegion, "summary.png");
```
**The seventh source is the machine.** Baselines generated on a laptop and diffed in CI produce a permanent full-frame difference on every subject, which trains the team to regenerate blindly, which destroys the suite.
Full code: [examples/determinism.md](examples/determinism.md)
### Pattern 4: Scope the Capture to the Subject
| Subject | Capture |
| ----------------------------------------- | ------------------------------ |
| A component or one of its states | Element capture |
| A band or region spanning components | Page capture with `clip` |
| Whole-page composition (landing, invoice) | Page capture with `fullPage` |
| Anything above the fold | Page capture, default viewport |
```typescript
await matchScreenshot(
page.getByRole("region", { name: /pro plan/i }),
"pricing-card-pro.png",
);
await matchScreenshot(page, "header-band.png", {
clip: { x: 0, y: 0, width: 1280, height: 96 },
});
```
A full-page capture used to prove a button changed fails on every unrelated header edit — and fails every _other_ full-page baseline at the same time, so the reviewer sees dozens of red subjects, no signal, and the fastest way out is a blanket regenerate.
Full code: [examples/core.md](examples/core.md)
### Pattern 5: Mask What Moves
```typescript
await matchScreenshot(page, "dashboard.png", {
mask: [byTestId("last-synced-at"), byTestId("live-visitor-count")],
maskColor: "#000000", // defaults are often magenta, which some UIs legitimately contain
});
```
The mask list doubles as documentation of exactly which parts of the frame are not covered. Raising `maxDiffPixelRatio` until the timestamp stops failing buys enough slack to hide a missing sidebar, and says nothing about where the slack went.
Where a volatile region has no stable handle to mask, hide it in the stabilizing stylesheet instead — that needs only a selector.
Full code: [examples/core.md](examples/core.md)
### Pattern 6: Baseline Lifecycle
Three legitimate events — **create** (a new subject appears), **accept** (an intended change is reviewed and approved), **retire** (the subject is deleted and its images go with it). Anything else is drift.
```bash
# WRONG - the "visual tests are failing" reflex
npm run test:visual -- --update-all
# RIGHT - look first
npm run test:visual # emits expected/actual/diff per failure
npm run test:visual:report # inspect every diff
npm run test:visual -- --update-changed # only after each one is understood
```
Updating only _mismatched_ baselines is not the same as updating everything: the blanket mode rewrites matching baselines too, silently resetting subjects nobody inspected.
**The update travels with the change.** New images belong in the same pull request as the code that changed the appearance. A separate "update baselines" commit is unreviewable — nobody can tell an intended redesign from a regression once the two are in different diffs.
**Ownership is explicit.** The author who changed the UI _proposes_ baselines and says in the description what should have changed; someone else _accepts_. Author self-acceptance removes the only human judgement in the loop.
Full code: [examples/ci.md](examples/ci.md)
### Pattern 7: CI Wiring and Cost Control
Visual checks run on pull requests and on every trunk build — trunk builds are what keep baselines valid through merges. They skip documentation-only and config-only changes, and they never run in update mode in an automated job.
```yaml
# Pipeline definition; the syntax varies by provider, the shape does not
- name: Visual tests
run: npm run test:visual
- name: Publish diffs
if: always-except-success-only # must also fire on timeout and cancellation
# upload the report and results directories as a build artifact
```
A job whose only output is "1 failed" forces a local reproduction of a container-specific render, so the reviewer regenerates instead — which is the failure this whole skill exists to prevent.
**Treat the matrix as a budget.** Every new cell multiplies the whole corpus: its capture time, its review surface, and its bill. Add one when a real class of bug lives there, and delete one that has never caught anything.
Full code: [examples/ci.md](examples/ci.md)
---
## Red flags
**Breaks at runtime:**
- A blanket baseline update run to clear a red build — from that commit the suite asserts whatever was on screen, including the regression, and will never report it again
- Committing baselines rendered on a developer machine — every subject shows a full-frame diff against CI, the suite is declared broken, and it gets disabled
- Capturing while the page is still settling — in-flight requests, entrance animations, unloaded fonts — which produces intermittent diffs that get "fixed" by regenerating
- An automated job that runs in update mode and commits the result — green by construction, asserting nothing
- Naming baselines after test order (`step-3.png`) — reordering silently reassigns approved images to different subjects
- Renaming a configuration cell — its name is part of the baseline path, so every image under it is orphaned and the next run happily creates fresh "baselines"
**Surprising behaviour:**
- Comparators self-stabilize by re-capturing until two consecutive frames match, then diffing the last — that removes render jitter, not application nondeterminism
- The screenshot assertion usually disables CSS animation and hides the caret already, so most per-call option noise is restating defaults
- Masked regions are filled with a default colour a UI can legitimately contain, which makes the mask invisible in the diff
- `clip` and `fullPage` exist on the page-level capture only; for an element, capture the element rather than computing its box
- Tolerance numbers do not port between harnesses — different algorithms on different scales, so a value copied across means nothing
- Cloud modes are keyed by **name**: changing a mode's viewport under the same name keeps comparing against the old baseline, while renaming one starts a fresh baseline with no history
- Modes and matrix cells **stack** across project, component and subject level rather than overriding, so a catalog-wide default multiplies everything beneath it
- Change-based targeting depends on intact git history and a lockfile in sync with the manifest; rebases, squashes and force pushes fall back to full rebuilds
- Loosening tolerance until the noise stops trades a named uncovered region for an unnamed one
- Masking so much of a frame that the remaining pixels prove nothing — at that point delete the test
- Visual assertions mixed into functional test files mean one flaky image blocks feedback on unrelated behaviour
- WebP baselines cut repository growth substantially against PNG once the corpus is large