vrt
vlmkit is a TypeScript visual-regression toolkit built on Playwright +
pixelmatch. Beyond raw pixel diffs, it surfaces agent-friendly
signal: computed-style deltas split into universal vs.
breakpoint-gated, per-section diffRatio against component bboxes,
worst-viewport screenshot paths inline, and (optionally) VLM-emitted
CHANGE lists feeding an LLM CSS-fix step.
Source: https://github.com/mizchi/vlmkit. CHANGELOG +
old-CLI-to-new-CLI mapping: CHANGELOG.md.
When to invoke this skill
- An agent edited CSS / HTML / a component and you need "what visibly changed, and where."
- You're auditing a refactor PR for unintended layout drift.
- You're comparing two URLs (dev server vs. preview deploy; baseline vs. variant).
- You want a CI gate that flags regression across PRs.
- You're swapping a framework / CSS library and need to verify the rewrite is visually equivalent.
When NOT to invoke
- Snapshot testing of pure-data structures (use Jest snapshots).
- Lighthouse / Web Vitals beyond CLS/LCP/FCP (use Lighthouse directly).
- Browser-driver-only flows with no diff component (use Playwright Test).
- Reviewing AI-generated screenshots for content correctness (use a
vision-LLM tool like
review-image).
Install
Prerequisite: Node 24+. The CLI and the workspace packages all
rely on Node's --experimental-strip-types (default-on at 24+).
Node 22 will not work — verify with node --version before
installing. Use nvm install 24 && nvm use 24 (or fnm / volta /
your preferred manager) to upgrade.
Pre-flight check: run scripts/doctor.sh to verify Node version,
Playwright Chromium, optional API keys, and installed sub-skills in
one pass. Two invocation forms by setup phase:
- Pre-install (no apm yet): one-shot via curl —
bash <(curl -sSL https://raw.githubusercontent.com/mizchi/skills/main/vlmkit/scripts/doctor.sh) - Post-install (
apm install -g mizchi/skills/vlmkit/vlmkitdone): local copy —bash ~/.claude/skills/vlmkit/scripts/doctor.sh
Severity rules: only Node 24+ and Playwright Chromium are
FAIL-class (block exit 0). Everything else — including vrt CLI not
yet on PATH — is WARN by design, because the script is meant to be
runnable mid-install (a WARN on vlmkit CLI is expected the first time
through; resolve all FAILs first, install vrt, re-run, and the WARN
clears).
# CLI (global)
pnpm add -g @mizchi/vrt
# or per-project
pnpm add -D @mizchi/vrt
npx playwright install chromium
# Library packages (deep imports via .ts source)
pnpm add @mizchi/vlmkit-core @mizchi/vlmkit-capture @mizchi/vlmkit-markup @mizchi/vlmkit-ai
CLI cheatsheet (0.5.0 verb groups)
I/O conventions used below: --output <dir> always takes a
directory path (the engine writes diff-report.json + per-viewport
PNGs into it). vlmkit diff agent writes Markdown to stdout by
default; pass --out <path> to write to a file instead.
Commands taking two positional args (e.g. vlmkit diff html <baseline> <variant>) accept either two local file paths or two URLs; use
--url/--current-url if you want to be explicit.
# Diff
vlmkit diff html <baseline> <variant> --output reports/ # → reports/diff-report.json + PNGs (dir output)
vlmkit diff agent reports/diff-report.json [--out diff.md] # → stdout (default) or --out path
vlmkit diff png base.png current.png # Direct PNG diff (positional files)
vlmkit diff elements --selector .card … # Element-level shift isolation
vlmkit diff browsers <url> # Chromium / Firefox / WebKit parity
vlmkit diff runs <dirs...> # Aggregate N VRT runs
# Snapshot (baseline + diff lifecycle)
vlmkit snapshot <url1> [url2]... --output snapshots/
vlmkit snapshot approve # Promote current → baseline
vlmkit snapshot stability <url...> --iterations 5 # FP-rate measurement
vlmkit snapshot flipbook --output snapshots/ # Embed PNGs in a self-contained HTML
# Check (CI gates)
vlmkit check a11y contrast|touch|focus <html> # WCAG scans
vlmkit check tokens <html> # Design-token scale conformance
vlmkit check theme <html> # prefers-color-scheme parity
vlmkit check perf <html> # CLS / LCP / FCP
# Inspect / Stress / Scan / Build
vlmkit inspect interact|explore|smoke <html|url>
vlmkit stress i18n|media <html>
vlmkit scan component|breakpoints <…>
vlmkit build component <target.png> <current.html> # Markup-from-screenshot loop
# Migration (framework / CSS-library swap audit). Three modes:
# compare — deterministic side-by-side audit; the default. Start here.
# blind — variant agent never sees baseline pixels (forces convergence from spec text alone).
# subagent — dispatched subagent runs `compare` + writes a verdict.
# `--mask` and `--output` work identically across all three modes.
vlmkit migration compare|blind|subagent <baseline> <variant> --output reports/
# Long-running / stateful
vlmkit watch <baseline> <variant> # File-watcher inner loop
vlmkit manifest add|list|rm|check # Approval rules
vlmkit diff-pr pin|verify|post # PR CI gate
vlmkit baseline pin|verify|post|list|rm # Canonical alias of diff-pr
The single-token commands from 0.4.x (vrt compare, vrt png-diff,
vrt theme-parity, …) remain as deprecation shims that print a
one-line hint and forward.
Sub-skill routing
Two repos, by design: this orient skill lives in
mizchi/skills (general-purpose
skills); the five vrt-specific sub-skills live in
mizchi/vlmkit/vlmkit under
.claude/skills/. The two apm install paths below look different
because they target different repos — that is intentional, not a typo.
The CLI is fully functional without any sub-skill installed —
sub-skills are agent-facing reference material that deepens the routing
for a specific task shape. Install only when an agent needs the extra
context; end users running vrt from the command line never need
them.
Scope boundary: this orient skill stops at routing. Once you know
which sub-skill to load, deeper operational detail — full flag lists,
persistence paths (e.g. .vrt/last-diff-for-agent.json for regression
watch), per-mode semantics, output schema — lives in the corresponding
sub-skill. Install it via the apm command above when an agent needs
that depth.
apm install mizchi/vlmkit/.claude/skills/<skill-name>
Pick by task shape:
| Sub-skill | Use when | Entry workflow |
|---|---|---|
vrt-visual-diff |
One-shot "did this CSS edit change pixels, and where?" | vlmkit diff html → vlmkit diff agent |
vrt-regression-watch |
CI gate / scheduled drift detection across runs | vlmkit diff agent --previous --fail-on-regression |
vrt-migration-eval |
Framework / CSS-lib / build-system swap audit (deliberate large diff) | vlmkit migration compare|blind|subagent |
vrt-markup-synth |
Screenshot → HTML/CSS, token / theme / i18n / a11y audits | vlmkit build|scan|check|stress |
vrt-css-fix-loop |
Automated CSS-repair loop with VLM + LLM | fix-loop.ts (VRT_VLM_MODEL=…) |
Routing heuristic (ask yourself once the user states the task):
Is the markup deliberately different (rewrite / framework swap)?
├─ yes → vrt-migration-eval
└─ no
├─ Need to detect change between runs over time? → vrt-regression-watch
├─ Want a CSS-fix loop (auto-repair)? → vrt-css-fix-loop
├─ Building from a screenshot / token audit? → vrt-markup-synth
└─ One-shot "what changed" → vrt-visual-diff
Output anatomy
vlmkit diff html writes:
<output>/
├── diff-report.json # canonical machine input for diff agent
├── migration-report.json # legacy alias, byte-identical (will be removed)
├── diff-mobile.png # per-viewport pixel diff
├── diff-desktop.png
└── diff-wide.png
vlmkit diff agent <report.json> then emits Markdown structured as:
# VRT diff (for agent)
### Diff by viewport (worst first)
| Viewport | Diff | Dominant category | Categories | Shift bands |
| ...
### Verified deltas (computed-style) × viewport ← HOISTED to #2
#### Universal pairs ← fix the base rule
#### Breakpoint-gated pairs ← fix or add @media rule
### Per-section diffRatio
### Heuristic fix candidates
The hoisted Verified deltas is the load-bearing signal — read Universal pairs first, then Breakpoint-gated, then drill into Per-section.
Common gotchas
- Masking is load-bearing. Pass
--mask ".marquee,.timestamp, [data-testid='live-counter']"for anything that flaps run-to-run. An unexpected baseline-noise diff is almost always a missing mask, not a real regression. - Output is a directory, not a file.
--output reports/makesreports/diff-report.json+ PNGs. Feed the JSON path (not the dir) tovlmkit diff agent. - Playwright must be installed. First run errors with "browser not
found" →
npx playwright install chromium. - Self-comparing the same URL with no mask is the false-positive
baseline. Use this to verify
--maskactually catches every flapping element before relying on diff% threshold. - The dist binary is bundled.
dist/vlmkit.mjsships with all leaves code-split. There is no source dependency at install time.
Environment variables
| Variable | Purpose | Default |
|---|---|---|
VRT_LLM_PROVIDER |
LLM provider for fix-loop (gemini / openrouter / anthropic) |
gemini |
VRT_LLM_MODEL |
LLM model override | Provider default |
VRT_VLM_MODEL |
VLM model (OpenRouter id, or gemini:/claude: prefix) |
bytedance/ui-tars-1.5-7b |
OPENROUTER_API_KEY / GEMINI_API_KEY / ANTHROPIC_API_KEY |
Per-provider auth | — |
VRT_CAPTURE_BACKEND |
local (default Playwright) or cloudflare (Browser Run) |
local |
DEBUG_VRT |
Verbose log output | unset |
Library use (vrt-core / vrt-capture / vrt-markup / vrt-ai)
When the CLI surface isn't enough, deep-import the relevant package:
import { compareScreenshots } from "@mizchi/vlmkit-core/heatmap.ts";
import { discoverViewports } from "@mizchi/vlmkit-capture/viewport-discovery.ts";
import { extractComponents } from "@mizchi/vlmkit-markup/component/component-extract.ts";
import { askVlm } from "@mizchi/vlmkit-ai";
@mizchi/vlmkit-core has no Playwright dependency for the lightweight
surface (image / DOM / a11y primitives). The other three pull in
Playwright transitively.
Reporting issues / contributing
- Issues / feature requests: https://github.com/mizchi/vlmkit/issues
- Source: https://github.com/mizchi/vlmkit
- Smoke gate before PRs:
bash scripts/smoke-dist.sh(strict, 11 probes against the bundleddist/vlmkit.mjs).