# Playtest > Drive a real browser against a game with `vg playtest`: smoke checks, scripted bot playtests, softlock detection, screenshots and visual diffs, on localhost or a deployed URL. - Skill: `kyh/playtest` (Agent Skill, multi-file: 5 files) - Install (CLI): `npx skillmds@latest add kyh/playtest` - Raw SKILL.md: https://api.skillmd.com/api/skills/kyh/playtest/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Coding & Dev Tools - Author: kyh (https://skillmd.com/u/kyh) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/kyh/playtest --- # Playtest A build that compiles proves nothing. A game that renders proves almost nothing. **Playtesting proves it plays** — that the loop runs, input moves the player, the objective is reachable, and nothing throws along the way. ## The Tool: `vg playtest` Everything here runs through one command. `vg playtest` is a passthrough to [agent-browser](https://github.com/vercel-labs/agent-browser), a native browser-automation CLI built for agents — `vg` installs it and its browser on first use, so **there is nothing to add to the game's `package.json`**. No test runner, no devDependencies, no MCP server to configure. ```sh vg playtest open http://localhost:5173 # launch + navigate (headless by default) vg playtest snapshot -i # interactive elements, with @eN refs vg playtest click @e2 # act on a ref from that snapshot vg playtest eval "window.__GAME_DIAGNOSTICS__" --json vg playtest screenshot /tmp/frame.png vg playtest close ``` A daemon persists between invocations, so one-call-per-step is cheap and the page stays open across commands. ### Load the upstream skill for the general command surface agent-browser ships its own skills and serves them from the installed binary, so the content always matches the version you actually have: ```sh vg playtest skills get core # the general browser-automation guide — read before driving anything vg playtest skills get core --full # + full command reference and templates vg playtest skills get dogfood # systematic exploratory testing / bug hunts vg playtest skills list # everything available on this version ``` **`skills get core` is the source of truth for the generic surface** — the snapshot-and-ref loop, sessions, waiting, forms, auth, troubleshooting. Don't re-derive it here, and don't trust a stale memory of it. An older agent-browser build with no `skills` subcommand falls back to `vg playtest --help`. This skill covers only what upstream can't know: the game diagnostics contract, the bot playtest, canvas/WebGL determinism, `--game`, and the traps we hit driving real games (see below). `references/cli-cheatsheet.md` is the game-shaped subset of commands. **Playtest a deployed game with no local setup at all:** ```sh vg playtest --game my-game # opens https://my-game.vibedgames.com vg playtest --game # reads the slug from ./vibedgames.json ``` That is the fastest possible loop after `vg deploy`: ship it, then play it. ## Start Here: The 60-Second Smoke Check Before anything clever, prove the game boots and doesn't throw: ```sh vg playtest batch \ "open http://localhost:5173" \ "wait --fn 'window.__GAME_TEST_HOOKS__ !== undefined'" \ "eval 'window.__GAME_TEST_HOOKS__.setState(\"active-play\")'" \ "wait --fn '(window.__GAME_DIAGNOSTICS__?.frame ?? 0) > 10'" \ "eval 'window.__GAME_DIAGNOSTICS__' --json" \ "screenshot /tmp/boot.png" vg playtest errors # uncaught exceptions — must be empty vg playtest console # console.error — treat as product failure ``` `batch` runs the whole sequence in one invocation and returns one result array. **Wait for the contract before waiting for frames.** A game sitting on its menu has a frame counter that never moves until `setState('active-play')` starts a run — wait on frames first and you deadlock on exactly the games that implement the contract correctly. If the _first_ wait times out, the game never published its diagnostics: it crashed on boot, or it doesn't implement the contract. If the _second_ times out, the hooks are no-ops. **Treat any console error or failed asset request as a failure** unless you can name why it's benign. ## The Diagnostics Contract The browser can see pixels; it can't see whether the player is stuck. Games expose two globals so a playtest can measure real state instead of guessing from screenshots: - `window.__GAME_DIAGNOSTICS__` — read-only per-frame telemetry (`frame`, `score`, `complete`, `player`, `entities`). - `window.__GAME_TEST_HOOKS__` — the mutations a playtest may perform (`seed`, `setState`, `setReducedMotion`, …). **The full field list and rules live in `references/bot-playtest.md`** — that is the address `games/lunerfall/src/sys/diag.ts` and `games/starfall/src/shared/diag.ts` both cite, so it stays the one copy to edit. Two rules worth knowing before you read it: JSON-serializable primitives only, never raw engine objects; and `seed(n)` must **restart** the run, not just reseed it, or everything measured afterwards is still unseeded. Adding this contract to a game is a prerequisite, not an optional extra. Without it a playtest can only assert "pixels changed". ## Bot Playtest: Prove It Plays A smoke check proves the game loads; a bot playtest proves it _plays_. It drives real held input and measures **progression**: ```bash # This skill's directory. Claude Code substitutes CLAUDE_SKILL_DIR (project, global # or plugin install); other agents fall back to wherever `skills add` put it. SKILL="${CLAUDE_SKILL_DIR}" [ -d "$SKILL" ] || for d in .agents/skills .claude/skills ~/.agents/skills ~/.claude/skills; do [ -d "$d/playtest" ] && SKILL=$d/playtest && break done ``` ```sh # from the project root node $SKILL/scripts/bot-playtest.mjs --url http://localhost:5173 --seed 12345 ``` The bundled script (zero dependencies — just Node and `vg`) drives a scripted sweep of held keys and pointer moves, samples diagnostics **during** each step, and prints a JSON report. It measures four things: the loop survived (`framesAdvanced`), input reaches the player (`distanceTravelled`), the objective is reachable (`scoreAfter` / `stepOfFirstScore`), and the player never wedged (`longestStuckRun`) — plus zero console and page errors. Exit `0` means it plays, `1` means it doesn't and the report names which check failed, `2` means the harness itself broke. Two defaults are deliberate, and both exist so a passing game passes: objective progression is a `warning` until you pass `--expect-progress`, and a game that steers with the mouse needs `pointer` steps rather than the WASD default. See [bot-playtest.md](references/bot-playtest.md). The pass thresholds live in `THRESHOLDS` at the top of the script; read them there rather than from prose that can drift. Adapt `--script` to the game's core verb: a runner holds forward and switches lanes, an arena game sweeps the space, a tower defense places towers through test hooks. Game-specific hooks (`forceWave()`) are encouraged where raw keys can't express the verb. Metric meanings, flags, difficulty/fairness runs, and the key-dispatch trap: `references/bot-playtest.md`. ## Canvas & WebGL Two things will mislead you if you don't know them. **Read the renderer string before you believe a frame rate**: headless can land on the real GPU (`vg playtest` does on a Mac — an `ANGLE Metal` renderer) or on SwiftShader (bare Playwright `launch()`); the bot report's `gpu` field says which. SwiftShader means functional-only evidence — it renders lit materials black and drops heavy games to ~3–4 fps, so a multiplayer host on it reads as frozen; a hardware renderer (ANGLE Metal/D3D/Vulkan on a real device) is a desktop-GPU signal, still not a phone. A headed or Playwright-launched run needs explicit flags to land on the real GPU (`references/canvas-determinism.md` § Headless Footguns). And **headless can't capture WebGPU canvases on Linux/Windows** (the screenshot comes out black even though rendering worked) — a `--headed` problem with real consequences for what you report. Determinism setup, readiness signals, flake classification, and the full footgun list: `references/canvas-determinism.md`. ## Screenshots and Visual Diffs Visual comparison is built in — there is no separate image-diff tool to install: ```sh vg playtest screenshot /tmp/baseline.png # … make a change, reload … vg playtest diff screenshot --baseline /tmp/baseline.png -o /tmp/diff.png vg playtest diff snapshot --baseline /tmp/before.txt # accessibility-tree diff, for DOM/HUD ``` A baseline taken without freezing the scene first is a flake generator — the freeze checklist and the "when not to take a baseline at all" judgment are in `references/canvas-determinism.md`. A diff catches a change; it can't tell you the frame was wrong to begin with. For 3D, pair the screenshots with the visual-defect rubric in the `threejs` skill (`references/debugging-and-profiling.md` § Visual defects) — z-fighting, shadow acne, wrong color space, and DPR blur all render "successfully" and pass every smoke check. **Metrics and eyes, both.** Appearance work reports numbers _and_ same-camera before/after pairs; a number that improves while the frame still looks wrong is a failure. A metric is worth most on the first problem and less on every one after — alternate a measured pass with a by-eye pass that carries no number, and ask "would you show this to someone?". A harness check can be true and useless: it covers what it was written for, not the frame. ## Anti-Patterns ❌ **Sleep-driven steps** — `wait 2000` then click ✅ `wait --fn "window.__GAME_DIAGNOSTICS__?.frame > 10"` ❌ **Screenshot-only verdicts** — "it looks right" ✅ Read `__GAME_DIAGNOSTICS__`; a screenshot is evidence, not an assertion ❌ **Asserting exact pixel positions** without fixed dt and seeded RNG ✅ Assert user-meaningful invariants: score rose, player moved, fail state triggered ❌ **Stale refs** — `@e2` from a snapshot two navigations ago ✅ Re-`snapshot` after anything that changes the page; refs are per-snapshot ❌ **`keydown`/`keyup` for game input** — agent-browser 0.34 sends them with an empty `code` and `keyCode: 0`, so Phaser and friends ignore them entirely ✅ Dispatch the event via `eval` (what the bot script does), and always release what you hold — see `references/bot-playtest.md` ❌ **`set device "iPhone 15"` as a phone check** — viewport + UA only; `pointer: coarse` stays false, so it's the desktop build ✅ Held CDP touch emulation, and assert `coarse: true` before believing anything — `references/cli-cheatsheet.md` § Environment ❌ **Calling a game verified because it rendered** ✅ Run the bot; renders ≠ plays ## When You're Done - [ ] Smoke check passes: boots, reaches a live frame, zero console/page errors - [ ] Diagnostics contract exposed and honest (no silent no-op hooks) - [ ] Bot playtest moves, scores under `--script --expect-progress` with the game's core verb, and reports `longestStuckRun` ≤ 2 - [ ] Fail state triggers and retry restores play (for games that can be lost) - [ ] Deployed build playtested with `vg playtest --game `, not just localhost - [ ] Evidence is fresh: a new run id / output dir whenever code or assets changed — never relabel an old report as current evidence - [ ] Expected viewport/state pairs declared before capturing; a required capture that failed is listed as failed, not dropped ## Bundled Resources - `scripts/bot-playtest.mjs` — the progression-measuring bot; run it, read the JSON report - `references/bot-playtest.md` — diagnostics contract, metrics, difficulty/fairness runs - `references/canvas-determinism.md` — deterministic mode, readiness, flake triage, Phaser/Three.js specifics - `references/cli-cheatsheet.md` — the game-shaped subset of the `vg playtest` command surface For anything generic — the snapshot/ref loop, sessions, auth, waiting, troubleshooting — read `vg playtest skills get core` rather than these files. ## Remember Any game becomes testable by adding one small, stable seam for readiness and state. The goal is not coverage — it's the ability to answer "does this actually play?" in under a minute, every time, without the human opening a browser.