Preview Spec

Preview Spec

Attach an executable UI verification spec to a pull request, then run it against the live preview deployment.

A reviewer verifies a UI change by clicking through the preview. preview-spec turns that click-through into an artifact an agent can follow: a short spec in the PR description, run against the deployed preview by aw-tester, reporting pass or fail.

This SKILL.md is a thin index. Detailed procedures live in rules/*.md and templates/*.md. Each operation loads only what it needs.

What this skill reuses

This skill owns three things and reuses the rest.

Operations

Parse $ARGUMENTS. The first token selects the operation.

If no operation token is present, default to author when a diff or branch context is in scope, and run when only a PR reference is given.

Drivers

run executes the spec through one of two runners — same grammar, same verdict, different engine. Pick with --driver:

author never touches a browser and takes no --driver.

Step 0: Resolve your GitHub access path

Both operations touch GitHub. Resolve which path you have — gh CLI, mcp__github__* tools, or neither — per agents/shared/rules/github-access.md. Resolve once, state the path, and use it for the whole run. The commands below are the gh-path form.

On the mcp path, use these equivalents. Naming them here is load-bearing: the gh-path form above is not a mapping, and a reader who has to invent one writes nothing to the PR.

author works on both paths; run's URL resolution does not. rules/preview-url-resolution.md reads the GitHub deployments API, and no mcp__github__* tool exposes deployments. So on the mcp path, run must take an explicit --url <preview-url> argument. Without one, report inconclusive: no access path for deployment lookup (pass --url) and stop — never report inconclusive: preview not deployed, which claims a fact about the deployment that was never checked.

This paragraph is a summary; the branch is enforced in rules/preview-url-resolution.md § The access-path precondition, which owns the resolution decision and which rules/runner.md § Step 2 treats as terminal. It has to live there because its condition is run invoked without --url — an argument this step cannot see.

Operation author

Inject one collapsed, marked UI verification spec into the PR body.

1. Read memory first. Load spec-authoring lessons and locator lessons per rules/memory.md § Read at author time. These tell you the app's navigation quirks and stable locators before you write a single step.
2. Reuse an existing spec source when present. Before writing anything, check for a spec artifact the surrounding flow already produced, in priority order (full contract: rules/spec-sources.md):
   • .agent/{branch}/specs.md — the autonomous-workflow planner's aw-tester specs, already run locally at Phase 4. Same grammar: lift its ## Spec N: blocks verbatim.
   • A /fix-bug reproduction artifact for a UI or visual bug — an e2e-testing flow or a repro/<id>.md checklist. Adapt its steps into the grammar. Both sources are gitignored, local-only files. This works because author runs in the same worktree that wrote them, and it copies their content into the committed PR body — the durable artifact run later reads. The gitignored file is never committed; only its lifted content reaches GitHub. See rules/spec-sources.md § Two artifacts, two lifetimes. When a source is found, seed the block from it and skip step 3, so the PR block matches what was verified locally rather than a second, divergent description of the same behavior.
3. Otherwise, write the spec from the diff. Read the diff (git diff <base>...HEAD --name-status plus the relevant files), then write one ## Spec N: block per user-visible behavior the diff changes, in aw-tester's grammar. Prefer role-and-name locators; use {testid: …} only as an escape hatch. Keep it to the behaviors a reviewer would actually click through — 1 to 3 specs, not an exhaustive suite.
4. Wrap and inject the spec in the marked collapsed block per rules/spec-format.md, and write it into the PR body with the body-write call for your resolved access path, preserving everything already there. Writing the block into the PR body is this operation's only deliverable, so a run that could not perform that write has not authored a spec. Report it as failed (no GitHub access path) rather than reporting the specs you drafted — a drafted spec that never reached the PR is indistinguishable from none to every later reader, including run.

The block is exempt from the create-pr description length ceiling and is preserved verbatim by review-loop's body refresh — both rules live in rules/spec-format.md and in the description contract.

Report: how many specs were authored, and the one-line goal of each.

Operation run

Run the embedded spec against the live preview.

Full procedure: rules/runner.md. In outline:

1. Get the spec. Extract it from the PR body between the <!-- preview-spec:v1 --> markers — the committed PR body is the only source that works on any checkout and in any later session. As a shortcut for a local author→run loop, run <specs-path> reads a local specs.md directly (no PR, no extraction). Absent → report no spec and stop.
2. Resolve the preview URL per rules/preview-url-resolution.md. A --url <preview-url> argument overrides resolution (required with a local specs-path, and required on the mcp path). Any inconclusive: … outcome from that file is terminal — report it and stop, without a pass or a fail. Its two commonest are inconclusive: no access path for deployment lookup (pass --url) (no lookup was possible) and inconclusive: preview not deployed (the lookup ran and found nothing).
3. Materialize an ephemeral specs.md and an aw-target.yml overlay (base_url = resolved URL) under .agent/{branch}/.preview-spec/, reading auth and fixtures from a committed .claude/aw-targets/preview.yml when one exists.
4. Select the driver and run per --driver (see Drivers and rules/runner.md § Step 4). auto invokes aw-tester-chrome in-session when the Chrome extension is connected; when Chrome is unavailable or a Chrome run returns fallback: playwright, it asks the user before running the aw-tester sub-agent rather than falling back silently. A forced --driver chrome/playwright never prompts. Mode --all.
5. Report the verdict (pass / fail / inconclusive, per spec) — identical shape from either driver.
6. Write lessons per rules/memory.md § Write at run time when a spec failed for a navigation or precondition reason — not for a locator miss, which is the runner's own lesson to write.

Hard rules

• Never fork the spec grammar. It is aw-tester's single source of truth. If a step cannot be expressed in it, say so — do not invent syntax.
• Never weaken a spec to make it pass. A red verdict is a finding, not a failure of this skill.
• Never store a secret in the spec, the target file, or a lesson. Preview-auth credentials live in the committed preview.yml's refresh command or in the environment, never in the PR body — the spec is public.
• The runner reports; it does not fix. Applying a fix for a failing spec is the author's job (a better spec) or the PR author's (a code change).