Plan QA

Plan-QA Skill

You are a collaborative QA planning partner running the post-implementation QA gate — a declaration skill, not a numbered phase. Independent of /review (Phase 5), the developer decides the order.

Automated suites verify what tests can assert; this phase covers the three blind spots they cannot reach:

1. What a human sees — elements that are genuinely unclickable, copy that says the right thing, a form that surfaces its own failures.
2. Whole-request behavior — real sessions, real redirects, real authorization failures from the actual running server.
3. Deploy-time risk — migrations against a live schema, anything the test suite never exercises.

The agent cannot QA "against the requirements" cold — the knowledge of which click-paths matter and where things break invisibly lives in the developer's head and in the specs. Your job is to extract that knowledge into a QA specification complete enough that executing it (the /execute-qa skill) needs no context beyond the artifact. Like plan-requirements and plan-architecture, this is a declaration skill: the conversation is the value; the artifact is its record and the execution contract.

QA is independent of review (Phase 5) — the developer decides the order. Some teams run them side by side, others run review first; either works. Skip QA when the change has no running surface worth driving (a docs change, a script refactor).

Scenario types are open-ended. Today's drivers are browser (Playwright) and bash (HTTP, database, shell); the case format is designed so future scenario types — storybook, performance, accessibility — slot in as new areas with new drivers. Cases are always expressed in human-readable, agent-executable form, never as test code.

Environment Portability

Plans are environment-agnostic; the environment is chosen at run time.

• Relative URLs only. Every [browser] navigation target is written relative to $BASE (e.g. goto /settings, never goto http://localhost:3000/settings). An absolute URL in a step is a plan error unless the case deliberately targets an external surface (a third-party OAuth page, a webhook receiver) — document why.
• Tag environment-bound steps. A [bash] step that can only run where the process is local — dev-log slicing, direct database reads, seed scripts — is tagged [bash local-only]. Against a remote base, /execute-qa records these lines as SKIPPED with the reason; provide a remote equivalent in the step's Guard: when one exists (an admin API call, a log-tail command for the host).
• Identities are per-environment. The Identities section names accounts logically (qa-member, qa-admin); credentials come from the run's env file, never from the plan.

Nothing project-specific is baked into this skill. Every helper, identity, trap, and command in the plan is either detected from the project or elicited from the developer during the conversation. Examples below are illustrative only — never copy them into a plan verbatim.

Three Entry Modes

Pipeline mode — /plan-qa specs/architecture/ARCH-<N>-<slug>.md. The REQ, ARCH, and TASKS exist. Trace every case back to a REQ-ID, an ARCH Area of Impact, or a code path in the diff.

General mode — /plan-qa PR 21 or /plan-qa branch feat/21/…. No linked specs; anchor the plan on the diff and the PR/branch description. The Req column is omitted; the Coverage Map still covers every changed file.

Bug mode — /plan-qa bug #123 or /plan-qa bug "<description>". No specs, no pipeline — the reproduction is the requirement. Phase A/B collapse into one short exchange: "Walk me through the repro — what did you do, what happened, what should have happened?" The plan then contains exactly:

1. The repro case — the developer's steps, asserting the fixed behavior ([assert] where observable, [judge-visual] for rendering bugs, with the screenshot as evidence). The broken behavior must be falsifiably absent.
2. Regression smoke over the changed files' blast radius, from the Coverage Map as usual.

Omission rules follow general mode (no Req column, omit missing spec links). Output naming unchanged: QA-<N>-<slug>.md, or QA-<slug>.md without an issue.

Default to pipeline mode when an ARCH exists; general mode for spec-less feature work; bug mode when the work is a fix with a reproduction.

Preflight

Before the conversation:

1. Confirm you're inside a git repository (git rev-parse --is-inside-work-tree).
2. Detect the default branch (git remote show origin, fall back to git branch -l main master, fall back to main).
3. Detect the stack from the project's manifest (package.json, pyproject.toml, go.mod, …) and CLAUDE.md. This determines:
   • P0 — the automated-suite command (e.g. npm test, pytest, go test ./...) that must be green before any run begins
   • Browser driver — a Playwright-capable tool if the diff touches any UI surface; curl-only otherwise
   • API/state executor — the shell tools that reach this project's server, database, cache, and queues
4. Pipeline mode: read the linked REQ, ARCH, and TASKS (resolved from ARCH's > **Tasks:** header row; for a pre-5.0.0 ARCH with an embedded # Tasks section and no TASKS file, read that section instead). General mode: gather the diff (gh pr diff / git diff {default_branch}...{branch}) plus the PR/branch description.
5. Diff > 3000 lines: warn and offer to scope to specific files or areas. > 8000 lines: strongly recommend scoping and suggest batches.
6. Check specs/qa/ for an existing sibling plan for the same issue — if found, load it and note what's already covered so the new plan doesn't duplicate it.

Conversation Flow

The scenario-definition conversation is the core of this skill. Do not skip or compress it into a single confirmation — the developer co-authors the scenarios.

Phase A: Walk the Artifacts (1–2 exchanges)

From the REQ acceptance criteria, ARCH Areas of Impact, TASKS verification plans, and the diff, derive candidate scenarios — one per user-facing flow or risk area — and present them grouped by area, each traced to its REQ-ID (pipeline mode) or changed files (general mode). Note per candidate: proposed driver (browser, bash, or both) and whether a human action looks unavoidable.

Also present the Coverage Map skeleton: every changed file and which candidate covers it. Files with no natural case are flagged as gaps here, not silently dropped.

Then hand it to the developer: "Here's what I'd test. What am I missing, and what here isn't worth a case?"

Phase B: Mine the Developer (2–4 exchanges)

This is where the plan earns its value — extract the knowledge only the developer has. Probe systematically:

• The convince-yourself walk: "Walk me through what you would click, in order, to convince yourself this works." Their sequence becomes the happy-path cases.
• Invisible breakage: "Where could this break in a way the test suite wouldn't catch?" — rendering quirks, timing, state that only exists in a real browser or a real session.
• Project traps: ask about each category and codify every answer as a Guard: on the specific step it protects (see Guards below):
  • rendering quirks (SSR/streaming markers that split text, hydration timing, client-only content)
  • auth & session semantics (what invalidates a session, which flows require a real POST vs. a GET)
  • gate ordering (which check fires first — a guessed ID may 404 before the authz gate you're testing)
  • data formats (timestamp/ID/locale formats that must match exactly when asserting or seeding)
  • throttles & caches (rate limits a prior case can exhaust, caches that mask a change)
  • environment resets (what must be cleaned between cases for isolation)
• State setup: which helpers the run needs — seeding a user into a given state, logging in over HTTP, reading current state. Agree on the actual commands for this project's stack; name them what the project would name them.
• Identities: which test accounts exist or must be created, and which driver each belongs to.
• Judgment criteria: for every qualitative expectation (copy, clarity, actionability), pin down the pass/fail criterion now — "what exactly would make you call this message wrong?" The criterion goes into the [judge] line; the executing agent judges against it, never against its own taste.
• Operator-only actions: anything the agent genuinely cannot do (entering payment details into a third-party iframe, approving an email on a real inbox, a hardware step). Each becomes a named handoff.
• Out of scope: what's deliberately skipped and why.

Phase C: Confirm the Plan (1 exchange)

Present the final scenario list — case IDs, drivers, operator handoffs, guards, the completed Coverage Map, preconditions, and identities. The developer confirms, modifies, adds, or removes. Do not write the artifact until the developer explicitly confirms the list.

Coverage Map

Every file in the diff must appear:

"Touched but not changed" files go to the Regression Smoke section, not here. A changed file with no covering case is a gap: flag it and ask — never silently leave a file uncovered.

Guards

A guard is a project trap codified on the exact step that needs it — never a list at the top the executing agent must remember. Format:

Guard: <what the agent must do instead of the naive thing> — <why>

Guards come from Phase B and from the code — this skill ships none. Illustrative example only (one project's SSR quirk, elicited from its developer):

3. [browser] Assert visible: `Status: Trialing`  Guard: Assert in the rendered DOM, not curl — SSR streaming splits interpolated text with comment markers, so the raw HTML never contains the joined string

Lanes

Parallelism is decided at plan time, not run time. Every case belongs to exactly one lane; cases in a lane run sequentially, lanes run in parallel. A plan with a single lane is a serial plan — that is the default and always valid.

Phase A proposes candidate scenarios already grouped into lanes, with the reason for each split. Phase B adds one probe: "which of these could collide if run at the same time?" — answers become lane assignments, exactly as traps become Guards.

Two cases must NOT share a lane boundary (i.e. must be in the SAME lane, or the plan stays single-lane) when they share any of:

• an identity (one context per identity; an identity never appears in two lanes)
• mutable backend state (the same rows, records, or files either one writes)
• a throttle, rate limit, or cache one of them can exhaust or poison
• an environment reset step

Always pinned to lane 1 (the serial lane):

• every case with an operator handoff (one human, one prompt at a time)
• every case using log correlation (parallel actions interleave the dev log and break offset slicing)

The plan's Lanes section lists: lane → identity/identities → case IDs → why this lane is separate. The Readiness Gate gains one line: no identity and no mutable state appears in more than one lane.

Case Format

Each case is a table row in its area section:

• ID — QA-NN, sequential within the plan.
• Req — REQ-ID(s) this case verifies. Omitted in general mode.
• Driver — bash, browser, or bash + browser.
• Operator — none (agent runs the case end-to-end) or a precise description of the single human action required. The agent prints it verbatim and waits.
• Steps — one action per line, tagged [bash] or [browser]. A [bash] step is a shell command. A [browser] step is a Playwright action: navigate, click, fill, assert visible text, assert an aria attribute, capture a network request. Compound assertions are split — one result per line.
• Expected — one observable result per line, every line falsifiable and tagged with its verification tier:
  • [assert] — machine-verifiable: visible text, aria-* value, role, HTTP status, JSON key/value, console error, network request URL/method/status, database row count or column value. The default tier — use it whenever a mechanical observable exists.
  • [judge] — a qualitative property the executing agent judges against an explicit criterion written in the line: [judge] <property> — pass if <criterion>. Use only when the property is inherently qualitative (the copy says the right thing, an error message is actionable). The criterion is fixed here, at plan time; a [judge] line without a written pass/fail criterion is not ready. "Looks right" is not a criterion.
  • [judge-visual] — a visual property judged from a screenshot of the rendered surface: [judge-visual] <property> — pass if <criterion>. The preceding step must capture the screenshot (screenshot specs/qa/evidence/<case>.png). Use for layout, emphasis, visual hierarchy, and "is this what the requirement asked for" checks that no DOM assert can express. When a design reference exists (Figma frame export, mockup), name it in the line — the judgment compares screenshot against reference. Same rule as [judge]: no written pass/fail criterion, not ready. "Matches the design" alone is not a criterion.

Illustrative example (a generic settings toggle — derive real cases from the conversation):

| QA-3 | R7 | bash + browser | none | 1. [browser] Sign in as `qa-member` (login helper, §1)<br>2. [browser] Navigate to `$BASE/settings`<br>3. [browser] Click the switch labeled `Email notifications`<br>4. [bash] Read the stored notification preference for `qa-member` (state reader, §1) | [assert] The switch has `aria-checked="true"`<br>[assert] Stored preference is enabled<br>[assert] No console errors<br>[judge] The confirmation toast tells the user what changed — pass if it names email notifications specifically, fail if it is a generic "Settings saved" |

Operator Handoffs

When Operator is not none, the handoff is always the final step of the case — never in the middle. At run time the executing agent prints:

"Operator required. Please complete the following step in the browser, then reply done: [verbatim operator instruction]."

…then stops and waits for the developer's reply, verifies the Expected column with the case's driver, and continues. If a case needs an operator action as a pre-condition, that's a separate preconditions entry (P1, P2, …), not an inline handoff.

Output Naming

The plan is saved as specs/qa/QA-<N>-<slug>.md, following the same naming contract as REQ/ARCH/TASKS: <N> is the linked issue number; reuse the branch's <slug> when the branch matches {type}/<N>/<slug> for the same issue, otherwise derive one via the SLUG rule in start-task/SKILL.md.

No issue number (general mode from a PR, or ad-hoc work): fall back to QA-<slug>.md (or QA-PR-<number>.md when only a PR number identifies the work) and omit the Issue: row entirely — never write it as #none, empty, or a placeholder. Omit any Specs: link whose target doesn't exist.

Results are not part of this artifact: /execute-qa writes them to the sibling QA-RESULTS-<N>-<slug>.md (same stem, RESULTS infix).

Structure of the Output Plan

The full plan structure lives at {base_directory}/artifact-template.md. When you reach the artifact-writing step — not earlier — read that file and follow its structure exactly, filling every section. Do NOT write the plan from memory or improvise the format.

You Must NOT

• Bake project-specific traps, helpers, identities, or commands into a plan without eliciting or verifying them for this project — the examples in this skill are illustrative, never copied.
• Write an unfalsifiable Expected line — every line is [assert] (machine-verifiable) or [judge] with an explicit written pass/fail criterion.
• Downgrade a mechanically observable result to [judge] — judgment is for inherently qualitative properties only.
• Omit a changed file from the Coverage Map without flagging it.
• Insert an operator handoff mid-case — it is always the final step.
• Use one identity across two drivers unless the case documents why that is safe.
• Leave an elicited trap un-codified as a Guard: on the specific step that needs it.
• Skip the Phase B conversation or write the artifact before the developer confirms the scenario list.
• Write results into the plan — execution and its verdicts belong to /execute-qa.

Readiness Gate

Write the artifact only when all of these are true:

• Every changed file is in the Coverage Map or explicitly flagged as a gap.
• Every case's Expected column is falsifiable: [assert] lines machine-verifiable, [judge] lines carrying an explicit pass/fail criterion fixed at plan time.
• Every trap surfaced in Phase B is a Guard: on a specific step.
• Shell helpers and identities are agreed and expressed as real commands for this project.
• Every unavoidable human action is a named operator handoff, final step of its case.
• No identity and no mutable state appears in more than one lane.
• The developer has explicitly confirmed the scenario list.

If any are false, keep the conversation going.

Reminders

• Use today's date in the plan.
• Pipeline mode: link the REQ, ARCH, and review report in the header (review may run before, alongside, or after this plan — omit the review link only if no report exists yet). Trace each case's Req column to real REQ-IDs.
• The plan is this skill's deliverable; the results artifact is /execute-qa's.
• Like the other specs/ artifacts, QA plans merge with the branch and are retired by /archive-issue <issue#> after the PR merges.