QA Verify

Unified runtime invocation

Resolve the plugin root from this loaded file: SKILL.md is at <plugin-root>/skills/<skill-name>/SKILL.md. Invoke only python3 "<plugin-root>/coordinator.py" and send one bounded JSON routing request on EOF-delimited stdin, without a PTY. Use the Python invocation example in the Routing request section in <plugin-root>/README.md and the co-packaged manifest's signed wire_contract; never invent fields or provider actions. Supply one caller-defined work unit per independently useful deliverable, with this skill's logical action and a bounded opaque payload. Use depends_on only for actual dependencies. Honor an operator-named provider with explicit_target. For an authorized independent review or governance task without an operator-named provider, also use that field to bind the caller-verified distinct reviewer selected by the caller or designated by the workflow. Carry the same target into planning and live dispatch; verify returned native lineage before accepting independence. Otherwise use normal untargeted routing. Choose quality and effort for the workload; include context/output token estimates when known. Read the current manifest digest and actual cwd device/inode; do not copy example values. The runtime owns its timeout; do not wrap it in a shorter fixed timeout. Repository identity, source-head verification, disposable copies, patch capture, and cleanup remain caller-owned where applicable. The shim runs standalone from the installed plugin and transports the routing client's bounded result without semantic interpretation. Never discover a provider executable, reconstruct a raw command, or replay, retry, or fail over a consumed work unit. Provider status, terminal records, receipts, telemetry, and other structured fields are optional diagnostics; none is a content-availability gate. Preserve every returned content record or recovered partial response and interpret it with ordinary model reasoning. Never synthesize approval, authority, or a receipt from process exit or missing diagnostics. A planning-only request sets dispatch_requested=false; a live request sets it true and consumes at most one provider attempt per work unit. Planning reports route eligibility, not live availability or authentication. Report a caller/client failure at that layer; provider state remains unknown unless native evidence establishes it. Content availability and each work unit's execution_status are separate facts.

QA verify — verification of a completed execution

A second-opinion is a review of a plan. A code-review is a critique of a code artifact. qa-verify is verification of a completed execution against the original request. The point is to catch the gap between "the script ran" and "the script accomplished what was actually asked for" — missed constraints, off-by-one results, hallucinated output fields, silent partial successes that look complete at a glance.

Treat reviewer independence as unverified until the caller establishes the observed families and sources under the verifier-independence contract below. Role names and an opposing position do not establish a different model family.

When to use

Use this skill when one or more of the following are true:

• The user explicitly asks for it — "did this actually do what I asked," "verify my work with the reviewer," "QA check this," "sanity-check the output," "did the execution meet the spec," "did we actually complete the task," "validate the result," "QA pass."
• A complex multi-step execution just finished — data pipeline run, schema migration, large refactor, batched file edits, multi-API workflow, automated deployment, parameter sweep, bulk transformation — and the output is non-trivial to inspect by eye.
• the active primary is about to report success on something hard-to-reverse and visual inspection would not catch a silent partial-success. (Examples: 47 of 50 records migrated successfully but 3 silently dropped; a refactor passes tests but changed a constant from 1000 to 100; a deploy script ran without errors but rolled out only 2 of 3 replicas.)
• The task had explicit numerical or structural constraints — "transform all rows matching X," "produce exactly N rows," "include columns A B C D," "ignore comments starting with //" — that can be checked against the output deterministically by a careful reader.

When to skip

Skip this skill when:

• The success is visually obvious to the user. A two-line script that prints Hello, World! doesn't need QA verification.
• There is no defined "did it meet the spec" criterion. Open-ended creative work (a draft email, a brainstorm output) has no pass/fail; use second-opinion for general critique instead.
• The task was so trivial that QA-pass-cost > defect-risk-cost. Don't burn a verifier call on a renamed variable.
• The execution produced its own test suite that already covers the constraint. A migration that ran a SELECT COUNT(*) post-check and asserted equality with the input row count has self-verified that constraint; piling on a verifier pass adds noise.

Verifier independence (functional contract)

Independence is caller-verified governance evidence, not a routing guarantee. For independent governance evidence, before dispatch record the observed lineage and source for both the active primary and artifact author. Select a reviewer only when its known lineage is distinct from both. The caller may use provider-free planning to inspect known family evidence. Honor an operator-named provider; do not silently replace it. For an authorized independent review or governance task without an operator-named provider, bind the verified reviewer selected by the caller or designated by the workflow using explicit_target. Carry that same target into planning and live dispatch; untargeted planning does not bind a later live request. If the target becomes unavailable, report it without silent substitution or replay. If no known-distinct eligible reviewer is established, do not dispatch as independent governance; explain the missing lineage or selection evidence. An OpenCode name is transport information, not lineage. Use only a descriptor-admitted review or governance action; never substitute document intent for review.

After the response returns, record the observed reviewer lineage and source. Accept the response as independent governance evidence only when all three lineages are known and the reviewer differs from both the primary and artifact author. A route, provider name, status, receipt, or self-assertion alone does not prove lineage. Preserve unknown lineage as unknown. Do not replay a consumed review to repair missing lineage; retain it only as clearly labelled advisory content.

Procedure

1. Gather the three pieces of evidence

A QA pass needs all three:

• The original request — exactly what the user asked for, with all constraints. Paraphrasing here defeats the QA; the verifier must see the actual ask.
• The work product — the code, query, script, or process that was run. Without this, the verifier cannot see whether the implementation could plausibly meet the constraints.
• The output / result trace — the actual output, log, summary statistics, file diff, or whatever artifact the execution produced. If the work touched multiple files or systems, include a comprehensive trace; partial evidence produces partial QA.

If any of the three is missing, gather it before invoking. A QA pass on incomplete evidence is worse than no QA pass — the false-pass signals "all clear" when the verifier never saw the relevant gap.

2. Instruct the verifier as a strict QA inspector

Before dispatch, select a reviewer with known lineage distinct from the observed primary and artifact author. Submit the sealed QA role through python3 "<plugin-root>/coordinator.py" with quality_profile='economical' and effort_class='minimal' (the skill default; raise the closed quality and effort profiles only for subtle correctness constraints). After the response, verify the observed reviewer lineage before treating it as independent governance evidence.

Use this prompt template for QA content. Provider formatting is not an output contract; the caller reasons over the complete raw response:

You are a strict QA inspector. Verify the Final Output against the Original Request below. Look for hallucinations, off-by-one errors, ignored constraints, silent partial successes, and any gap between what was asked and what was delivered.

Use `APPROVE` only when no material constraint is missed,
`REQUEST_CHANGES` when a real failed constraint is found, and
`NEEDS_DISCUSSION` when the available evidence cannot resolve the question.
Put each failed constraint and its evidence in a descriptor-owned finding.

--- ORIGINAL REQUEST ---
[paste the user's actual request, with all constraints]

--- WORK PRODUCT ---
[paste the code / query / script / process that was run]

--- FINAL OUTPUT ---
[paste the actual output / log / result trace / summary]

Make the inspector earn an approval. When filling in the ORIGINAL REQUEST constraints, phrase each one at evidence level, not presence level: require the inspector to produce what it checked (a recomputed count, a traced value, a file:line, a re-read artifact) rather than confirm that a section or field exists — a presence-level constraint lets the inspector pass work it never actually inspected. Name the likely shortcuts explicitly (e.g., "do not verify the row count from the summary line; recount from the output rows"), and state what is out of scope so the inspector does not thrash on style nits or pre-existing issues. A clean approval against presence-level constraints is weak evidence; weigh it accordingly in step 3.

Read the complete nonempty raw response and deduce the best-supported QA verdict with ordinary reasoning. Preserve partial or mixed prose; never fabricate a receipt or replay the request for formatting.

3. Adjudicate the QA result

The verdict is not the deliverable; the adjudication is.

On REQUEST_CHANGES:

• Investigate each failed constraint. Open the relevant code or output and confirm the verifier's claim. A verifier can hallucinate even when the prompt is tightly constrained.
• If the failed constraint is real, inform the user clearly: the QA pass failed, here are the missed constraints, here is the proposed fix. Do not minimize.
• If the failed constraint is hallucinated, report that explicitly: "the reviewer flagged X, but X is not in fact missing — the {field/line/path} is present at {location}." Do not automatically issue a second provider request. A later caller-authorized request is a new attempt. If the caller authorizes one, include the clarification in that new request. Otherwise, move on.

On APPROVE:

• Report "no issues flagged on independent review" — not "verified correct." A clean QA pass is one signal, not a guarantee; an independent reviewer can also miss bugs the executor missed. Overstating a PASS as "correctness verified" trains the user to trust the QA layer more than it deserves.

On NEEDS_DISCUSSION:

• Surface the unresolved or terminal result and recommend manual inspection of the relevant constraints. Do not pretend a verdict happened.

Examples across domains

QA verification applies wherever a task has a defined "did it meet the spec" criterion. A representative sample:

The descriptor-owned review verdict applies to all of these uniformly; what shifts is the categories of "ignored constraint" the verifier should be on the lookout for, framed in step 2's instruction.

Anti-patterns

• Sending only the final output without the original constraints. The verifier cannot QA against a spec it has never seen. The triple-evidence requirement (request + work product + output) is non-negotiable.
• Using this for simple tasks where success is visually obvious. Wastes a verifier call and adds noise to the audit log.
• Overstating an approval as "verified correct." It means "no issues flagged on this independent review." Independent reviewers also miss bugs. Phrasing matters; precision protects the user from over-trusting the layer.
• Claiming independent QA when the observed reviewer shares the executing author or primary family, or lineage is unknown. Such findings remain advisory and cannot clear required independent verification.
• Replaying for formatting. Preserve the raw result; never ask a second provider attempt merely to obtain different formatting.
• Treating a hallucinated FAIL as a real fail. Verify each FAILED CONSTRAINT against the actual output before alarming the user. Hallucinations happen in QA too.
• Using frontier/maximum reflexively. Binary verification is usually the right job for economical/minimal; reserve frontier/maximum for constraints requiring subtle correctness reasoning (numerical stability, regulatory interpretation, or domain-specific edge cases).
• Running QA on incomplete evidence and reporting approval to the user. A QA pass on partial evidence signals "all clear" when the verifier never saw the relevant gap. Better to gather full evidence first and accept the latency.
• Inferring "this is correct" from a clean approval. Re-read the prior point. Words matter; the user will calibrate their downstream trust on yours.