Independent QA

Independent QA

Dispatch isolated reviewer subagents to verify your code. Harnessed uses independent QA to lower self-evaluation bias and improve issue-finding rate. Self-critique is useful for improving a draft, but it does not replace the independent QA loop.

When This Skill Activates

• After completing a round of code changes (new code, bug fix, refactor)
• After fixing issues from a previous QA round (re-evaluation)
• NOT for micro tasks (single-line, config-only, comment-only changes)

Execution Flow

Step 1: Locate the Contract

Determine the acceptance criteria source:

• Standalone mode: Read .harnessed/contract.md
• Complementary mode (Superpowers): Read the spec from docs/superpowers/specs/, normalize it into .harnessed/contract.md, and use that as the shared criteria source

If no contract/spec exists: STOP. Invoke harnessed:contract-writing first.

Step 2: Detect Verification Tier

Check the project for available verification infrastructure.

Tier 2 — test suite indicators present (package.json test script, pytest, go test, Playwright/Cypress, or equivalent) Tier 1.5 — no test suite, but a dev server is running Tier 1 — code review only

Record the active tier.

Step 2b: Pre-Flight Checks

Before dispatching reviewers, run available zero-LLM checks:

• type checker
• linter
• test suite

If these fail, fix them first.

Step 2c: Git State Checks

Before gathering context:

• stop on active merge conflict (.git/MERGE_HEAD)
• use git diff HEAD to capture staged + unstaged changes
• include untracked files by appending their contents to the diff context

Extract the changed file paths from this diff. Use them to drive any scope-based policy selection.

Step 2d: Risk Classification and Review Mode

Classify the task as standard or high-risk.

A task is high-risk if it touches auth, authz, secrets, crypto, payments, privacy, destructive data operations, infra/prod config, new public endpoints, or if the user explicitly says it is security-sensitive or release-blocking.

Choose one review mode:

• standard-review — one primary evaluator
• corroborated-review — primary evaluator + second independent evaluator for high-risk tasks
• security-review — primary evaluator + dedicated security reviewer for security-sensitive tasks
• tie-break-review — dispatched only when prior reviewers disagree materially

Material disagreement means any of the following:

• different overall grades
• one reviewer says PASS and another says FAIL/PARTIAL for the same criterion
• one reviewer says SHIP while another requires human review for a high-risk boundary

Step 2e: Calibration and Drift Governance

Read docs/evaluator-calibration.md before dispatching reviewers.

This step exists to make evaluator drift explicit rather than silently trusted.

Set Calibration Status to:

• current — calibration guidance exists and matches the current evaluator configuration
• stale — evaluator prompt, model, or high-risk workflow changed since the last calibration review
• missing — no calibration guidance exists

For high-risk tasks, stale or missing calibration means the best possible fully automated outcome is SHIP_WITH_HUMAN_REVIEW, not plain SHIP.

Step 2f: Select Relevant Policy Summaries

If .harnessed/policies/ exists, select only the policy files relevant to the current diff scope.

Use simple path and keyword matching rather than semantic inference:

• auth-related paths or keywords -> auth.md
• service/API changes -> response.md + db.md + validation.md
• test changes -> testing.md
• security-sensitive paths or keywords -> security.md

Read only the matched files. Keep the token budget bounded: inject at most the small set of matching policy summaries, not the full policy directory. If no policy matches, skip policy injection entirely and keep the existing behavior.

Step 3: Gather Context for Reviewers

Collect the following — this is ALL the reviewers see:

DO NOT include:

• your reasoning or thought process
• your self-assessment of the code
• your conversation history with the user
• planning notes

The reviewer must judge the code, not your intent.

Step 4: Dispatch the Review Set

Construct reviewer prompts from the prompt files in skills/independent-qa/:

• primary evaluator → evaluator-prompt.md, writes .harnessed/qa-report.md
• second evaluator for corroborated-review → same prompt, writes .harnessed/qa-report-secondary.md
• security reviewer for security-review → security-reviewer-prompt.md, writes .harnessed/qa-report-security.md
• tie-break reviewer → tie-break-reviewer-prompt.md, writes .harnessed/qa-report-tiebreak.md

When using evaluator-prompt.md, replace:

• {CONTRACT}
• {DIFF}
• {STACK}
• {TIER}
• {MODE}
• {RISK_LEVEL}
• {CALIBRATION_STATUS}
• {VERIFICATION_COMMANDS}
• {RELEVANT_POLICIES}
• {GRADING_RUBRIC}

Step 4b: Static Analysis for Security-Sensitive Tasks

If the task is security-sensitive, run available tooling before final synthesis:

• Semgrep
• CodeQL
• bandit

Treat tool findings as strong evidence. Treat tool absence as uncertainty, not as proof that the code is secure.

Step 4c: Verify Reviewer Output

After reviewers return, verify that each expected report:

• exists
• contains # QA Report
• is newer than dispatched_at in .harnessed/qa-state.md

If a required report is missing or malformed, retry once. If it still fails, treat as BLOCKED.

Step 5: Synthesize Results

Use the most conservative defensible outcome.

• If reviewers materially disagree, dispatch the tie-break reviewer
• If the tie-break reviewer still cannot resolve the issue, escalate to the user
• If any automatable criterion has weak evidence, downgrade it to PARTIAL or MANUAL_REVIEW_NEEDED rather than PASS
• If the task is high-risk and calibration is stale/missing, the best possible result is SHIP_WITH_HUMAN_REVIEW
• If the task is security-sensitive and security review remains heuristic or tooling is absent, force explicit human review even when all automatable checks pass

Step 6: Process Final Grade

If final grade is SHIP or SHIP_WITH_HUMAN_REVIEW:

• proceed to harnessed:verification-gate
• note any pending human review items

If final grade is ITERATE:

• fix the issues
• re-run QA
• maximum 3 iterations

If final grade is BLOCKED:

• do not attempt speculative fixes
• present the blocking reason to the user

Step 7: Record Failure Patterns

After ITERATE or BLOCKED, update .harnessed/failure-patterns.md from the evaluator's ## Failure Categories table.

Apply the decay and cap rules already defined by Harnessed.

Step 8: Write QA Summary

After a SHIP or SHIP_WITH_HUMAN_REVIEW result, append:

## QA Summary
- Tier: {1, 1.5, or 2}
- Risk level: {standard or high-risk}
- Review mode: {standard-review / corroborated-review / security-review / tie-break-review}
- Calibration status: {current / stale / missing}
- Confidence: {High / Medium / Low}
- Uncertainty: {brief list or None}
- Iterations: {count}
- Final grade: {SHIP or SHIP_WITH_HUMAN_REVIEW}
- Key findings fixed: {brief list of issues caught and fixed during iterations, if any}

Iteration Rules

• Each QA round uses FRESH reviewer context
• Each round uses the latest diff
• Never lower the bar between iterations
• qa-state.md must track:
  • iteration
  • dispatched_at
  • head_commit
  • contract_hash
  • risk_level
  • review_mode
  • calibration_status

Anti-Rationalization

Self-critique can improve a draft. It cannot replace independent verification.

High-risk disagreement must be resolved or escalated — never silently ignored.