QA → Review → Fix Loop Protocol
Comprehensive, evidence-based QA protocol. Layer-based testing with mandatory re-verification. Works on any project — web app, mobile app, trading engine, bot, API, CLI tool, monorepo. Every layer reaches a state that is independently verified, not claimed, to function correctly.
Source: QA_REVIEW_FIX_LOOP_PROTOCOL.md — universal QA protocol
Overview
QA Review-Fix Loop implements an iterative quality cycle: review code → identify issues → fix → re-review → verify. Maintains a fix queue and tracks resolution progress until all issues are resolved.
When to Use
Trigger phrases:
- "Run QA on this codebase"
- "Full QA cycle"
- "Test everything end-to-end"
- "Pre-release testing"
- "Codebase audit"
- "Find and fix all bugs"
- "Verify this works"
Use cases:
- Pre-release QA cycles
- Post-refactor regression testing
- New codebase onboarding (understand + verify)
- Defect-driven development
- Cross-layer integration verification
When NOT to use:
- Single bug fix (use
skill://systematic-debugging instead)
- Code review only (use
skill://code-reviewer)
- Unit test writing (use
skill://test-driven-development)
When NOT to Use
- For throwaway prototypes (skip the ceremony)
- When the fix is a single-line change with no side effects
- When the codebase already has a working solution
Process
Step 0 — Discover the Codebase
Do not assume the stack. Inspect the repository first:
- Read
README.md, package.json / pyproject.toml / go.mod / Cargo.toml to identify languages, frameworks, entry points
- Map actual layers present — only test what exists:
- Frontend (web, mobile, desktop UI)
- Backend services / servers
- REST / GraphQL / RPC APIs
- Databases & migrations
- MCP servers / tool integrations
- Bots (chat, trading, Discord/Telegram/Slack)
- Engines (trading, simulation, rules, scoring)
- Background jobs / workers / cron tasks
- CLI tools
- Infra/config (Docker, CI/CD, env handling)
- Third-party integrations (payment, auth, data providers)
- For each layer, identify how it's tested (existing suite?
npm test? pytest? manual? none?)
- Produce a Layer Inventory before testing begins
Step 1 — The Loop (per layer)
For each layer in the Layer Inventory, repeat:
[1] QA PASS → run tests, click through, call endpoints, log every defect
[2] REVIEW PASS → root-cause each defect, confirm reproducibility,
reject fixes that only mask symptoms
[3] FIX PASS → implement one fix at a time per defect
[4] RE-QA → re-run the EXACT test that caught the defect,
PLUS regression check on adjacent code
[5] EXIT CHECK
├─ New or surviving defects? → back to [2] REVIEW
├─ Zero defects this pass? → layer CLEARED → next layer
└─ Same defect survives 3 attempts? → STOP, escalate with root-cause writeup
Rules:
- No layer is "done" off a single pass. Done = zero new defects + regression check
- No batching unrelated fixes — fix and re-verify as separate changes
- A fix that only suppresses a symptom is rejected at Review
- After all layers CLEARED, run one full cross-layer end-to-end pass
Step 2 — Coverage Requirements (per layer)
Frontend / UI:
- Every page/screen, route (including 404, redirects)
- Every button, menu, dropdown, modal, tab, tooltip, toast
- Every form: valid/invalid/empty input, boundary values, required fields
- Every interactive state: loading, empty, error, success, disabled, offline
- Navigation flows end-to-end
- Responsive breakpoints if applicable
Backend / Services:
- Every endpoint × every HTTP method
- Auth & permission boundaries (authenticated vs not, role A vs role B, expired tokens)
- Input validation: valid, invalid, missing, boundary, malicious
- Error handling: correct status codes, no stack traces to clients
- Idempotency where it matters (payments, order creation)
- Concurrency/race conditions (double-submits, parallel writes)
APIs:
- Response shape matches declared schema exactly
- Status codes correct for every branch (2xx/4xx/5xx)
- Rate limiting behavior if implemented
- Backward compatibility if breaking-change-sensitive
MCP Servers / Tool Integrations:
- Every tool called with: valid args, invalid args, missing args, wrong types
- Returned data matches declared output schema
- Failures surfaced as informative errors, never silent no-ops
- Tool descriptions match actual behavior
Bots (chat, trading, messaging):
- Every command/trigger phrase, including near-misses and typos
- Conversation state transitions (multi-step flows, cancel, timeout)
- Failure recovery: malformed input, network drop, upstream failure
Engines (trading, simulation, rules):
- Core calculations verified against known expected values
- Edge cases: zero, negative, null, extreme values, empty datasets
- A wrong number that doesn't crash is worse than a crash — silent miscalculation must be tested
- Determinism check where expected, or documented non-determinism where intentional
Background Jobs / Workers / Cron:
- Job runs on schedule/trigger as configured
- Failure and retry behavior
- Idempotency on re-run
- Resource cleanup (no orphaned processes, locks, temp files)
Infra / Config / CI-CD:
- Environment variable handling (missing var fails loudly)
- Build succeeds from clean clone
- CI pipeline actually runs the test suite it claims
Cross-cutting (always check):
- Logging: real errors visible, not swallowed
- Secrets: none hardcoded in code
- Dependency drift:
npm install / pip install from clean environment works
Step 3 — End-to-End Cross-Layer Pass
After every layer is CLEARED, run at least one full journey crossing layers:
User action → API call → backend logic → engine/bot processing →
database write → response back → UI reflects new state correctly
Pick 2-4 most important real-world journeys and run them fully after the last fix.
Step 4 — Definition of Done
Done when, and only when:
- Every layer in the Layer Inventory is CLEARED (zero open defects, confirmed by re-test)
- Cross-layer end-to-end pass run after the last fix, and is itself defect-free
- Final Evidence Report complete, including explicit list of anything not tested and why
If any of these three are missing, the work is not done.
Verification
Evidence Requirements (replaces "100% working" claims)
For every layer, report:
LAYER: <name>
Method: [ ] Automated (suite/command name)
[ ] Manual (what was clicked/called/observed)
Test cases run: <N>
Passed: <N>
Failed → fixed → re-verified: <N>
Open defects: <N> (must be 0 to mark CLEARED)
Not tested: <list skipped items + exact reason>
An honest "not verified — here's why" is strictly better than a false "100% confirmed."
Anti-Patterns to Reject
- Claiming "100% tested" without evidence table
- Marking a layer done after single QA pass with no re-test
- Fixing a bug by catching/silencing an error instead of addressing root cause
- Batch-fixing unrelated defects in one change
- Skipping harder layers while testing easy ones
- Reporting "no defects found" without stating what was actually run
Tracking Template
Use this for each layer:
| Pass |
Method |
Defects Found |
Fixed |
Re-verified |
Open |
| 1 |
Automated/Manual |
N |
N |
N |
N |
| 2 |
Automated/Manual |
N |
N |
N |
0 |
Status progression: Not started → In QA → Fixing → Re-QA → CLEARED
Anti-Rationalization Table
| Rationalization |
Reality |
| "Tests slow me down" |
Bugs slow you down 10x more. Tests are speed, not overhead. |
| "I will refactor later" |
Technical debt compounds. Refactor as you go. |
| "It works on my machine" |
If it is not in CI, it does not work. Ship proof, not claims. |
Related Skills
skill://verification-before-completion — Pre-completion verification
skill://code-reviewer — Code review process
skill://systematic-debugging — Individual bug investigation
skill://test-driven-development — TDD workflow
skill://engineering-hard-rules — Engineering enforcement protocol
skill://requesting-code-review — Request external review
1---2name: qa-review-fix-loop3description: Use when comprehensive QA→Review→Fix loop protocol for any codebase.4 Layer-based testing with evidence requirements. Use when performing full QA cycles,5 codebase audits, pre-release testing, or.6domain: development7author: oyi778license: Apache-2.09subdomain: software-development10tags:11- qa12- testing13- quality-assurance14- review-loop15- defect-tracking16- regression-testing17- evidence-based18version: 1.0.019category: development20metadata:21 source: https://skillmd.com/skills/oyi77/qa-review-fix-loop22---232425# QA → Review → Fix Loop Protocol2627Comprehensive, evidence-based QA protocol. Layer-based testing with mandatory re-verification. Works on any project — web app, mobile app, trading engine, bot, API, CLI tool, monorepo. Every layer reaches a state that is **independently verified**, not claimed, to function correctly.2829**Source:** QA_REVIEW_FIX_LOOP_PROTOCOL.md — universal QA protocol3031## Overview3233**QA Review-Fix Loop** implements an iterative quality cycle: review code → identify issues → fix → re-review → verify. Maintains a fix queue and tracks resolution progress until all issues are resolved.343536## When to Use3738**Trigger phrases:**39- "Run QA on this codebase"40- "Full QA cycle"41- "Test everything end-to-end"42- "Pre-release testing"43- "Codebase audit"44- "Find and fix all bugs"45- "Verify this works"4647**Use cases:**48- Pre-release QA cycles49- Post-refactor regression testing50- New codebase onboarding (understand + verify)51- Defect-driven development52- Cross-layer integration verification5354**When NOT to use:**55- Single bug fix (use `skill://systematic-debugging` instead)56- Code review only (use `skill://code-reviewer`)57- Unit test writing (use `skill://test-driven-development`)585960## When NOT to Use6162- For throwaway prototypes (skip the ceremony)63- When the fix is a single-line change with no side effects64- When the codebase already has a working solution656667## Process6869### Step 0 — Discover the Codebase7071Do not assume the stack. Inspect the repository first:72731. Read `README.md`, `package.json` / `pyproject.toml` / `go.mod` / `Cargo.toml` to identify languages, frameworks, entry points742. Map actual layers present — only test what exists:75 - Frontend (web, mobile, desktop UI)76 - Backend services / servers77 - REST / GraphQL / RPC APIs78 - Databases & migrations79 - MCP servers / tool integrations80 - Bots (chat, trading, Discord/Telegram/Slack)81 - Engines (trading, simulation, rules, scoring)82 - Background jobs / workers / cron tasks83 - CLI tools84 - Infra/config (Docker, CI/CD, env handling)85 - Third-party integrations (payment, auth, data providers)863. For each layer, identify how it's tested (existing suite? `npm test`? `pytest`? manual? none?)874. Produce a Layer Inventory before testing begins8889### Step 1 — The Loop (per layer)9091For **each layer** in the Layer Inventory, repeat:9293```94[1] QA PASS → run tests, click through, call endpoints, log every defect95[2] REVIEW PASS → root-cause each defect, confirm reproducibility,96 reject fixes that only mask symptoms97[3] FIX PASS → implement one fix at a time per defect98[4] RE-QA → re-run the EXACT test that caught the defect,99 PLUS regression check on adjacent code100[5] EXIT CHECK101 ├─ New or surviving defects? → back to [2] REVIEW102 ├─ Zero defects this pass? → layer CLEARED → next layer103 └─ Same defect survives 3 attempts? → STOP, escalate with root-cause writeup104```105106**Rules:**107- No layer is "done" off a single pass. Done = zero new defects + regression check108- No batching unrelated fixes — fix and re-verify as separate changes109- A fix that only suppresses a symptom is rejected at Review110- After all layers CLEARED, run one full cross-layer end-to-end pass111112### Step 2 — Coverage Requirements (per layer)113114**Frontend / UI:**115- Every page/screen, route (including 404, redirects)116- Every button, menu, dropdown, modal, tab, tooltip, toast117- Every form: valid/invalid/empty input, boundary values, required fields118- Every interactive state: loading, empty, error, success, disabled, offline119- Navigation flows end-to-end120- Responsive breakpoints if applicable121122**Backend / Services:**123- Every endpoint × every HTTP method124- Auth & permission boundaries (authenticated vs not, role A vs role B, expired tokens)125- Input validation: valid, invalid, missing, boundary, malicious126- Error handling: correct status codes, no stack traces to clients127- Idempotency where it matters (payments, order creation)128- Concurrency/race conditions (double-submits, parallel writes)129130**APIs:**131- Response shape matches declared schema exactly132- Status codes correct for every branch (2xx/4xx/5xx)133- Rate limiting behavior if implemented134- Backward compatibility if breaking-change-sensitive135136**MCP Servers / Tool Integrations:**137- Every tool called with: valid args, invalid args, missing args, wrong types138- Returned data matches declared output schema139- Failures surfaced as informative errors, never silent no-ops140- Tool descriptions match actual behavior141142**Bots (chat, trading, messaging):**143- Every command/trigger phrase, including near-misses and typos144- Conversation state transitions (multi-step flows, cancel, timeout)145- Failure recovery: malformed input, network drop, upstream failure146147**Engines (trading, simulation, rules):**148- Core calculations verified against known expected values149- Edge cases: zero, negative, null, extreme values, empty datasets150- **A wrong number that doesn't crash is worse than a crash** — silent miscalculation must be tested151- Determinism check where expected, or documented non-determinism where intentional152153**Background Jobs / Workers / Cron:**154- Job runs on schedule/trigger as configured155- Failure and retry behavior156- Idempotency on re-run157- Resource cleanup (no orphaned processes, locks, temp files)158159**Infra / Config / CI-CD:**160- Environment variable handling (missing var fails loudly)161- Build succeeds from clean clone162- CI pipeline actually runs the test suite it claims163164**Cross-cutting (always check):**165- Logging: real errors visible, not swallowed166- Secrets: none hardcoded in code167- Dependency drift: `npm install` / `pip install` from clean environment works168169### Step 3 — End-to-End Cross-Layer Pass170171After every layer is CLEARED, run at least one full journey crossing layers:172173```174User action → API call → backend logic → engine/bot processing →175database write → response back → UI reflects new state correctly176```177178Pick 2-4 most important real-world journeys and run them fully after the last fix.179180### Step 4 — Definition of Done181182Done when, and only when:1831841. Every layer in the Layer Inventory is CLEARED (zero open defects, confirmed by re-test)1852. Cross-layer end-to-end pass run after the last fix, and is itself defect-free1863. Final Evidence Report complete, including explicit list of anything not tested and why187188If any of these three are missing, the work is **not done**.189190## Verification191192### Evidence Requirements (replaces "100% working" claims)193194For **every layer**, report:195196```197LAYER: <name>198Method: [ ] Automated (suite/command name)199 [ ] Manual (what was clicked/called/observed)200Test cases run: <N>201Passed: <N>202Failed → fixed → re-verified: <N>203Open defects: <N> (must be 0 to mark CLEARED)204Not tested: <list skipped items + exact reason>205```206207An honest "not verified — here's why" is strictly better than a false "100% confirmed."208209### Anti-Patterns to Reject210211- Claiming "100% tested" without evidence table212- Marking a layer done after single QA pass with no re-test213- Fixing a bug by catching/silencing an error instead of addressing root cause214- Batch-fixing unrelated defects in one change215- Skipping harder layers while testing easy ones216- Reporting "no defects found" without stating what was actually run217218### Tracking Template219220Use this for each layer:221222| Pass | Method | Defects Found | Fixed | Re-verified | Open |223|------|--------|---------------|-------|-------------|------|224| 1 | Automated/Manual | N | N | N | N |225| 2 | Automated/Manual | N | N | N | 0 |226227Status progression: `Not started` → `In QA` → `Fixing` → `Re-QA` → `CLEARED`228229## Anti-Rationalization Table230231| Rationalization | Reality |232|---|---|233| "Tests slow me down" | Bugs slow you down 10x more. Tests are speed, not overhead. |234| "I will refactor later" | Technical debt compounds. Refactor as you go. |235| "It works on my machine" | If it is not in CI, it does not work. Ship proof, not claims. |236237## Related Skills238239- `skill://verification-before-completion` — Pre-completion verification240- `skill://code-reviewer` — Code review process241- `skill://systematic-debugging` — Individual bug investigation242- `skill://test-driven-development` — TDD workflow243- `skill://engineering-hard-rules` — Engineering enforcement protocol244- `skill://requesting-code-review` — Request external review