Agent QA Gates
A field-tested validation system for AI agent output. Born from production failures, not theory.
Quick Start
Before any agent delivers output, run the Pre-Ship Checklist:
- Accurate? — every number/date/metric has a source. Unsourced → prefix "estimated"
- Complete? — no missing pieces, no "I'll do that next"
- Actionable? — ends with clear next step or decision point
- Fits the channel? — check character limits for your delivery surface
- No leaks? — no internal context, private data, or secrets
- Not a duplicate? — verify no recent identical send
- Would the human be embarrassed? — if yes, don't ship
Gate Tiers
Four ascending tiers by risk level:
| Gate |
Scope |
Key Checks |
| Gate 0 |
Internal (files, config, memory) |
Mechanism changed not just text, no placeholders, file exists |
| Gate 1 |
Human-facing (briefings, summaries) |
Key info in first 2 lines, ≤3-line paragraphs, channel length limits |
| Gate 2 |
External (email, public content, client materials) |
No internal context leaked, recipient-appropriate tone, dedup check |
| Gate 3 |
Code & technical |
Builds clean, no secrets in code, error handling, tests pass |
See references/gates-detail.md for full gate checklists.
Severity Classification
Not all failures are equal:
- 🔴 BLOCK — cannot ship (secrets, privacy, hallucinated data, wrong recipient)
- 🟡 FIX — fix before shipping, <2 min (formatting, too long, missing citation)
- 🟢 NOTE — log and ship (style preference, minor optimization)
Protocol Gates
Recurring failure modes need dedicated gates. These are the most common:
Heartbeat / Periodic Check Output
- Binary output: alert text ONLY or status-OK ONLY. Never mixed.
- Every data point verified by current-session tool call. No hallucinated metrics.
- No stale data from previous cycles or pre-compaction sessions.
Post-Compaction / Context Reset
- Do not trust facts from the pre-reset session — verify from files and tools.
- Rerun pending checks from scratch.
- Zero carryover for periodic checks.
Scheduled Job / Cron Changes
- Explicit timeout set
- Explicit model set
- Verify schedule after creation
- Output fits destination channel limits
Sub-Agent Output Review
- Does output match the brief's success criteria?
- Any uncertainty flags unresolved?
- Is the reasoning (not just the conclusion) sound?
Gate Evolution
Gates should evolve based on real failures, not imagination:
- When a failure occurs → log it with root cause
- Same failure class occurs 2+ times → add a gate item
- Monthly: prune gates that haven't caught anything in 60 days
Anti-Patterns
- Gates that sound good but never catch anything → kill them
- Per-agent checklists that duplicate general gates → merge or reference
- "ADHD-friendly" or "high-quality" as gate items → not testable, replace with mechanical checks
- Aspirational gates nobody runs → either automate or cut
Adapting to Your System
This skill provides the pattern. Adapt it:
- Start with the Pre-Ship Checklist — it works for any agent system
- Add Protocol Gates for your top 3 recurring failure modes
- Set channel limits for your delivery surfaces
- Map real failures to gates — if a failure isn't gated, add the gate
- Kill gates that never fire — a shorter, sharper checklist wins
For the full reference implementation, see references/gates-detail.md.
For automation scripts, see scripts/qa-check.sh.
1---2name: agent-qa-gates3description: Output validation gates for AI agent systems. Prevents hallucinated data, leaked internal context, wrong formats, duplicate sends, and post-compaction drift. Use when building or operating an agent that delivers output to humans or external systems. Provides a tiered gate system (internal → user-facing → external → code), protocol gates for recurring failure modes, severity classification, and a feedback loop for gate evolution. Triggers on phrases like "QA gates", "validation", "output quality", "prevent hallucination", "delivery checklist", "agent QA".4---56# Agent QA Gates78A field-tested validation system for AI agent output. Born from production failures, not theory.910## Quick Start1112Before any agent delivers output, run the **Pre-Ship Checklist**:13141. **Accurate?** — every number/date/metric has a source. Unsourced → prefix "estimated"152. **Complete?** — no missing pieces, no "I'll do that next"163. **Actionable?** — ends with clear next step or decision point174. **Fits the channel?** — check character limits for your delivery surface185. **No leaks?** — no internal context, private data, or secrets196. **Not a duplicate?** — verify no recent identical send207. **Would the human be embarrassed?** — if yes, don't ship2122## Gate Tiers2324Four ascending tiers by risk level:2526| Gate | Scope | Key Checks |27|------|-------|------------|28| Gate 0 | Internal (files, config, memory) | Mechanism changed not just text, no placeholders, file exists |29| Gate 1 | Human-facing (briefings, summaries) | Key info in first 2 lines, ≤3-line paragraphs, channel length limits |30| Gate 2 | External (email, public content, client materials) | No internal context leaked, recipient-appropriate tone, dedup check |31| Gate 3 | Code & technical | Builds clean, no secrets in code, error handling, tests pass |3233See `references/gates-detail.md` for full gate checklists.3435## Severity Classification3637Not all failures are equal:3839- 🔴 **BLOCK** — cannot ship (secrets, privacy, hallucinated data, wrong recipient)40- 🟡 **FIX** — fix before shipping, <2 min (formatting, too long, missing citation)41- 🟢 **NOTE** — log and ship (style preference, minor optimization)4243## Protocol Gates4445Recurring failure modes need dedicated gates. These are the most common:4647### Heartbeat / Periodic Check Output48- Binary output: alert text ONLY or status-OK ONLY. Never mixed.49- Every data point verified by current-session tool call. No hallucinated metrics.50- No stale data from previous cycles or pre-compaction sessions.5152### Post-Compaction / Context Reset53- Do not trust facts from the pre-reset session — verify from files and tools.54- Rerun pending checks from scratch.55- Zero carryover for periodic checks.5657### Scheduled Job / Cron Changes58- Explicit timeout set59- Explicit model set60- Verify schedule after creation61- Output fits destination channel limits6263### Sub-Agent Output Review64- Does output match the brief's success criteria?65- Any uncertainty flags unresolved?66- Is the reasoning (not just the conclusion) sound?6768## Gate Evolution6970Gates should evolve based on real failures, not imagination:71721. When a failure occurs → log it with root cause732. Same failure class occurs 2+ times → add a gate item743. Monthly: prune gates that haven't caught anything in 60 days7576## Anti-Patterns7778- Gates that sound good but never catch anything → kill them79- Per-agent checklists that duplicate general gates → merge or reference80- "ADHD-friendly" or "high-quality" as gate items → not testable, replace with mechanical checks81- Aspirational gates nobody runs → either automate or cut8283## Adapting to Your System8485This skill provides the pattern. Adapt it:86871. **Start with the Pre-Ship Checklist** — it works for any agent system882. **Add Protocol Gates** for your top 3 recurring failure modes893. **Set channel limits** for your delivery surfaces904. **Map real failures to gates** — if a failure isn't gated, add the gate915. **Kill gates that never fire** — a shorter, sharper checklist wins9293For the full reference implementation, see `references/gates-detail.md`.94For automation scripts, see `scripts/qa-check.sh`.
dvcrn