# AI Launch Check > Run the unified pre-production launch checklist for an AI feature before shipping it. Use when the user explicitly runs /ai-launch-check or asks to "review my AI feature for launch", "is this ready to ship", "pre-production review", or similar ritual phrasing. - Skill: `kumaran-is/ai-launch-check` (Agent Skill) - Install (CLI): `npx skillmds@latest add kumaran-is/ai-launch-check` - Raw SKILL.md: https://api.skillmd.com/api/skills/kumaran-is/ai-launch-check/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Coding & Dev Tools - Author: kumaran-is (https://skillmd.com/u/kumaran-is) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/kumaran-is/ai-launch-check --- # AI Feature Launch Check > **Iron Law:** Do not output GO if any checklist item is ❌. A written exception in the Decision Record with an owner and deadline is required for every ⚠️ item. Verbal assertions do not count as evidence for any item. Run the unified pre-production launch checklist from the AI Playbook (Layer 4 §4.1). This consolidates the pre-flight questions, pre-production gate, kill-switch tests, and adversarial test catalog into one launch review. ## Procedure 1. Ask the user for the feature slug. Look for the Decision Record at `docs/ai-decisions/.md`. - If it does not exist: **stop**. Tell the user to run `/ai-decision-record` first. Do not proceed without one. 2. Read the Decision Record. Cross-check every item below against what the record says. 3. For each checklist item below, ask the user to **show evidence**, not assert compliance. Examples of evidence: - "Output Contract implemented" → point to the validation code; read it. - "Eval set" → point to the file; show the size and a sample. - "Kill switch tested" → show the test run log. 4. Mark each item: ✅ verified / ⚠️ partial / ❌ missing. 5. **Do not pass the launch check if any item is ❌.** Partial is allowed only when the user provides a written exception in the Decision Record with an owner and a deadline. 6. Produce a launch decision: **GO / NO-GO / GO-WITH-EXCEPTIONS** with a written rationale. ## The pre-flight (10 questions) Walk through these first. Any "no" or "I don't know" is a NO-GO. - [ ] 1. Can deterministic code solve this reliably? (If yes → why are we shipping AI?) - [ ] 2. Is the input genuinely unstructured or ambiguous? - [ ] 3. Does AI measurably beat code on accuracy, speed, cost, or UX? - [ ] 4. Is the decision reversible, or is there a downstream verifier? - [ ] 5. Off the critical latency path, or fast fallback exists? - [ ] 6. What's the cost when AI is wrong, and who absorbs it? - [ ] 7. Is human approval required for high-impact outcomes? - [ ] 8. Can every AI output be audited and explained later? - [ ] 9. Enough labeled examples for risk-tiered evaluation? - [ ] 10. Deterministic fallback when AI fails or times out? ## The artifacts checklist Each item must point to a file, a log, or a running piece of code. Verbal assertions do not count. - [ ] Decision Record signed off (`docs/ai-decisions/.md`) - [ ] Risk-tiered eval set with target metrics (size matches Layer 2 §2.14 for the declared risk tier) - [ ] Output Contract implemented: schema → types → business rules → permissions → confidence → audit → route (Layer 2 §2.4) - [ ] Adversarial tests in CI (Layer 3 §3.2): prompt injection, tool misuse, retrieval poisoning, bad schema outputs - [ ] RAG requirements met if applicable (Layer 2 §2.7): hybrid search, metadata filtering, ACLs at query time, citations, no-answer test, separate retrieval/answer grading, chunk provenance - [ ] Multi-modal handling reviewed if applicable (Layer 2 §2.8): each modality validated, cost recalculated, sandboxes for code execution - [ ] Model and routing strategy decided (Layer 2 §2.9) - [ ] Cost & latency budget filled (Layer 3 §3.4): max calls, max tokens, p50/p95/p99, cost ceiling, fallback after timeout, cache strategy, rate limit - [ ] 12-month AI tax estimated and approved (Layer 2 §2.11) - [ ] Vendor abstraction in place; model and prompt versions pinned (Layer 2 §2.12) - [ ] Observability schema implemented (Layer 3 §3.5): model_version, prompt_version, input_hash, validation result, confidence, routed_to, downstream outcome - [ ] Human review queue defined: what goes, who reviews, SLA per Layer 2 §2.1 - [ ] Data governance review completed (Layer 2 §2.13): context contents, logging, retention, training-use, tenant isolation, deletion propagation, redaction - [ ] Security review completed (Layer 2 §2.5): the two questions answered with code, not prompts - [ ] Rollout plan with promotion criteria (Layer 4 §4.2): offline → dogfood → shadow → beta → percentage → full - [ ] Incident response plan (Layer 4 §4.3): named on-call, rollback triggers with thresholds, affected-user identification, quarantine, reversal, post-mortem template, leading indicators - [ ] **Kill-switch tests passing** (Layer 4 §4.4): all eight tests in staging or shadow, with logs ## Kill-switch tests — must all pass Demand the test logs. If they don't exist, fail the launch check. - [ ] Disable feature via flag → graceful degradation - [ ] Force model downgrade → output contract still passes or routes to review - [ ] Simulate vendor outage → timeout fires, fallback serves, no infinite retry - [ ] Simulate retrieval poisoning → contract/security rejects; provenance logs source - [ ] Simulate cost spike (10× tokens) → rate limit + cost ceiling fire; alerts route correctly - [ ] Simulate prompt injection from user input AND from a retrieved doc → least-privilege scope prevents harm - [ ] Roll back one prompt or model version → traffic shifts cleanly; metrics return to baseline - [ ] Verify deletion propagation → user-deleted data gone from embeddings, caches, logs, evals within SLA ## Decision output After running through every item, output: ```md # Launch Check: **Decision**: GO | NO-GO | GO-WITH-EXCEPTIONS **Reviewer**: — **Decision Record**: docs/ai-decisions/.md ## Verified (✅) - ... ## Partial — exceptions filed (⚠️) - : exception in Decision Record, owner , deadline ## Missing — blockers (❌) - ... ## Recommendation ## If GO-WITH-EXCEPTIONS: follow-up SLAs - ... ``` ## Default stance - Bias toward NO-GO. A launch check is a forcing function; if the team passes too easily, the check isn't doing its job. - Do not let "we'll fix it after launch" pass for kill-switch tests, output contract, or security review. Those are pre-launch by definition. - Tier-3+ features with no human approval gate are an automatic NO-GO unless the Decision Record has a written domain-risk exception. ## If this skill cannot proceed ``` Abort condition: Decision Record missing → Hard stop. Output: "Run /ai-decision-record first. The launch check cross-references the Decision Record for every checklist item. Cannot proceed without it." Abort condition: user provides verbal assertions instead of file:line evidence → Mark item ⚠️ at best, ❌ if critical (kill switch, output contract, security review). → Output: "Verbal assertions are not evidence. Point to the file, log, or test output." Abort condition: all 8 kill-switch tests have no logs → Automatic NO-GO regardless of other items. → Output: "Kill-switch tests must be run in staging or shadow with logs. 'We tested it manually once' does not satisfy this requirement." ``` ## Verify (after producing the launch decision) ``` ✅ Every checklist item has a status: ✅ / ⚠️ / ❌ — no blanks ✅ Every ⚠️ item has a named owner and a written deadline in the Decision Record ✅ Decision (GO / NO-GO / GO-WITH-EXCEPTIONS) matches the item statuses — no ❌ items in a GO ✅ Launch check report written to docs/ai-audits//04-launch-check.md ``` ## References - AI Playbook Layer 4 §4.1 Unified launch checklist: `.claude/skills/ai-playbook/playbook.md` - AI Playbook Layer 4 §4.2 Rollout strategy: `.claude/skills/ai-playbook/playbook.md` - AI Playbook Layer 4 §4.4 Kill-switch tests: `.claude/skills/ai-playbook/playbook.md` - AI Playbook Layer 2 §2.14 Eval size floors: `.claude/skills/ai-playbook/playbook.md` - Prerequisite: `/ai-decision-record` (must exist before launch check runs)