# Failure Path Review

> Use when writing or reviewing a consumer, worker, background loop, job runner, retry or backoff path, or shutdown handling; or after an incident involving stuck, lost, or duplicated work. Not for synchronous handlers, and not for diagnosing one live failure.

- Skill: `morzecrew/failure-path-review` (Agent Skill, multi-file: 3 files)
- Install (CLI): `npx skillmds@latest add morzecrew/failure-path-review`
- Raw SKILL.md: https://api.skillmd.com/api/skills/morzecrew/failure-path-review/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: morzecrew (https://skillmd.com/u/morzecrew)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/morzecrew/failure-path-review

---


# Failure-Path Review

Happy paths verify themselves: development exercises them, demos exercise them, the first day in production exercises them. Failure paths run only when things go wrong — which is rare, unobserved, and usually concurrent with an incident — so they rot silently. A multi-package review of five broker/durable integrations found exactly this split: *every* happy path sound, and a defect list consisting entirely of poison handling, redelivery, shutdown, and recovery. Review the unhappy paths as their own deliberate pass, because nothing else will.

The review targets any system that processes work it didn't synchronously receive: message consumers, queue workers, background loops, schedulers, relays, durable workflows.

## The passes

Full sweep in [references/review-passes.md](references/review-passes.md), with
the checklist form in [references/review-checklist.md](references/review-checklist.md).
What they all share: **failure code only runs when things go wrong,** so it is
the code least likely to have been executed even once before it matters.

The seven questions each pass is an instance of:

- **What happens to a message that can never succeed?** If the answer is "it is retried", the answer is "forever".
- **Which failures are retryable, and who decided?** A blanket retry on a permanent error is an outage amplifier; a blanket give-up on a transient one is data loss.
- **What happens to work in flight when the process dies?** Crash-redelivery is the normal case, not the exceptional one.
- **Does shutdown drain or abandon?** A handler that stops accepting work but never finishes what it holds loses exactly the work that was in progress.
- **What grows without bound?** Queues, retry counters, in-memory buffers, dead-letter stores.
- **What restarts a loop that died?** A consumer that exits its own `while True` on an unhandled error stays dead, and nothing upstream necessarily notices.
- **How would anyone know any of this happened?** A failure path with no signal is one that has been working perfectly since the day it was written, as far as anyone can tell.

## Verifying the review

Failure paths cannot be verified by running the app — they need forced tests: inject the decode failure, kill the process between effect and ack, force the redelivery, deliver the poison N+1 times, stop the consumer mid-batch. Deterministic fault injection (`determinism-by-design`) makes these repeatable; the detection branches this review adds are exactly the code that must not be dead (`fewer-tests-more-proof`), and each fix found follows `reproduce-then-fix` — forced red first.

## Related skills

- `error-taxonomy` — supplies the retryable/terminal classification every pass here consumes
- `reproduce-then-fix` — each finding becomes a forced red reproduction before its fix
- `determinism-by-design` — deterministic fault injection for repeatable failure tests
- `self-audit` — its failure-path and cleanup-path passes are the diff-scoped version of this sweep