# Resilience Patterns

> Designs fault-tolerant integrations using timeouts, retries with backoff/jitter, circuit breakers, bulkheads, fallbacks, and graceful degradation. Use when a payment, bank, market-data, KYC, or other provider is slow, rate-limited, or down.

- Skill: `rockclaver/resilience-patterns` (Agent Skill)
- Install (CLI): `npx skillmds@latest add rockclaver/resilience-patterns`
- Raw SKILL.md: https://api.skillmd.com/api/skills/rockclaver/resilience-patterns/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: rockclaver (https://skillmd.com/u/rockclaver)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/rockclaver/resilience-patterns

---


# Resilience Patterns

Pair with `idempotent-financial-workflows` (retries) and `observability-instrumentation` (visibility).

## Workflow

1. **Map dependencies**: for each outbound call, note who's called (money-moving/read-only/best-effort), latency budget, and retry-safety.
2. **Apply the right pattern**:
   - **Timeout**: explicit, short connect/read timeout on every call — no unbounded waits.
   - **Retry**: only transient, idempotent ops; backoff+jitter, capped attempts; never retry a non-idempotent move without an idempotency key.
   - **Circuit breaker**: trips after N failures so a dead provider fails fast; half-open probes recovery.
   - **Bulkhead**: isolate pools/concurrency per dependency so one slow provider can't starve others.
   - **Fallback**: serve cached/stale data, queue for later, or return "unavailable" instead of hanging.
   - **Backpressure**: shed or queue load when downstream is saturated instead of piling on.
3. **Decide degradation**: best-effort calls fail open; critical money moves fail closed (stop, persist intent, surface it, retry later).
4. **Test**: timeout/slow response; 5xx/429 recovery; breaker open→half-open→closed; fallback returns degraded result; retry doesn't double-charge.
5. **Implement**: reuse existing HTTP client, retry settings, or resilience library; keep policy declarative.

## Guardrails

- Do not retry non-idempotent money moves without a durable idempotency key.
- Do not set infinite/default-large timeouts — an unbounded wait is an outage.
- Do not retry without backoff/jitter; synchronized retries cause thundering herds.
- Do not mark an op permanently failed when the side effect may have succeeded.
- Do not hide degradation; emit a metric or log so operators see it.