Retry Safety Review

Retry Safety Review

Install

Save this file as ~/.claude/skills/retry-safety-review/SKILL.md, or .claude/skills/retry-safety-review/SKILL.md to scope it to one repo. Claude Code auto-discovers it. Invoke with /retry-safety-review or by asking "is it safe to retry this?".

Why this exists

Retries are added during an outage, by someone who wants the errors to stop. The wrapper goes around everything, the errors do stop, and a category of much worse bug is created silently: work that ran twice.

The trap is that the two cases look identical from the caller's side. A connection refused before the request left your process did nothing. A timeout after the request arrived may have committed, charged, or sent. Both surface as "the call failed". Retrying the first is free. Retrying the second duplicates.

The question is never "did it fail?" It is "do I know whether it ran?"

Step 1: Find every retry that already exists

rg -n "retry|retries|backoff|attempt|maxAttempts|reconnect" -i \
  --type-add 'code:*.{ts,tsx,js,jsx,py,rb,go,java}' -t code
rg -n "p-retry|async-retry|tenacity|backoff|resilience4j|Polly" -t code

Also find the retries you do not control:

• Message queue redelivery (SQS, Pub/Sub, Kafka, BullMQ, Sidekiq).
• Webhook senders. Stripe, GitHub, and Shopify all retry on non-2xx.
• Load balancers and service meshes configured to retry idempotent methods.
• Browser or client SDK retries.
• A human clicking a button twice.

Every one of those is a retry your code must survive, whether or not you wrote a retry loop.

Step 2: Classify each retried operation

For every operation inside a retry, put it in exactly one bucket:

SAFE (naturally idempotent). Running it twice produces the same end state as running it once. Pure reads. SET x = 5. DELETE WHERE id = 1. Writes keyed by a unique constraint that would reject the duplicate.

SAFE ONLY WITH A KEY. Not naturally idempotent, but made safe by an idempotency key, a unique constraint, or a conditional write. Creating a charge with Idempotency-Key. INSERT ... ON CONFLICT DO NOTHING. Check the key is actually derived from the WORK, not generated per attempt: a key created inside the retry loop is a new key every time and protects nothing.

UNSAFE. Running it twice does damage and nothing prevents it. Relative updates (balance = balance + 10). Sending email, SMS, or a push notification. Appending to a ledger. Calling a third-party API that creates a resource without an idempotency key. Anything that increments a counter.

Be specific about which line makes it unsafe.

Step 3: Classify each failure mode

This is the step people skip. For each error your retry catches, decide:

DEFINITELY DID NOT RUN. Connection refused, DNS failure, TLS handshake failure, request rejected before dispatch, client-side validation error. The work provably never started. Safe to retry anything.

MAY HAVE RUN. Timeout, connection reset mid-flight, 5xx from a server that already received the body, a database driver error raised after the statement was sent, any error where the response was lost rather than never produced.

The second category is the whole point. A retry policy that treats "timed out" the same as "connection refused" will double-apply the first time a slow write succeeds after the client gave up.

Go read the error codes your retry actually matches, and sort them into these two buckets by hand. If a code is ambiguous, it belongs in MAY HAVE RUN. Ambiguity is not a reason to retry; it is the reason not to.

Step 4: Cross the two lists

Build the grid. Only one cell is a defect:

For every UNSAFE operation retried on a MAY HAVE RUN error, you have found a real bug. Describe the exact interleaving that duplicates, with the concrete sequence of events, not a general worry.

Step 5: Check what the retry does to the rest of the system

Even correct retries cause damage at scale:

• No backoff. Immediate retries against a struggling service finish it off.
• No jitter. Every client retries at the same instant and the recovery attempt becomes a second outage.
• No cap. Retries stacked at several layers multiply: 3 client x 3 gateway x 3 service is 27 attempts from one user action.
• Retrying inside a held transaction or lock. Multiplies the time the lock is held by the number of attempts.
• Retry budget vs caller timeout. If the caller gives up after 5s and the retry policy needs 12s, the retries only ever burn capacity for a response nobody will read.

Compute the real worst-case latency and attempt count, and compare it to the timeout of whatever is calling.

Step 6: Report

[SEVERITY] <operation> retried on <error class>
  Where:      file:line
  Bucket:     SAFE / SAFE WITH KEY / UNSAFE
  On failure: DEFINITELY DID NOT RUN / MAY HAVE RUN
  Duplicates: <the exact sequence that causes double-apply, or "no">
  Fix:        <idempotency key, narrow the retried error set, or do not retry>

Severity:

• CRITICAL an UNSAFE operation is retried on an ambiguous error today.
• HIGH a stacked or uncapped retry that can amplify an outage, or an idempotency key regenerated per attempt.
• MEDIUM missing jitter or backoff, or a retry budget longer than the caller's timeout.

End with the grid from Step 4 filled in for this codebase. If every retry is correctly scoped, show the grid as the evidence, and name the error codes you deliberately excluded. An excluded ambiguous code is a design decision worth recording, and it is exactly what a future reader will otherwise "helpfully" add back.

Rules

• Never widen a retry to cover a new error class without stating which bucket that class is in.
• Never claim an operation is idempotent because it "looks like" a write that is. Trace the actual statement.
• A retry you did not write still counts. Queue redelivery and webhook senders are retries.
• Do not add a retry as the fix for a bug you have not diagnosed. Retrying a deterministic failure just fails more slowly.

From Toolbay. Free to use, modify, and share. Keep this line and others can find it too.