gabrielras Background Job & Queue Patterns
Overview
Every queue system — regardless of language or backend (Redis, Postgres, a message broker) — solves the same core problem: deliver work at least once to a worker that might crash, hang, or run twice. The same 14 patterns fix this in any stack; only the syntax changes.
Core principle: the pattern is the reusable unit, not any one library's implementation.
When to Use
- Reviewing code that enqueues or processes jobs, in any language
- Diagnosing: duplicate side effect on retry, job stuck "active"/"busy" forever, retries landing in a synchronized burst against an external API, worker memory that only climbs, a job silently missing after a deploy, DB connection pool exhausted by workers
- Choosing a queue architecture (Redis vs Postgres
SKIP LOCKEDvs a real message broker)
Quick Reference
| Pattern | Prevents | Detail |
|---|---|---|
| Idempotency key | Duplicate side effects on retry | patterns.md #1 |
| Visibility timeout / lease | Job "lost" if worker dies mid-run | patterns.md #2 |
| Dead-letter queue | Infinite retry loop | patterns.md #3 |
| Backoff + jitter | Retry storm / thundering herd | patterns.md #4 |
| Concurrency ≤ weakest downstream resource | Connection exhaustion | patterns.md #5 |
| Transactional outbox | Job fired before commit, or lost after a crash | patterns.md #7 |
| Competing consumers | Throughput capped by a single worker | patterns.md #13 |
| Circuit breaker | Retrying against a dependency that's already down | patterns.md #14 |
Diagnosis Flow
A symptom almost always maps to one broken pattern. Check in this order:
- Duplicate side effect? → missing idempotency key (pattern 1)
- Job stuck "active"/"busy" forever? → worker died without releasing its lock — check lease/lock duration (pattern 2; the mechanism's name differs per stack, but it's always the same reservation-with-expiry)
- Retries landing in a synchronized burst? → backoff without jitter (pattern 4)
- Worker memory only climbs? → before assuming a leak, confirm: allocator fragmentation, an unreleased C-level handle in a long-running process, or a genuine leak — the root cause differs per runtime, the symptom doesn't
- Connection pool exhausted, but queue throughput looks fine? → concurrency not bounded by the weakest downstream resource (pattern 5)
- Jobs silently missing after a deploy? → missing explicit restart signal for the worker (pattern 10)
Reference
- references/patterns.md — all 14 patterns in full: core principle, when to use, ❌ BEFORE / ✅ AFTER pseudocode, common mistakes
Common Mistakes
- Assuming the queue guarantees order — most don't by default (pattern 8)
- Treating any unhandled failure as "just that job failed" without confirming the runtime's concurrency unit — in some models it kills the entire process (pattern 6)
- Recommending a queue library without checking it's still actively maintained