martinholovsky SOTA Async & Concurrency
Purpose
Concurrency bugs are the most expensive class of defect: they pass tests, ship, and then corrupt data or hang production under load. This skill encodes the 2026 state of the art for concurrent design and the bug catalog auditors need to spot defects by reading code, without reproducing them. Concepts are cross-language; per-runtime notes are inlined where semantics genuinely differ (GIL, goroutine scheduling, tokio executors, Node's single loop).
Two operating modes. Pick one explicitly before starting.
BUILD mode
When writing new concurrent code:
- Classify the workload first. I/O-bound → async/event loop. CPU-bound →
threads (if runtime has real parallelism) or processes. Mixed → async
front-end + bounded worker pool. Read
rules/01before choosing. - Structured concurrency is the default. Every task lives inside a scope (TaskGroup / nursery / errgroup / JoinSet) that joins or cancels it. Spawning a task with no owner is a design smell requiring written justification.
- Bound everything. Every queue, channel, connection pool, in-flight request set, and spawn loop gets an explicit capacity. Unbounded = OOM with a delay timer.
- Every await gets a timeout policy — a number, or a documented reason why it inherits one from an enclosing scope.
- Cancellation is a feature you build, not an exception you ignore. Propagate context/AbortSignal/CancelledError; clean up in finally blocks; design shutdown order (stop intake → drain → deadline → force).
- Shared mutable state needs an owner. Prefer message passing or single-owner tasks; if you must lock, define lock ordering and never hold a lock across an await.
- Re-read the audit checklists at the end of each rules file against your own diff before declaring done.
AUDIT mode
When auditing existing code, you find races, deadlocks, and leaks by reading — grep is your debugger. Workflow:
- Map the concurrency topology. What spawns tasks/threads? What shares state? What are the queues and their bounds? Draw the lock set and the channel graph mentally before judging any line.
- Sweep with targeted greps, then read each hit in context:
- Fire-and-forget:
create_task(/ensure_future(without a stored handle; barego func(;.then(with no.catch(; floating promises;tokio::spawnwhose JoinHandle is dropped. - Missing await: async calls whose return value is discarded.
- Blocking in async:
time.sleep,requests.,open(, sync DB drivers,fs.readFileSync,bcrypt.hashSync,std::thread::sleepinside async fns. - Lock across await:
async with lock:/mutex.lock()bodies containingawait/.await. - Unbounded:
Queue()with no maxsize,make(chan T)fed by fast producers,unbounded_channel, spawn-in-loop with no semaphore. - Check-then-act:
if x in d:…d[x], exists-then-create, read-modify-write on shared counters without atomics/locks.
- Fire-and-forget:
- For each suspect, prove the interleaving. State the two (or more) execution orders and which one breaks. A finding without an interleaving is a style note, not a concurrency bug.
- Read
rules/07for the full bug catalog with signatures.
Severity conventions
| Severity | Criteria | Examples |
|---|---|---|
| CRITICAL | Data corruption, deadlock, or unbounded resource growth reachable under normal load | Lost-update race on money/state; lock-ordering deadlock on hot path; unbounded queue fed by network input |
| HIGH | Hang, leak, or wrong result under plausible (load/error/timeout) conditions | Fire-and-forget swallowing exceptions; no timeout on external call; lock held across await; blocking call on event loop |
| MEDIUM | Degraded behavior, starvation, or fragility under contention | Writer starvation on RwLock; missing jitter on retries; thundering herd on cache expiry; spurious-wakeup-unsafe condvar wait |
| LOW | Latent hazard or convention violation with no current trigger | Orphanable task that today happens to finish first; missing cancellation propagation in a path that is never cancelled yet |
Escalate one level when the affected state is money, auth, or durability.
Finding format
[SEVERITY] file:line — short title
Race window / failure mode: the exact interleaving or condition (T1 does X,
T2 does Y between X and Z → consequence).
Trigger likelihood: what load/error pattern makes it fire.
Fix: concrete minimal change (primitive, bound, timeout value, scope).
Rules index
| File | Read this when... |
|---|---|
rules/01-models-and-structure.md |
Choosing event loop vs threads vs processes vs actors; CPU/I-O decision tree; structured concurrency, task groups, no orphaned tasks |
rules/02-correctness.md |
Reasoning about data races vs race conditions, atomicity, memory ordering/visibility, TOCTOU, deadlock prevention, livelock, starvation, idempotency under retries |
rules/03-primitives.md |
Picking or reviewing mutexes, RwLocks, semaphores, condition variables, channels (bounded vs unbounded), select/race, once/lazy init |
rules/04-event-loop-hygiene.md |
Anything runs on an event loop: blocking calls, CPU work, offloading to pools, long-task chunking, microtask vs macrotask |
rules/05-cancellation-timeouts-shutdown.md |
Timeout policy, propagating cancellation (context/AbortSignal/CancelledError), cleanup on cancel, graceful shutdown sequencing |
rules/06-backpressure-flow-control.md |
Queues between components, producer/consumer rate mismatch, load shedding, token buckets, pull-based streaming |
rules/07-audit-bug-catalog.md |
AUDIT mode: the signature, severity, and fix for every common async bug — fire-and-forget, missing await, lock-across-await, retry storms, thundering herd, unbounded spawns |
Top 10 non-negotiables
- No orphaned tasks. Every spawned task has an owner that awaits/joins it or cancels it on scope exit. Fire-and-forget requires an error handler and a written reason.
- No unbounded queues or channels. An unbounded queue is a memory leak with extra steps. Choose a capacity and a full-policy (block, drop, shed).
- Never block the event loop. No sync I/O, sync crypto, or CPU loops on the loop thread — offload to a worker pool.
- Never hold a lock across an await point. It serializes the system at best and deadlocks it at worst.
- Every await has a timeout policy. External calls get explicit deadlines; internal ones inherit a scope deadline. "Forever" is a decision, not a default.
- No check-then-act on shared state. Make the check and the act atomic: one lock region, an atomic primitive, or a DB constraint/UPSERT.
- Acquire locks in one global order, and never call unknown/user code while holding a lock.
- Cancellation propagates and cleans up. Catch-and-rethrow CancelledError; pass context/AbortSignal down every call chain that can block.
- Retries are bounded, jittered, and idempotent. Exponential backoff + full jitter + retry budget; never retry a non-idempotent operation without a dedupe key.
- Condition-variable waits loop on the predicate (
while not pred: wait()), and shutdown follows the sequence: stop accepting → drain with deadline → cancel stragglers → release resources.