Python Async
Asyncio gives one thread the throughput of hundreds of connections, but only if nothing on the event loop ever blocks - a single stray time.sleep or requests.get freezes every coroutine in the process. The costly mistakes this skill prevents are the silent ones: code that runs correctly but sequentially (awaiting in a loop), tasks that vanish mid-flight (un-referenced fire-and-forget), and a "fast" service whose p99 collapses because one handler does CPU work on the loop.
Mental model
One event loop runs many coroutines cooperatively on a single thread. await is the only place control can switch; between awaits, your code has an exclusive lock on the whole process. Concurrency comes from running tasks, not from awaiting sequentially - await a(); await b() is exactly as slow as sync code.
Operating procedure
Step 1: Gather inputs
- Python version - 3.11+ unlocks
TaskGroup and asyncio.timeout; below that, use gather and wait_for.
- The workload type per operation: network I/O (async-friendly), disk/blocking-library I/O (needs
to_thread), or CPU-bound (needs a process pool).
- Every library on the I/O path, checked for an async API:
httpx/aiohttp not requests, asyncpg not psycopg2, asyncio.sleep not time.sleep. If unsure whether a call blocks, treat it as blocking.
- The concurrency target and the downstream limits (DB pool size, API rate limits).
Step 2: Choose the concurrency primitive
- Known, finite set of operations that should all complete or all fail together:
asyncio.TaskGroup (3.11+). It cancels siblings on failure and propagates exceptions as a group - the structured default.
- Need the list of results in order, possibly tolerating individual failures:
asyncio.gather(..., return_exceptions=True), then check each result for an exception.
- Background work that outlives the current function (consumers, listeners):
asyncio.create_task, and store the reference - the loop holds tasks weakly, so an unreferenced task can be garbage-collected mid-flight. Keep a set, add a done-callback that logs exceptions and discards the task.
- Decision rule:
gather/TaskGroup when the caller waits for the results here; create_task only when the work genuinely detaches from the current scope.
import asyncio
async def main():
async with asyncio.TaskGroup() as tg:
t1 = tg.create_task(fetch(url1))
t2 = tg.create_task(fetch(url2))
# both done here; any failure cancelled the sibling and raised
asyncio.run(main())
Step 3: Bound the parallelism
Unbounded fan-out is a self-inflicted DDoS on your own database or a partner API.
asyncio.Semaphore(n) around the operation, with n sized to the bottleneck (e.g. DB pool size minus headroom, or the API's rate limit).
- For pipelines, a bounded
asyncio.Queue(maxsize=...) between producer and consumers - the maxsize is your backpressure.
Step 4: Keep blocking work off the loop
- Blocking I/O (sync drivers, file reads, DNS in odd libraries):
await asyncio.to_thread(blocking_fn, arg).
- CPU-bound work (parsing, crypto, image processing):
ProcessPoolExecutor via loop.run_in_executor - threads don't help CPU work under the GIL.
- Rule of thumb: anything that could hold the loop for more than ~50ms doesn't belong on it.
- Call async from sync only at the top level via
asyncio.run; never nest event loops.
Step 5: Handle cancellation and timeouts deliberately
- Every external await gets a timeout:
async with asyncio.timeout(5): (3.11+) or asyncio.wait_for. Untimed awaits are where services hang forever.
- Cancellation raises
CancelledError at the next await point. Never swallow it - catch, clean up in try/finally, re-raise.
asyncio.shield only for genuinely must-finish work (e.g. committing a payment record), and sparingly - shielded work outlives its caller.
Step 6: Verify under the debug loop
Run tests with PYTHONASYNCIODEBUG=1 (or loop.set_debug(True)): it reports coroutines never awaited and callbacks that held the loop longer than 100ms - the two failure classes that are invisible in normal runs. Use async with/async for for async context managers and iterators; a bare with around an async resource leaks it.
Worked artifact: the three classic bugs, bad/good
Bad - sequential awaits masquerading as concurrency; blocking call on the loop; lost task:
async def handler(urls):
results = []
for u in urls: # runs one at a time
results.append(await fetch(u))
data = requests.get(META_URL) # freezes the entire event loop
asyncio.create_task(audit_log(data)) # unreferenced: may vanish silently
return results
Good:
import asyncio, httpx
_background: set[asyncio.Task] = set()
def spawn(coro):
t = asyncio.create_task(coro)
_background.add(t)
t.add_done_callback(lambda t: (_background.discard(t),
t.exception() and log.error("bg task failed", exc_info=t.exception())))
return t
async def handler(urls, client: httpx.AsyncClient):
sem = asyncio.Semaphore(10) # bounded fan-out
async def bounded(u):
async with sem:
async with asyncio.timeout(5):
return await fetch(client, u)
async with asyncio.TaskGroup() as tg:
tasks = [tg.create_task(bounded(u)) for u in urls]
data = await client.get(META_URL) # async client, not requests
spawn(audit_log(data)) # referenced, exceptions logged
return [t.result() for t in tasks]
Deliverable
Produce working async code (or a review) in which every concurrent operation uses the right primitive with a one-line justification, every external await has a timeout, fan-out is bounded with a number tied to the downstream limit, no blocking call runs on the loop, and background tasks are referenced with exception logging.
Do NOT
- Do not call blocking I/O (
requests, time.sleep, sync DB drivers) inside a coroutine - one call stalls every request in the process.
- Do not await in a loop when the operations are independent; that is sync code with extra syntax.
- Do not fire-and-forget with a bare
create_task - hold the reference and log its exceptions.
- Do not swallow
CancelledError; it breaks timeouts and shutdown for everything above you.
- Do not use threads for CPU-bound work - the GIL makes that a no-op; use a process pool.
- Do not create event loops manually or nest
asyncio.run; one loop, entered once at the top.
- Do not fan out unbounded to a database or API; size a semaphore to the real limit.
Quality bar
- Every
await on an external system sits under an explicit timeout.
gather vs TaskGroup vs create_task choice is justified per call site.
- Grep for
requests., time.sleep, and sync DB drivers inside async def comes back empty.
- Concurrency bounds are numbers with a stated source, not defaults.
- The test suite passes with
PYTHONASYNCIODEBUG=1 and no slow-callback warnings.
Escalation
For the HTTP API's shape rather than its concurrency, use api-design. For rate-limit handling strategy against third parties, pair with rate-limit-handler. Queue-based cross-service pipelines belong to kafka-pipelines.
1---2name: python-async3description: Writes and reviews correct asyncio code - structured concurrency with TaskGroup, gather vs create_task decisions, cancellation and timeout handling, bridging blocking code with to_thread, and bounded parallelism - with good/bad pairs for the pitfalls that silently freeze the event loop. Use when someone asks "why is my async code slow", "gather or create_task", "how do I run these requests concurrently", "my FastAPI endpoint blocks everything", or is converting sync Python to asyncio. Do NOT use for general pandas or data-analysis performance - use pandas-expert instead - and do NOT use for API endpoint design itself - use api-design instead.4metadata:5 source: https://skillmd.com/skills/skillmedev/python-async6---78# Python Async910Asyncio gives one thread the throughput of hundreds of connections, but only if nothing on the event loop ever blocks - a single stray `time.sleep` or `requests.get` freezes every coroutine in the process. The costly mistakes this skill prevents are the silent ones: code that runs correctly but sequentially (awaiting in a loop), tasks that vanish mid-flight (un-referenced fire-and-forget), and a "fast" service whose p99 collapses because one handler does CPU work on the loop.1112## Mental model1314One event loop runs many coroutines cooperatively on a single thread. `await` is the only place control can switch; between awaits, your code has an exclusive lock on the whole process. Concurrency comes from running tasks, not from awaiting sequentially - `await a(); await b()` is exactly as slow as sync code.1516## Operating procedure1718### Step 1: Gather inputs19201. Python version - 3.11+ unlocks `TaskGroup` and `asyncio.timeout`; below that, use `gather` and `wait_for`.212. The workload type per operation: network I/O (async-friendly), disk/blocking-library I/O (needs `to_thread`), or CPU-bound (needs a process pool).223. Every library on the I/O path, checked for an async API: `httpx`/`aiohttp` not `requests`, `asyncpg` not `psycopg2`, `asyncio.sleep` not `time.sleep`. If unsure whether a call blocks, treat it as blocking.234. The concurrency target and the downstream limits (DB pool size, API rate limits).2425### Step 2: Choose the concurrency primitive2627- Known, finite set of operations that should all complete or all fail together: `asyncio.TaskGroup` (3.11+). It cancels siblings on failure and propagates exceptions as a group - the structured default.28- Need the list of results in order, possibly tolerating individual failures: `asyncio.gather(..., return_exceptions=True)`, then check each result for an exception.29- Background work that outlives the current function (consumers, listeners): `asyncio.create_task`, and store the reference - the loop holds tasks weakly, so an unreferenced task can be garbage-collected mid-flight. Keep a set, add a done-callback that logs exceptions and discards the task.30- Decision rule: `gather`/`TaskGroup` when the caller waits for the results here; `create_task` only when the work genuinely detaches from the current scope.3132```python33import asyncio3435async def main():36 async with asyncio.TaskGroup() as tg:37 t1 = tg.create_task(fetch(url1))38 t2 = tg.create_task(fetch(url2))39 # both done here; any failure cancelled the sibling and raised4041asyncio.run(main())42```4344### Step 3: Bound the parallelism4546Unbounded fan-out is a self-inflicted DDoS on your own database or a partner API.4748- `asyncio.Semaphore(n)` around the operation, with n sized to the bottleneck (e.g. DB pool size minus headroom, or the API's rate limit).49- For pipelines, a bounded `asyncio.Queue(maxsize=...)` between producer and consumers - the maxsize is your backpressure.5051### Step 4: Keep blocking work off the loop5253- Blocking I/O (sync drivers, file reads, DNS in odd libraries): `await asyncio.to_thread(blocking_fn, arg)`.54- CPU-bound work (parsing, crypto, image processing): `ProcessPoolExecutor` via `loop.run_in_executor` - threads don't help CPU work under the GIL.55- Rule of thumb: anything that could hold the loop for more than ~50ms doesn't belong on it.56- Call async from sync only at the top level via `asyncio.run`; never nest event loops.5758### Step 5: Handle cancellation and timeouts deliberately5960- Every external await gets a timeout: `async with asyncio.timeout(5):` (3.11+) or `asyncio.wait_for`. Untimed awaits are where services hang forever.61- Cancellation raises `CancelledError` at the next await point. Never swallow it - catch, clean up in `try/finally`, re-raise.62- `asyncio.shield` only for genuinely must-finish work (e.g. committing a payment record), and sparingly - shielded work outlives its caller.6364### Step 6: Verify under the debug loop6566Run tests with `PYTHONASYNCIODEBUG=1` (or `loop.set_debug(True)`): it reports coroutines never awaited and callbacks that held the loop longer than 100ms - the two failure classes that are invisible in normal runs. Use `async with`/`async for` for async context managers and iterators; a bare `with` around an async resource leaks it.6768## Worked artifact: the three classic bugs, bad/good6970Bad - sequential awaits masquerading as concurrency; blocking call on the loop; lost task:7172```python73async def handler(urls):74 results = []75 for u in urls: # runs one at a time76 results.append(await fetch(u))77 data = requests.get(META_URL) # freezes the entire event loop78 asyncio.create_task(audit_log(data)) # unreferenced: may vanish silently79 return results80```8182Good:8384```python85import asyncio, httpx8687_background: set[asyncio.Task] = set()8889def spawn(coro):90 t = asyncio.create_task(coro)91 _background.add(t)92 t.add_done_callback(lambda t: (_background.discard(t),93 t.exception() and log.error("bg task failed", exc_info=t.exception())))94 return t9596async def handler(urls, client: httpx.AsyncClient):97 sem = asyncio.Semaphore(10) # bounded fan-out98 async def bounded(u):99 async with sem:100 async with asyncio.timeout(5):101 return await fetch(client, u)102 async with asyncio.TaskGroup() as tg:103 tasks = [tg.create_task(bounded(u)) for u in urls]104 data = await client.get(META_URL) # async client, not requests105 spawn(audit_log(data)) # referenced, exceptions logged106 return [t.result() for t in tasks]107```108109## Deliverable110111Produce working async code (or a review) in which every concurrent operation uses the right primitive with a one-line justification, every external await has a timeout, fan-out is bounded with a number tied to the downstream limit, no blocking call runs on the loop, and background tasks are referenced with exception logging.112113## Do NOT114115- Do not call blocking I/O (`requests`, `time.sleep`, sync DB drivers) inside a coroutine - one call stalls every request in the process.116- Do not await in a loop when the operations are independent; that is sync code with extra syntax.117- Do not fire-and-forget with a bare `create_task` - hold the reference and log its exceptions.118- Do not swallow `CancelledError`; it breaks timeouts and shutdown for everything above you.119- Do not use threads for CPU-bound work - the GIL makes that a no-op; use a process pool.120- Do not create event loops manually or nest `asyncio.run`; one loop, entered once at the top.121- Do not fan out unbounded to a database or API; size a semaphore to the real limit.122123## Quality bar124125- Every `await` on an external system sits under an explicit timeout.126- `gather` vs `TaskGroup` vs `create_task` choice is justified per call site.127- Grep for `requests.`, `time.sleep`, and sync DB drivers inside `async def` comes back empty.128- Concurrency bounds are numbers with a stated source, not defaults.129- The test suite passes with `PYTHONASYNCIODEBUG=1` and no slow-callback warnings.130131## Escalation132133For the HTTP API's shape rather than its concurrency, use api-design. For rate-limit handling strategy against third parties, pair with rate-limit-handler. Queue-based cross-service pipelines belong to kafka-pipelines.