Python Performance
This skill is measurement-first optimization in CPython.
If the task is idioms, typing, or packaging, use python-patterns. If it is asyncio orchestration, use python-async-patterns. Primitive choice (which executor, queue shutdown, start method) lives in python-patterns concurrency.md; this skill is cost, GIL, and measurement.
When to activate
- Confirming a latency, throughput, CPU, or memory regression in Python
- Deciding whether threads, processes, subinterpreters, or a free-threaded build will actually parallelize
- Profiling a hot path before and after a change
- Evaluating experimental JIT or other runtime knobs
Rules of engagement
- Profile before changing code. Algorithm and I/O dominate micro-tweaks.
- Change one variable at a time. Keep a before/after number.
- Optimize the proven hot path, not code that merely looks slow.
- Do not treat free-threading or the JIT as default. Both are opt-in; JIT is experimental.
Outcome expectations
- Claims have a reproducible before/after measurement.
- Parallelism choice matches GIL state (
sysconfig.get_config_var("Py_GIL_DISABLED") vs sys._is_gil_enabled()).
- Experimental flags stay labeled experimental.
Workflow
- Make the symptom measurable (
timeit, a benchmark script, or a representative load).
- Capture evidence: CPU (
cProfile / py-spy), allocations (tracemalloc).
- Classify: algorithm, I/O wait, GIL serialization, allocation churn, or lock contention.
- Apply one targeted fix. Re-measure.
- Only then consider runtime builds (free-threaded) or experimental JIT.
Symptom to first tool
- High CPU in Python frames ->
cProfile or py-spy record
- Many cores idle on threaded CPU work -> GIL / parallelism (
gil-and-parallelism.md)
- RSS growth or allocation churn ->
tracemalloc, then allocations-gc.md
- Event-loop stalls ->
python-async-patterns (don't block the loop); profile with py-spy
- Shipped binary already profile-tuned ->
compiler-and-runtime.md
Resources
Load on demand:
references/profiling.md — load when choosing cProfile, py-spy, scalene, or tracemallocreferences/gil-and-parallelism.md — load for GIL, free-threading (3.13 experimental / 3.14 supported optional), InterpreterPoolExecutor, multiprocessing vs threadsreferences/allocations-gc.md — load for allocation churn, gc, slots, cachesreferences/compiler-and-runtime.md — load for experimental JIT (PYTHON_JIT), tail-call interpreter, PGO; not for first-pass code fixes
1---2name: python-performance3description: Python performance workflow: measure with profilers, then cut allocations, pick the right parallelism (threads vs processes vs subinterpreters vs free-threading), and only then touch experimental JIT/runtime knobs. Use after you have evidence CPython is the bottleneck — not for style or API design.4license: MIT5---67# Python Performance89This skill is **measurement-first optimization** in CPython.1011If the task is idioms, typing, or packaging, use `python-patterns`. If it is asyncio orchestration, use `python-async-patterns`. Primitive choice (which executor, queue shutdown, start method) lives in `python-patterns` `concurrency.md`; this skill is cost, GIL, and measurement.1213## When to activate1415- Confirming a latency, throughput, CPU, or memory regression in Python16- Deciding whether threads, processes, subinterpreters, or a free-threaded build will actually parallelize17- Profiling a hot path before and after a change18- Evaluating experimental JIT or other runtime knobs1920## Rules of engagement2122- **Profile before changing code.** Algorithm and I/O dominate micro-tweaks.23- **Change one variable at a time.** Keep a before/after number.24- Optimize the proven hot path, not code that merely looks slow.25- Do not treat free-threading or the JIT as default. Both are opt-in; JIT is experimental.2627## Outcome expectations2829- Claims have a reproducible before/after measurement.30- Parallelism choice matches GIL state (`sysconfig.get_config_var("Py_GIL_DISABLED")` vs `sys._is_gil_enabled()`).31- Experimental flags stay labeled experimental.3233## Workflow34351. Make the symptom measurable (`timeit`, a benchmark script, or a representative load).362. Capture evidence: CPU (`cProfile` / `py-spy`), allocations (`tracemalloc`).373. Classify: algorithm, I/O wait, GIL serialization, allocation churn, or lock contention.384. Apply one targeted fix. Re-measure.395. Only then consider runtime builds (free-threaded) or experimental JIT.4041## Symptom to first tool4243- High CPU in Python frames -> `cProfile` or `py-spy record`44- Many cores idle on threaded CPU work -> GIL / parallelism (`gil-and-parallelism.md`)45- RSS growth or allocation churn -> `tracemalloc`, then `allocations-gc.md`46- Event-loop stalls -> `python-async-patterns` (don't block the loop); profile with `py-spy`47- Shipped binary already profile-tuned -> `compiler-and-runtime.md`4849## Resources5051Load on demand:5253- `references/profiling.md` — load when choosing cProfile, py-spy, scalene, or tracemalloc54- `references/gil-and-parallelism.md` — load for GIL, free-threading (3.13 experimental / 3.14 supported optional), `InterpreterPoolExecutor`, multiprocessing vs threads55- `references/allocations-gc.md` — load for allocation churn, `gc`, `slots`, caches56- `references/compiler-and-runtime.md` — load for experimental JIT (`PYTHON_JIT`), tail-call interpreter, PGO; not for first-pass code fixes
AeonDave