# Coding Performance Review

> Use when slow, or to profile, benchmark or review performance before optimising; when a speedup or an A/B win could be noise; on latency, throughput, N+1 queries, unbounded memory, loops doing per-item I/O, blocking calls, repeated work or algorithmic cost; when timing two versions, builds or configurations against each other, reading an A/B that shows no difference, or checking a speedup claimed in a diff or commit; and for any language without a performance skill of its own (bash, JavaScript, Go, SQL). Routes Python to coding-python-performance-review and Rust to coding-rust-performance-review.

- Skill: `bitranox/coding-performance-review` (Agent Skill, multi-file: 12 files)
- Install (CLI): `npx get-skill add bitranox/coding-performance-review`
- Raw SKILL.md: https://api.skillmd.com/api/skills/bitranox/coding-performance-review/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Data & Analytics
- Author: bitranox (https://skillmd.com/u/bitranox)
- Updated: 2026-11-14
- Page: https://skillmd.com/skills/bitranox/coding-performance-review

---


# Performance review and measurement (any language)

Two halves: how to measure so that a number means what you think it means, and what to look for
in code. The language skills add their own tools on top of this one; the method is shared.

## Route by language

| Code under review | Load as well                                                                  |
|-------------------|-------------------------------------------------------------------------------|
| Python            | `bitranox:coding-python-performance-review` (profiling pipeline, AST finders) |
| Rust              | `bitranox:coding-rust-performance-review` (clippy, criterion, profilers)      |
| anything else     | nothing: this skill alone                                                     |

## Method: a number you can believe

1. **Profile before optimising.** Find where the time goes on a real workload (the project's own
   test suite, a recorded production input, the real request), not on a hand-made
   micro-benchmark. An optimisation outside the hot path is cost without benefit.
2. **Reproduce the recorded BEFORE figure first.** When the defect already has a measured before
   number (an issue, a backlog line, a saved profile), run your probe on the unfixed code and
   require it to land within an order of magnitude of that figure before you trust your after
   number. A probe that hits an early return is fast in both arms, so the fix looks enormous
   while the slow path never ran. If BEFORE is far too fast, count calls into the hot function
   to find the guard that returns early, change the input so it reaches the slow path, repeat
   this check, and leave a comment in the test saying why its input has that shape. When there is
   no recorded figure, only a claim ("20 percent faster"), rebuild the before arm yourself from
   the commit or diff the claim is about, run both arms on the same input (save one first when
   none is recorded, per step 1), and apply this check to the before figure you measured.
3. **Identical settings on both sides.** Same build mode and flags, input, data size, warm or cold
   state, host and concurrency. Diff what each side RESOLVED, defaults included, not its config
   file.
4. **Interleave the arms: A, B, A, B, at least three runs each.** All-A-then-all-B confounds the
   arm with the clock (thermal state, other load, caches). Report the median and the spread per
   arm; a difference smaller than the spread within one arm is noise. On a shared or loaded host
   a wall-clock suite timing cannot resolve a few-percent change: compare user CPU
   (`/usr/bin/time -f %U`) on a serial subset that exercises the hot path, with the old arm in a
   detached worktree proven to run its own code.
5. **Pin the core on a hybrid CPU.** On a CPU with performance and efficiency cores (on Linux,
   compare `cpuinfo_max_freq` across `/sys/devices/system/cpu/cpu*/cpufreq/`), identical code
   spreads widely by core placement alone. Pin each run (`taskset -c N`, `sched_setaffinity`) or
   record which core type it ran on.
6. **Prove the change took effect.** Before believing an A/B, above all a null one, show that the
   variable moved inside the system under test (the cache was hit, the flag was read, the new
   build is the one that ran) and that nothing else moved.
7. **Do not write off an approach measured with your own constraint still on it.** List every
   constraint you added to make a variant correct (a lock, serialisation, a cap) and measure it
   without the incidental ones before calling the approach slow.
8. **Measure a concurrency or queueing change; never predict it** from the architecture. The
   cost often sits somewhere the design diagram does not show.
9. **Read current load from two samples, not from a lifetime average.** `ps -o pcpu` averages
   over the whole life of the process; use the second sample of `top -b -n2`, `pidstat`, or two
   reads of `/proc/<pid>/stat`.
10. **Prove a speed rewrite equal before timing it.** A rewrite that changes HOW the work runs
    (compile once instead of re-parsing a program per call) needs a differential run: old and new
    on random valid AND malformed inputs, comparing the result or the (exception type, message),
    plus a planted one-character bug as the control that must show differences. Unit tests often
    pass in every variant. If the old code evaluated while it parsed, prepare each constant while
    reading its term, not lazily at first use: lazy preparation changes which error a malformed
    input raises first.

    Before believing a zero-difference result, tabulate per input kind how many inputs were
    refused and by which exception type. A valid kind refused 100 percent, or any kind refused
    only by a type the code under test does not raise for it, means the harness answered, not the
    code: a `TypeError` from passing a keyword-only argument positionally makes both trees refuse
    every call and report 0 differences. Measured: 500 of 500 and 9000 of 9000 refused, 0 differ;
    with the call fixed, 500 parsed, 4380 refused, 0 differ, and the plants fired.

    Run the control as a real copy of the tree carrying the planted bug, assert the plant's anchor
    occurs exactly once, and compare its output per kind with the real run. Output identical to
    the real run means the control did not run or the plant cannot change behaviour: check the
    plant first (padding a value to a width no larger than the value is an equivalent mutant),
    then treat the harness as faulty. Measured: a `--plant` flag existed only in a script's
    docstring, argv ignored it and the run printed the same table; a real planted tree showed
    28192 differences across 7 kinds.
11. **Re-measure a candidate on top of what you already adopted.** Before dropping a candidate
    that shows nothing against today's code, re-measure it on the tree holding the changes already
    adopted: less remaining work makes a fixed saving a larger share. Re-measure one that wins
    alone there too, since an adopted change may already have removed the work it saves. Measured:
    one cache read went from -1.6 percent alone (noise) to -5.6 percent on the adopted tree; two
    other candidates, +0.2 and -0.5 percent alone, stayed no-gain there.
12. **Check every speed claim in a diff.** `validate_perf_claims.py` (in this skill's directory)
    extracts phrases like "40% faster", "3x speedup" or "reduced latency by 30%" from a unified
    diff's added lines and prints each with its `path:line`. It reaches no verdict: check each
    claim against a measured run and report the ones that are unproven or contradicted.

```bash
git diff origin/main...HEAD > changes.diff
python3 "<this skill's directory>/validate_perf_claims.py" changes.diff   # 0 scanned, 2 unreadable
```

Report every finding with the measured number, the spread, the runs per arm and the command that
produced it. A figure you did not run yourself is labelled as read or estimated, never as
measured.

## What to flag in code (any language)

Severity: SEVERE can exhaust memory or time on real input; MEDIUM costs measurably on a hot
path; MINOR is cheap to fix but not shown to be hot. Each finding names the input size at which
it starts to hurt.

- **Unbounded memory (usually SEVERE).** Reading a big file, a huge query result or a huge log
  fully into memory, or accumulating an unbounded list, map or string, instead of streaming,
  iterating, chunking or paginating. Only a provably small, fixed-size input may be materialised.
- **Unbounded or unsafe input.** Input with no length bound, unvalidated types, or unhandled
  encodings and characters (non-ASCII, emoji, CJK, control characters, binary). Adversarial input
  must not blow up time or memory (catastrophic regex backtracking, quadratic blow-ups,
  out-of-memory); the bound is validated and tested.
- **Untrusted structured input.** A map, JSON document or deserialised payload used without
  parsing or validating its shape at the boundary, unless the project has deliberately accepted
  skipping it.
- **Quadratic work.** A nested loop over the same collection, a membership test against a list
  inside a loop, a repeated sort or search inside a loop, or string building by repeated
  concatenation where the language copies on every step.
- **N+1 calls.** A query, HTTP request, subprocess or RPC per item where one batched call would
  do.
- **Per-item I/O.** An open, close, flush, fsync or system call per record; unbuffered writes.
- **Repeated pure work.** The same parse, compile (a regex, a template) or lookup redone on every
  call with the same input. Hoist or cache it after confirming the work is pure and the hit rate
  is real. A result derived from mutable I/O (a file's line count, an upstream response) is not
  pure: key the cache on the file's length and mtime, or state the immutability assumption.
- **Blocking calls on a hot or async path.** A sleep, synchronous network or disk I/O inside an
  event loop, or a lock held while waiting.
- **Eager work that is mostly thrown away.** Building a full list to take its first element, or
  loading every column to use one.

## Common mistakes

| Mistake                                          | Fix                                                                    |
|--------------------------------------------------|------------------------------------------------------------------------|
| Optimising before profiling                      | Profile the real workload first; fix only the hot path                 |
| Timing all A runs, then all B runs               | Interleave A, B, A, B, at least three each                             |
| Trusting an after figure on its own              | Reproduce the recorded before figure with the same probe               |
| Accepting "N times faster" from a commit message | Measure it; `validate_perf_claims.py` scans diffs, not commit messages |
| Reading a null A/B as "no effect"                | Prove the variable moved and its mechanism fired                       |
| Quoting `ps` %CPU as current load                | Take two samples                                                       |

