Profile before you optimise
Performance work without a measurement is refactoring with a story attached. Reproduce
the slowness, profile it, fix what the profile points at, and show the same measurement
afterwards. Most wins come from doing less work (fewer queries, allocations, bytes), not
cleverer code. Sources: Brendan Gregg (USE method, flame graphs); Gil Tene on coordinated
omission; Google SRE book ch. 6 (percentiles, not averages); the benchstat/hyperfine
documentation on repeated runs and variance.
Steps
- Define the number and the target. Metric, percentile, load and environment:
"p95 of
GET /orders ≤ 200 ms at 50 req/s open-model load", "job wall time",
"heap after 10k iterations", "gzipped bundle bytes". Averages hide tails; report
p50/p95/p99. Write it down before changing code; the acceptance criterion should be
this number. Call search_lore for known hot spots and past decisions.
- Build a reproducible harness. Use the repo's tooling (
autocannon, k6,
wrk2, pytest-benchmark, go test -bench, hyperfine, JMH, criterion): fixed
inputs, realistic data volume, warm-up, and enough repetitions to see variance
(≥ 10 runs or the tool's statistical mode). For latency under load use an
open-model / constant-arrival-rate generator (k6 constant-arrival-rate, wrk2 -R): closed-loop tools that wait for each response under-report tail latency
(coordinated omission). Record the exact command and machine. Use only tools already
installed; never install one (the hook blocks npm/pip installs, go install,
cargo install, uv add, pipx run and npx -y; a plain fetching npx <tool>
and uv sync are not blocked but are equally forbidden — run npx --no -- <tool>;
node_modules is shared). If a needed tool is missing, say so in the evidence and
use what exists.
- Find the bottleneck resource first. For each resource (CPU, memory, disk,
network, connection pools, locks) check utilisation, saturation and errors (USE). A
request waiting on a pool or a lock is not a CPU problem.
- Profile, do not guess. CPU:
node --cpu-prof, py-spy record, go tool pprof,
perf + flame graph, async-profiler. Memory: heap snapshots / allocation profiles
taken at two points to see what grows. Database: query log with timings and
per-request query count. Frontend: the bundler's analyzer. Fix the widest frame in the
flame graph; if the profile disagrees with your hunch, the profile is right.
- Fix the top item with the least clever change. Usual order: remove an N+1 (batch
or join), add the index the plan wants (the
sql-query-performance skill), stop
serialising unused data, stream instead of buffering, cache a pure or hot read (the
caching-strategy skill), move work off the request path (the background-jobs
skill). Algorithmic rewrites and micro-optimisations last.
- Keep behaviour identical, including under concurrency. Run the full suite. A
new cache, batch boundary, memoisation or parallelism is a new code path: test it
directly, including concurrent callers (stale reads, lost updates, shared mutable
state; the
concurrency-and-async skill). Never drop a validation, authorisation
check or lock to win time.
- Measure again under identical conditions and compare distributions, not single
runs: same command, data and machine, repeated; use the tool's comparison
(
benchstat old.txt new.txt, hyperfine with several commands) and report the
change with its variance. A difference within noise is not a result. If the target
is not met, say so and what the profile shows now.
- Guard the win with a deterministic check. Prefer assertions that do not flake on
shared CI hardware: query count per request, allocation count, bundle-size budget,
algorithmic complexity on a fixed input. Wall-clock assertions only with generous
margins or on dedicated runners.
Done when
Baseline and after numbers exist for the same command under the same conditions with
variance reported, the target is met (or the shortfall is stated with the current
profile), the suite passes, a regression guard is in place, and each acceptance
criterion has evidence via the record-evidence skill (command, baseline, profile top
entries, change, after numbers).
Anti-patterns
- Optimising code the profile shows is cold; "it looked slow".
- Benchmarks on ten rows, cold caches, or a single run; closed-loop latency tests.
- Averaging percentiles across runs or hosts (merge histograms instead).
- Caches with no invalidation story or no bound on size.
- Timing assertions that make CI flaky.
Review checklist
Capture lore
This skill is one of the places durable, reusable knowledge naturally surfaces:
While profiling you learn where this repo is actually hot and what a change to that path costs in measured terms. That kind of fact is lore. Capture it via the lore-capture
protocol in your brief (CLAUDE.factory.md, step 11 "Memory contribution"):
call the Memory MCP suggest_lore once at the close of your work — reusable
conventions, gotchas, decisions, and boundaries only, never per-ticket trivia.
1---2name: performance-profiling3description: Use when a ticket is about speed or resource use — a slow endpoint, a hot loop, memory growth, a test suite that takes too long, "make X faster" — and the change must be driven by a measurement, not a hunch, and proven with a before/after number under the same conditions. Invoke for latency, throughput, CPU, memory, bundle size, or startup-time work.4stack: [node, python, go, java, rust, csharp, kotlin, ruby, react, web]5area: backend6metadata:7 source: https://skillmd.com/skills/tmj-90/performance-profiling8---910# Profile before you optimise1112Performance work without a measurement is refactoring with a story attached. Reproduce13the slowness, profile it, fix what the profile points at, and show the same measurement14afterwards. Most wins come from doing less work (fewer queries, allocations, bytes), not15cleverer code. Sources: Brendan Gregg (USE method, flame graphs); Gil Tene on coordinated16omission; Google SRE book ch. 6 (percentiles, not averages); the `benchstat`/`hyperfine`17documentation on repeated runs and variance.1819## Steps20211. **Define the number and the target.** Metric, percentile, load and environment:22 "p95 of `GET /orders` ≤ 200 ms at 50 req/s open-model load", "job wall time",23 "heap after 10k iterations", "gzipped bundle bytes". Averages hide tails; report24 p50/p95/p99. Write it down before changing code; the acceptance criterion should be25 this number. Call `search_lore` for known hot spots and past decisions.262. **Build a reproducible harness.** Use the repo's tooling (`autocannon`, `k6`,27 `wrk2`, `pytest-benchmark`, `go test -bench`, `hyperfine`, JMH, `criterion`): fixed28 inputs, realistic data volume, warm-up, and enough repetitions to see variance29 (≥ 10 runs or the tool's statistical mode). For latency under load use an30 open-model / constant-arrival-rate generator (`k6` `constant-arrival-rate`, `wrk231 -R`): closed-loop tools that wait for each response under-report tail latency32 (coordinated omission). Record the exact command and machine. Use only tools already33 installed; never install one (the hook blocks npm/pip installs, `go install`,34 `cargo install`, `uv add`, `pipx run` and `npx -y`; a plain fetching `npx <tool>`35 and `uv sync` are not blocked but are equally forbidden — run `npx --no -- <tool>`;36 `node_modules` is shared). If a needed tool is missing, say so in the evidence and37 use what exists.383. **Find the bottleneck resource first.** For each resource (CPU, memory, disk,39 network, connection pools, locks) check utilisation, saturation and errors (USE). A40 request waiting on a pool or a lock is not a CPU problem.414. **Profile, do not guess.** CPU: `node --cpu-prof`, `py-spy record`, `go tool pprof`,42 `perf` + flame graph, `async-profiler`. Memory: heap snapshots / allocation profiles43 taken at two points to see what grows. Database: query log with timings and44 per-request query count. Frontend: the bundler's analyzer. Fix the widest frame in the45 flame graph; if the profile disagrees with your hunch, the profile is right.465. **Fix the top item with the least clever change.** Usual order: remove an N+1 (batch47 or join), add the index the plan wants (the `sql-query-performance` skill), stop48 serialising unused data, stream instead of buffering, cache a pure or hot read (the49 `caching-strategy` skill), move work off the request path (the `background-jobs`50 skill). Algorithmic rewrites and micro-optimisations last.516. **Keep behaviour identical, including under concurrency.** Run the full suite. A52 new cache, batch boundary, memoisation or parallelism is a new code path: test it53 directly, including concurrent callers (stale reads, lost updates, shared mutable54 state; the `concurrency-and-async` skill). Never drop a validation, authorisation55 check or lock to win time.567. **Measure again under identical conditions** and compare distributions, not single57 runs: same command, data and machine, repeated; use the tool's comparison58 (`benchstat old.txt new.txt`, `hyperfine` with several commands) and report the59 change with its variance. A difference within noise is not a result. If the target60 is not met, say so and what the profile shows now.618. **Guard the win with a deterministic check.** Prefer assertions that do not flake on62 shared CI hardware: query count per request, allocation count, bundle-size budget,63 algorithmic complexity on a fixed input. Wall-clock assertions only with generous64 margins or on dedicated runners.6566## Done when6768Baseline and after numbers exist for the same command under the same conditions with69variance reported, the target is met (or the shortfall is stated with the current70profile), the suite passes, a regression guard is in place, and each acceptance71criterion has evidence via the `record-evidence` skill (command, baseline, profile top72entries, change, after numbers).7374## Anti-patterns7576- Optimising code the profile shows is cold; "it looked slow".77- Benchmarks on ten rows, cold caches, or a single run; closed-loop latency tests.78- Averaging percentiles across runs or hosts (merge histograms instead).79- Caches with no invalidation story or no bound on size.80- Timing assertions that make CI flaky.8182## Review checklist8384- [ ] Metric, percentile, load and target stated before the change.85- [ ] Profile evidence identifies the fixed hotspot.86- [ ] Before/after from the same harness with repeated runs and variance.87- [ ] New code paths (cache, batch, parallelism) tested, including concurrent use.88- [ ] A deterministic regression guard exists.8990## Capture lore9192This skill is one of the places durable, reusable knowledge naturally surfaces:93**While profiling you learn where this repo is actually hot and what a change to that path costs in measured terms.** That kind of fact is *lore*. Capture it via the **lore-capture94protocol in your brief** (`CLAUDE.factory.md`, step 11 "Memory contribution"):95call the Memory MCP `suggest_lore` once at the close of your work — reusable96conventions, gotchas, decisions, and boundaries only, never per-ticket trivia.