Auto Performance

Auto Performance

Overview

Turn "make it faster" into a disciplined, measured loop: define the metric, baseline it, profile to the real hotspots, and improve them one at a time — each improvement earning its place with a before/after number and a clean correctness check, or getting reverted. The measurement gate is the whole point: it's what separates real optimization from complexity-adding cargo-culting.

Phase 0: Define the metric and baseline it (measure first)

• Pin the METRIC to the target: latency (p50/p95/p99), throughput, memory/allocations, bundle size, or a web vital (LCP/CLS/INP) — with a numeric target if one exists (from the spec or a budget).
• Build a REPRODUCIBLE benchmark for it: fixed input, controlled warm/cold state, enough iterations to see past variance. Record the baseline (with its variance/spread — a single number isn't enough).
• Open a checkpoint-resume run.

Success criteria: a metric, a numeric target (or "reduce X"), and a reproducible baseline with known variance.

Phase 1: Profile to the real hotspots

• Run a profiler / measurement appropriate to the stack (CPU/alloc profiler, query logs, flame graph, bundle analyzer, DevTools performance trace). Identify where the time/memory actually goes.
• Rank hotspots by contribution to the metric. Ignore cold paths — optimizing them is wasted complexity.
• For each hotspot, note the likely class (algorithmic complexity, N+1/roundtrips, needless allocation, blocking I/O, re-render/re-compute, oversized payload).

Success criteria: a ranked list of the hotspots that actually move the metric, each with a suspected cause.

Phase 2: Optimize one hotspot, prove it (converge, benchmark-gated)

Run converge-loop toward the target; per hotspot (highest-impact first):

1. Hypothesize the change and the expected effect on the metric.
2. Apply the smallest optimization that tests the hypothesis.
3. Re-benchmark and verify (adversarial-verify, two lenses):
   • measurement lens — is the delta real (beyond variance, apples-to-apples)? or noise / a broken benchmark?
   • regression lens — does the full relevant test suite still pass? does any edge case now behave differently (the classic "optimized the happy path, broke the boundary")?
4. Accept or revert — keep ONLY if the improvement is real AND correctness holds. Otherwise revert and record why (no gain / regressed / not worth the complexity). Update the checkpoint.

Exit when the target is met, or when the remaining hotspots offer gains too small to justify their complexity cost.

Success criteria: each kept change has a measured before/after and a clean correctness check; unproven changes are reverted.

Phase 3: Report

Close the checkpoint and report: baseline → final metric (with the real delta and variance), the optimizations kept (each with its measured gain), the ones tried-and-reverted (why), and whether the target was met or where it stalled and why.

Success criteria: an honest, measured account — no claimed gain lacks a number.

Common Rationalizations

Red Flags

• A change accepted with no before/after measurement.
• Optimizing code the profiler didn't flag as hot.
• A benchmark that isn't apples-to-apples (different input, warm vs cold, changed between runs).
• Correctness tests not re-run after an optimization (or a subtle edge-case behavior change ignored).
• Claimed speedups within run-to-run variance.
• Complexity piled on for sub-threshold gains past the target.

Guardrails

• Never optimize without a baseline; never accept a change without a re-benchmark showing a real delta.
• Never optimize a non-hotspot; profile first.
• Never keep an optimization that regresses correctness; revert on any behavioral change.
• Never claim an unmeasured improvement.
• Stop at the target / diminishing returns — don't trade clarity for noise-level gains.

When To Load References

• converge-loop (skill) — the optimize-toward-target loop with termination + anti-thrash.
• adversarial-verify (skill) — the measurement + regression lenses that gate each accept.
• checkpoint-resume (skill) — durable perf-run state.
• auto-test (skill) — ensure a correctness safety net exists before optimizing under-covered code.

Output Contract

Report:

1. metric + target; baseline (with variance) → final (with the real delta)
2. optimizations kept — each with its measured gain and the hotspot it addressed
3. optimizations tried and reverted — with why (no gain / noise / regression / not worth complexity)
4. target met, or where it stalled and the reason