Loop Optimization: Hand vs Compiler
A decision rule for the five common loop transformations. Its value
is knowing when manual application is redundant (the compiler already
does it) or harmful (it defeats the vectorizer or fools your
benchmark).
When To Use
- Reviewing a hot loop and a hand-rolled transform appears (unrolled
body, shift-instead-of-multiply, bespoke SIMD).
- Authoring a loop that profiling proved hot, deciding whether to
optimize it by hand.
- Pushing back on a "this is faster" claim about a loop micro-opt.
When NOT To Use
- The loop is not proven hot by a profiler. Optimize nothing first.
- Architecture-level performance (caching layers, sharding): use
Skill(pensive:architecture-review).
- Detecting complexity hotspots (O(n^2) shapes):
Skill(pensive:performance-review).
The decision rule
- Profile first. No loop transform without a hot loop proven by a
profiler.
- In compiled languages (C, C++, Rust), trust the compiler for
loop-invariant code motion and strength reduction: both run
automatically at
-O2/-O3, so the manual form is redundant. Leave
unrolling to the compiler as well. Unlike the other two it is not on
by default (GCC needs -funroll-loops), but the compiler owns the
profitability decision and manual unrolling routinely defeats the
auto-vectorizer.
- If a loop will not vectorize, fix aliasing (
restrict /
__restrict__) and loop shape first. Confirm with an optimization
report (-fopt-info-vec-missed, -Rpass-missed=loop-vectorize).
Reach for intrinsics last and accept the portability cost.
- The manual transforms that still pay: explicit SIMD on loops the
compiler misses, loop fusion (guard against register and cache
pressure), and multi-accumulator unrolling to break a floating-point
reduction chain the compiler legally will not reorder.
- In Python, the levers are: hoist invariants out of the loop,
vectorize via NumPy, fuse passes via numexpr/Numba. Do not hand-unroll
or hand-strength-reduce: the cost is bytecode dispatch, not loop
control.
- Validate every claimed speedup on production-distribution data.
Per-technique reality
| Technique |
Helps where |
When NOT to apply by hand |
| Unrolling |
C/C++/Rust FP reduction chains (multi-accumulator) |
Auto-vectorizable loops (defeats vectorizer); OOO CPUs; icache pressure; Python |
| SIMD / vectorization |
C/C++/Rust loops the compiler misses; Python via NumPy |
Before fixing aliasing/loop shape; short trip counts; unverified that emitted SIMD runs |
| Loop fusion |
Bandwidth-bound array loops; Python via numexpr/Numba |
When it spills registers or mixes strided access; compute-bound bodies; blocks vectorization |
| Hoisting (LICM) |
Python (no compiler does it); C/C++/Rust only when aliasing blocks the proof |
-O2+ compiled code: redundant and can lengthen live ranges |
| Strength reduction |
Compilers do it; near-useless by hand |
-O2+ compiled code: blocks the compiler's IV analysis and vectorization |
Two traps that invalidate "it is faster"
- Synthetic-benchmark trap. A loop micro-opt validated on reused, small,
or synthetic input can invert to slower on production data, because
synthetic input hides effects such as branch misprediction on real
value distributions. Benchmark on production-distribution data with
optimizer barriers, or do not claim the win.
- Emitted is not executed. Auto-vectorization fails silently. "The
compiler emitted SIMD" does not mean "SIMD ran." Confirm with codegen
or optimization reports, not source inspection.
Both traps tie into Skill(imbue:proof-of-work): a speedup claim needs
evidence on representative data, not assertion.
Exit Criteria
Source: athola/claude-night-market → plugins/leyline/skills/loop-optimization/SKILL.md
1---2name: loop-optimization3description: Decides hand-vs-compiler for loop transforms (unrolling, SIMD, fusion, hoisting). Use when reviewing/authoring a hot loop or tempted to hand-optimize one.4---5
6
7# Loop Optimization: Hand vs Compiler
8
9A decision rule for the five common loop transformations. Its value
10is knowing when manual application is redundant (the compiler already
11does it) or harmful (it defeats the vectorizer or fools your
12benchmark).
13
14## When To Use
15
16- Reviewing a hot loop and a hand-rolled transform appears (unrolled
17 body, shift-instead-of-multiply, bespoke SIMD).
18- Authoring a loop that profiling proved hot, deciding whether to
19 optimize it by hand.
20- Pushing back on a "this is faster" claim about a loop micro-opt.
21
22## When NOT To Use
23
24- The loop is not proven hot by a profiler. Optimize nothing first.
25- Architecture-level performance (caching layers, sharding): use
26 `Skill(pensive:architecture-review)`.
27- Detecting complexity hotspots (O(n^2) shapes):
28 `Skill(pensive:performance-review)`.
29
30## The decision rule
31
321. Profile first. No loop transform without a hot loop proven by a
33 profiler.
342. In compiled languages (C, C++, Rust), trust the compiler for
35 loop-invariant code motion and strength reduction: both run
36 automatically at `-O2`/`-O3`, so the manual form is redundant. Leave
37 unrolling to the compiler as well. Unlike the other two it is not on
38 by default (GCC needs `-funroll-loops`), but the compiler owns the
39 profitability decision and manual unrolling routinely defeats the
40 auto-vectorizer.
413. If a loop will not vectorize, fix aliasing (`restrict` /
42 `__restrict__`) and loop shape first. Confirm with an optimization
43 report (`-fopt-info-vec-missed`, `-Rpass-missed=loop-vectorize`).
44 Reach for intrinsics last and accept the portability cost.
454. The manual transforms that still pay: explicit SIMD on loops the
46 compiler misses, loop fusion (guard against register and cache
47 pressure), and multi-accumulator unrolling to break a floating-point
48 reduction chain the compiler legally will not reorder.
495. In Python, the levers are: hoist invariants out of the loop,
50 vectorize via NumPy, fuse passes via numexpr/Numba. Do not hand-unroll
51 or hand-strength-reduce: the cost is bytecode dispatch, not loop
52 control.
536. Validate every claimed speedup on production-distribution data.
54
55## Per-technique reality
56
57| Technique | Helps where | When NOT to apply by hand |
58|-----------|-------------|---------------------------|
59| Unrolling | C/C++/Rust FP reduction chains (multi-accumulator) | Auto-vectorizable loops (defeats vectorizer); OOO CPUs; icache pressure; Python |
60| SIMD / vectorization | C/C++/Rust loops the compiler misses; Python via NumPy | Before fixing aliasing/loop shape; short trip counts; unverified that emitted SIMD runs |
61| Loop fusion | Bandwidth-bound array loops; Python via numexpr/Numba | When it spills registers or mixes strided access; compute-bound bodies; blocks vectorization |
62| Hoisting (LICM) | Python (no compiler does it); C/C++/Rust only when aliasing blocks the proof | `-O2`+ compiled code: redundant and can lengthen live ranges |
63| Strength reduction | Compilers do it; near-useless by hand | `-O2`+ compiled code: blocks the compiler's IV analysis and vectorization |
64
65## Two traps that invalidate "it is faster"
66
671. Synthetic-benchmark trap. A loop micro-opt validated on reused, small,
68 or synthetic input can invert to slower on production data, because
69 synthetic input hides effects such as branch misprediction on real
70 value distributions. Benchmark on production-distribution data with
71 optimizer barriers, or do not claim the win.
722. Emitted is not executed. Auto-vectorization fails silently. "The
73 compiler emitted SIMD" does not mean "SIMD ran." Confirm with codegen
74 or optimization reports, not source inspection.
75
76Both traps tie into `Skill(imbue:proof-of-work)`: a speedup claim needs
77evidence on representative data, not assertion.
78
79## Exit Criteria
80
81- [ ] The loop in question was profiled and is genuinely hot, or the
82 recommendation is "do not optimize."
83- [ ] For compiled languages, unrolling/LICM/strength-reduction were
84 left to the compiler unless an optimization report shows the
85 compiler failed (aliasing) and the manual form was verified faster.
86- [ ] Any manual SIMD was preceded by an aliasing/loop-shape fix and a
87 check that the vectorized path actually executes.
88- [ ] Every speedup claim cites a benchmark on production-distribution
89 data, not synthetic or reused input.
90
91---
92
93**Source:** [`athola/claude-night-market`](https://github.com/athola/claude-night-market) → `plugins/leyline/skills/loop-optimization/SKILL.md`
thedixitjain