Go Perf Budgets And Lifecycle

Go Performance Budgets & Lifecycle

"Set performance goals; confirm you're not meeting them." — step one of the workflow in go-perfbook

go-perf-methodology owns the measure-first loop for a single optimization — profile, change one thing, measure again. This skill is the layer around it: when in the development cycle performance work belongs, how to turn "fast enough?" into an executable budget, and how a team keeps a service fast over months, not for one heroic afternoon. The statistics and commands that enforce a budget (benchstat, the CI A/B) live in go-perf-benchmarking-statistics; this skill owns the process those commands serve.

All Go below builds clean under gofmt, go vet, and go test on Go 1.26.

1. Where Performance Work Belongs in the Cycle

Performance is not a phase you bolt on at the end, nor work you sprinkle through every feature. It attaches to specific points in the lifecycle, and doing it at the wrong point is itself the mistake:

The two load-bearing judgments: optimize late (after measuring, on a real workload), but budget early (at design, before the code exists). Premature optimization and a missing budget are the same disease — both substitute opinion for a number — they just fail at opposite ends of the cycle. Everything below is one or the other half of that sentence: §2–§3 set the budget early; §4–§6 are how it is enforced and kept honest late, automatically, so the team is not relying on one engineer remembering to check.

2. Don't Optimize During Feature Development — Budget Instead

While a feature is being built, the value is correctness and clarity; an unmeasured speed hack here is premature optimization — "you can't tell where a program is going to spend its time … don't put in a speed hack until you've proven that's where the bottleneck is" (Pike). Optimization is "refactoring … [that] improves performance … generally … at the cost of readability" (go-perfbook) — a cost you pay after a profile proves the path is hot, not during the first draft.

But "don't optimize now" is not "ignore performance now." The counterweight is premature pessimization — don't "write code that is slower than it needs to be … when equivalently complex code would be faster" (go-perfbook). During build:

• Do pick the right data structure and algorithm — "data dominates … the algorithms will almost always be self-evident" (Pike); these are the wins that dwarf any micro-tune, and they are nearly free at design time and expensive to retrofit.
• Do avoid the obvious waste (an O(n²) in a loop, an allocation that copies a slice every iteration) — that is hygiene, not optimization.
• Don't hand-roll the encoder, reach for unsafe, or shave allocations on a path nothing has profiled. Defer it. Write it down as a budget (§3) and let CI or staging tell you whether the path is actually hot.

The discipline that decides "is this worth optimizing now?" is go-perf-methodology (Amdahl, the order of operations). This skill decides when in the cycle the question even gets asked: at design you set the budget, at test/CI/staging you discover whether you're missing it.

3. Performance Budgets — Turning "Fast Enough?" Into an Assertion

A performance budget is go-perfbook's step one made concrete: "set performance goals; confirm you're not meeting them" (go-perfbook). It converts a subjective "is this fast enough?" into an objective assertion something can check:

• p99 < 50 ms for the request handler (an SLO — depth in go-perf-tail-latency);
• 0 allocs/op on the hot encode path;
• decode ≥ 500 MB/s, or throughput ≥ 20k req/s at the target concurrency.

Budget the cheap, deterministic thing first. allocs/op is deterministic per code path — it does not vary with CPU frequency, thermals, or a noisy neighbor — so it is the cheapest, most reliable budget you can write and the first to enforce. A jump from 1 → 3 allocs/op is a real regression even when ns/op is buried in noise. Prefer an allocs/op budget over a ns/op budget wherever the path's cost is allocation-driven; reserve absolute time budgets for the SLO boundary (where users actually feel latency), and measure those relatively against a baseline, never as an absolute floor on a shared runner (§4). The benchstat mechanics that read these numbers live in go-perf-benchmarking-statistics.

Because allocation count is deterministic, a zero-alloc budget can be a plain unit test — testing.AllocsPerRun makes the budget fail the build without benchstat or a noisy timing threshold:

// budget_test.go — the encode-path budget, executable and deterministic.
func BenchmarkEncode(b *testing.B) {
	payload := representativeInput(b)
	b.ReportAllocs()
	for b.Loop() {
		_, _ = Encode(payload)
	}
}

// A guard test turns "0 allocs/op on the hot encode path" into a hard gate:
// AllocsPerRun is deterministic, so this fails locally and in CI with no
// benchstat and no flaky ns/op threshold.
func TestEncodeAllocBudget(t *testing.T) {
	payload := representativeInput(t)
	got := testing.AllocsPerRun(1000, func() { _, _ = Encode(payload) })
	if got != 0 {
		t.Fatalf("encode alloc budget exceeded: got %.0f allocs/op, want 0", got)
	}
}

The timing side of a budget (p99, MB/s) is not deterministic, so it cannot be a bare equality assertion — it goes through the relative benchstat A/B of §4 instead. A budget you write down but never check is a comment, not a budget. The next two sections are how a budget becomes load-bearing: CI enforces it on every PR (§4), and production monitoring catches the part CI can't see (§5).

4. The CI Regression Gate — Keeping Perf From Rotting

Without a gate, performance rots by a thousand cuts: each PR adds an alloc or a copy, none big enough to notice, until the hot path is 3× slower and no single commit is to blame. A regression that isn't gated lands silently. The gate is a CI job that runs the tracked benchmarks on the PR and its base on the same runner, diffs them with benchstat, and fails the build on a significant regression. The exact commands (-count≥10, -benchmem, -run='^$', the benchstat invocation and thresholds) belong to go-perf-benchmarking-statistics; the process rules that keep the gate trustworthy:

• Compare PR vs base on the same runner — never an absolute ns/op floor. A shared CI runner's clock varies run-to-run; an absolute threshold flakes red on a slow runner and green on a fast one. Relative A/B in one job cancels the runner out. Treat benchstat's ~ ("no statistically significant difference") as "no regression."
• Gate allocs/op before ns/op. Deterministic per path, it is the gate that doesn't flake; make it the hard failure and treat time as advisory.
• Track the key benchmarks over time, not just per-PR. Store the geomean of the budgeted suite so slow drift across many "no-regression" PRs is still visible on a chart, not only the single-PR delta.
• Keep the gate visible. A red check on the PR with the benchstat table inline is how the whole team learns the budget exists (§7) — a gate nobody sees teaches nobody.

5. Production & the Error-Budget Analogy

CI gates a benchmark; production is where the SLO is actually spent, under load and input distributions no bench reproduces — "some issues not apparent on your laptop might be visible once deployed" at high request rates (go-perfbook). So the lifecycle does not end at merge: monitor the SLO in prod, capture regressions and incidents (depth in go-perf-production-incidents), and feed what you learn back into the budgets and benchmarks.

The SRE error-budget practice is the disciplined analogy for a latency budget (apply it carefully — it was written for reliability, not CPU time). Google's SRE book frames an objective: "the error budget provides a clear, objective metric that determines how unreliable the service is allowed to be," and crucially "100% is probably never the right reliability target: not only is it impossible to achieve, it's typically more reliability than a service's users want or notice" (Google SRE Book, Embracing Risk). Two ideas transfer:

• A budget is a number you are allowed to spend, not a wall. "When the budget is large, the product developers can take more risks. When the budget is nearly drained, the product developers … push for more testing or slower push velocity" (SRE Book). A p99 budget works the same way: headroom under the SLO is room to ship features; near the limit, optimization moves up the priority list.
• The budget aligns the team. It "aligns incentives and emphasizes joint ownership" (SRE Book) — the same reason a visible perf budget stops the argument about whether a PR is "fast enough."

The lever for a tail-latency budget is usually lowering the allocation rate (fewer allocations → less GC → smaller pauses), ahead of any GC knob; SLO/percentile depth and coordinated-omission traps live in go-perf-tail-latency.

The loop closes when production feeds back: every regression caught in prod that CI missed is a gap in the bench suite. Add the workload that exposed it as a tracked benchmark so the gate catches the next one before merge — the bench suite grows toward the production input distribution over time, and the incident becomes a permanent regression test rather than a one-off firefight (go-perf-production-incidents).

6. Re-baseline After Go Upgrades

A baseline is tied to a runtime. A Go version bump can shift it out from under you, so a green budget on the old toolchain is not evidence on the new one — re-record the baseline after every upgrade before trusting the gate. Recent runtime shifts that move the floor:

• Green Tea GC default in Go 1.26 (an experiment in 1.25) changes GC behavior and the allocation/pause baseline (Go 1.26).
• Container-aware GOMAXPROCS in Go 1.25 — under a cgroup CPU limit the default drops to that limit, changing parallelism (and thus throughput and contention numbers) in containers (Go 1.25).
• Swiss Tables maps in Go 1.24 changed map performance characteristics, so map-heavy benchmarks may move on upgrade (Go 1.24).

After bumping the toolchain: re-run the budgeted suite, record the new old.txt, and reset the CI comparison base to it. Version floors for these features are catalogued in go-version-feature-map.

PGO profiles drift too. A profile-guided-optimization build is optimized for the code that produced the profile; as the source moves, the profile goes stale — "degradation will slowly accumulate over time since code is rarely refactored back to its old form, so it is important to collect new profiles regularly to limit source skew from production" (PGO). The intended lifecycle is a loop — release, collect a production profile, rebuild from latest source with it, repeat (PGO) — so a PGO refresh is a recurring lifecycle task, not a one-time setup. Mechanics in go-perf-pgo.

7. Team Knowledge Transfer — Make "How We Do Perf Here" Legible

Performance survives staff turnover only if the practice is visible, not folklore in one engineer's head. A newcomer should be able to answer three questions from the repo alone:

• What are the budgets? The tracked benchmarks and their thresholds live in the repo (a bench_test.go plus a documented budget), not in tribal memory.
• Where is the gate? The CI perf job is a required check; its benchstat output is on the PR, so "you regressed the encode path" is a visible red check, not a code-review opinion (§4).
• What is the norm? Measure first. The team default — no optimization PR without a benchstat A/B, no budget claim without a number — is the go-perf-methodology discipline made into a team habit, documented where contributors will read it.

This is the difference between a service that is fast today (one person tuned it) and one that stays fast (the team has a reproducible practice). The budgets, the gate, and the measure-first norm are the three artifacts that make perf a property of the team, not of an individual.

8. Don't

• Don't optimize during feature development before any profile exists — that's premature optimization; budget the path at design and defer the tuning until CI or staging proves it's hot (Pike).
• Don't read "defer optimization" as "pessimize freely" — pick the right data structure and avoid obvious waste while building; that's hygiene, not premature optimization (go-perfbook).
• Don't leave "fast enough?" as an opinion — write it as an executable budget (0 allocs/op, p99 < 50 ms) (go-perfbook).
• Don't budget ns/op when the cost is allocation — gate the deterministic allocs/op first; time on a shared runner is noisy.
• Don't ship without a CI perf gate — an ungated regression lands silently and rots the hot path over many small PRs (commands → go-perf-benchmarking-statistics).
• Don't gate on an absolute ns/op floor on a shared runner — compare PR vs base in the same job; treat ~ as "no regression."
• Don't trust an old baseline after a Go upgrade — re-baseline; Green Tea GC (1.26), container-aware GOMAXPROCS (1.25), and Swiss Tables maps (1.24) move the floor (Go 1.26).
• Don't set a PGO profile once and forget it — refresh it regularly as the source drifts (PGO).
• Don't keep the budgets in one person's head — the budgets, the gate, and the measure-first norm must be visible in the repo (§7).

9. Routing to Related Skills

This plugin (go-perf-*):

• go-perf-methodology — the measure-first loop, USE method, Amdahl, order of operations: is this worth optimizing, and how do I know it worked? This skill decides when in the cycle to ask.
• go-perf-benchmarking-statistics — the commands a budget/gate runs on: benchstat, -count, p-value/~/geomean, the CI A/B invocation, perflock.
• go-perf-tail-latency — p99/p999 SLO budgets, coordinated omission, allocation rate as the tail lever.
• go-perf-code-review — review-time perf flags: the free wins to catch and the unmeasured micro-opts to wave through.
• go-perf-production-incidents — capturing and triaging a production regression, feeding it back into budgets and benchmarks.
• go-perf-pgo — PGO build mechanics and the production-profile collection this skill schedules as a recurring refresh.

Parent / sibling marketplace (go-*):

• go-performance — the altitude measure-first overview these go-perf-* skills deepen.
• go-version-feature-map — version floors for the re-baseline facts: Swiss Tables maps 1.24, container-aware GOMAXPROCS 1.25, Green Tea GC default 1.26.
• go-idiomatic-discipline — the clarity-vs-unmeasured-micro-opt axis that "don't pessimize, defer micro-opts" sits on.

10. Reference Files

Process anti-patterns in how teams place and sustain performance work — each a wrong/right contrast with a citation:

references/common-mistakes.md

Source provenance for every claim in this skill:

references/sources.yaml