Go Perf False Sharing

Go Performance: False Sharing

"Prevents false sharing on widespread platforms with 128 mod (cache line size) = 0." — comment on poolLocal.pad, src/sync/pool.go

This skill is a leaf of go-perf-methodology (measure first) and the mirror image of go-perf-struct-layout. That skill packs — reorders fields largest-to-smallest to shrink a struct and cut padding. This skill pads — deliberately adds padding to separate two hot fields onto different cache lines. Opposite mechanics, both correct: pack cold/shared data to save memory and GC scan work; pad hot per-core data to save cache-coherence traffic. Reach for padding only after a profile proves multi-core scaling is the problem (§5).

All Go below builds clean under gofmt/go vet on Go 1.26.

1. What False Sharing Is

A CPU does not move bytes between cores; it moves whole cache lines — 64 bytes on amd64 and most arm64. When two goroutines run on two cores and write to different variables that happen to land on the same line, the cache-coherence protocol (MESI) must give exactly one core exclusive ownership of that line per write. Each write on core A invalidates the line in core B's cache; B's next write pulls it back and invalidates A. The line ping-pongs across the interconnect, and the two writes serialize — even though the program has no shared variable and no lock. The contention is an accident of memory layout, not of logic; that is why it is "false."

Concretely, in MESI terms: a line a core may write must be in the Modified or Exclusive state, which is exclusive to one core. When core B writes its variable, it sends a Read-For-Ownership that transitions A's copy to Invalid; A's next write re-acquires ownership and invalidates B. Reads alone are fine — a line every core only reads stays Shared in all caches forever. False sharing is therefore a write/write (or write/read) phenomenon on one line, and its cost is an off-CPU coherence stall, not extra instructions — which is why a CPU profile barely shows it.

The runtime documents exactly this hazard wherever it pads: _ cpu.CacheLinePad // prevents false-sharing between full and empty (src/runtime/mgc.go, the GC's global workType). The standard library's own definition: "CacheLinePad is used to pad structs to avoid false sharing." (src/internal/cpu/cpu.go).

Where it shows up in real Go code: sharded counters/metrics keyed by hash or P index; per-P or per-worker scratch state; the head and tail indices of a single-producer/single-consumer ring buffer (producer writes head, consumer writes tail — pack them adjacently and the two threads fight one line every operation); and any [N]T array of small, independently-mutated structs.

2. The Symptom — Scaling That Goes Backward

False sharing is invisible at GOMAXPROCS=1 and gets worse as you add cores. The classic tell is a sharded counter or per-P array built specifically to avoid contention that nonetheless refuses to scale:

// WRONG: 256 counters packed adjacently — 8 bytes each, so 8 share every
// 64-byte line. Two goroutines on different shards still fight over a line.
type Counter struct {
	shards [256]atomic.Uint64
}

Eight atomic.Uint64 shards fit in one 64-byte line, so goroutines hashing to shards 0..7 contend on a single line despite touching "independent" counters. The byte layout makes it plain — one cache line, eight "independent" counters, one owner at a time:

cache line 0 (bytes 0..63)                          | cache line 1 (64..127)
[ s0 ][ s1 ][ s2 ][ s3 ][ s4 ][ s5 ][ s6 ][ s7 ]    | [ s8 ][ s9 ]...
  ^core A writes s2        ^core B writes s5
  -> both writes serialize on line 0's ownership

The benchmark signature: throughput is flat or negative from 1→N cores, and a CPU profile looks innocent because the cost is off-CPU coherence stalls, not instructions — perf c2c or cache-miss counters are what reveal it (→ go-perf-os-tooling). Blaming the hash, the algorithm, or the atomic is the usual wrong turn (→ common-mistakes).

The shape to watch for — more cores making each op slower, the opposite of what sharding promised:

# packed [256]atomic.Uint64 (false-shared)   # padded [256]shard (fixed)
Counter-1    12.0ns ± 1%                      Counter-1    12.1ns ± 1%
Counter-2    23.4ns ± 3%   ← slower!          Counter-2     6.2ns ± 2%
Counter-4    41.8ns ± 5%                       Counter-4     3.2ns ± 2%
Counter-8    78.0ns ± 6%                       Counter-8     1.7ns ± 3%

The packed column degrades because every increment must steal the line back from whichever core touched it last; the padded column halves with each core doubling, as an uncontended sharded counter should. (Numbers illustrative — your hardware differs; the trend is the diagnostic, not the absolute figures.)

3. The Fix — Pad Each Hot Field to a Full Line

Make every independently-written field occupy its own cache line. Use cpu.CacheLinePad from golang.org/x/sys/cpu (the public mirror of the runtime's internal/cpu):

import (
	"sync/atomic"

	"golang.org/x/sys/cpu"
)

type shard struct {
	n atomic.Uint64
	_ cpu.CacheLinePad // fills out the rest of the line; next shard starts fresh
}

type Counter struct {
	shards [256]shard // each shard now owns ≥1 cache line; no cross-shard bouncing
}

func (c *Counter) Inc(key uint64) { c.shards[key%256].n.Add(1) }

cpu.CacheLinePad is struct{ _ [cacheLineSize]byte } — a whole line of padding (golang/x/sys cpu/cpu.go). Placed after an 8-byte counter it makes the per-shard stride exceed one line, so no two shards ever share one. Verify the stride rather than assuming it: unsafe.Sizeof(shard{}) should be ≥ the line size and the array stride should put each element on its own line (→ go-perf-struct-layout for the Sizeof/Offsetof mechanics).

(An array of pointers to separately heap-allocated, line-sized shards also avoids the sharing, since the allocator spaces large objects out — but it adds an indirection and pointer chasing the prefetcher dislikes. Prefer the inline-padded value array unless you need the shards to outlive the array.)

For the ring-buffer case, separate the two indices the same way so producer and consumer never share a line:

type ring struct {
	head atomic.Uint64 // written only by the producer
	_    cpu.CacheLinePad
	tail atomic.Uint64 // written only by the consumer
	_    cpu.CacheLinePad
}

The runtime's precise pattern fills exactly the remainder instead of a whole extra line. sync.Pool pads its per-P poolLocal so each lands on its own line (src/sync/pool.go):

type poolLocal struct {
	poolLocalInternal
	// Prevents false sharing on widespread platforms with
	// 128 mod (cache line size) = 0 .
	pad [128 - unsafe.Sizeof(poolLocalInternal{})%128]byte
}

The GC's heap uses the same idea to give each mcentral.lock its own line: "the padding makes sure that the mcentrals are spaced CacheLinePadSize bytes apart, so that each mcentral.lock gets its own cache line." (src/runtime/mheap.go).

The two forms differ in the resulting stride. With a whole cpu.CacheLinePad field, unsafe.Sizeof(shard{}) on amd64 is 72 (8-byte counter + 64-byte pad); since 72 > 64, consecutive shards[i] and shards[i+1] never land on one line — correct, with 8 bytes slack per element. The modulo-fill form (pad [cpu.CacheLinePadSize - unsafe.Sizeof(inner{})%cpu.CacheLinePadSize]byte) instead rounds the struct up to exactly one line (64), the memory-tight choice for large arrays. Pick the readable separator unless the slack matters at scale; either way, confirm the stride with unsafe.Sizeof before trusting the fix.

4. Cache Line Size Is Per-Architecture — Don't Hardcode 64

There is no runtime detection; Go assumes a constant per GOARCH: "There is currently no runtime detection of the real cache line size so we use the constant per GOARCH CacheLinePadSize as an approximation." (src/internal/cpu/cpu.go). The values (from src/internal/cpu/cpu_*.go):

This is why sync.Pool pads to 128, not 64: a single constant that is a multiple of every platform's real line size is safe everywhere (128 mod (cache line size) = 0). Use cpu.CacheLinePad rather than a literal [64]byte so the pad tracks the target arch — a [64]byte pad under-pads arm64 (128-byte lines) and the false sharing survives.

5. Measure It — Across GOMAXPROCS, Not One Core

False sharing only appears under real parallelism, so a single-goroutine benchmark is blind to it (→ go-perf-benchmarking-statistics for the statistics). Benchmark the contended path with b.RunParallel so the runtime spreads goroutines across all P's, and sweep core counts:

func BenchmarkCounter(b *testing.B) {
	var c Counter
	var key atomic.Uint64
	b.RunParallel(func(pb *testing.PB) {
		k := key.Add(1) // distinct shard per goroutine
		for pb.Next() {
			c.Inc(k)
		}
	})
}

go test -bench=Counter -cpu=1,2,4,8 -count=10 >runs.txt && benchstat runs.txt

A healthy sharded structure scales roughly linearly with -cpu; one with false sharing flattens or regresses as cores climb. -cpu sets GOMAXPROCS for the run, so the sweep directly answers "does adding cores help?" — the only question false sharing answers wrongly. Note that b.RunParallel already fans the body across GOMAXPROCS goroutines, so the bug is latent even at -cpu=1 (one P, no cross-core writes) and only emerges as you raise the count — which is exactly why a fixed single-core benchmark misses it.

Confirm the mechanism with OS tooling — perf c2c ("cache-to-cache") attributes HITM (hit-modified) coherence events to the exact line and offset, and cache-miss/LLC-load-misses counters spike on the false-shared line (→ go-perf-os-tooling). A Go-level corroboration: the time is invisible in a CPU profile (off-CPU stall) yet wall-clock under load is high — that gap between "profile looks cheap" and "it's slow" is itself a clue. Pad, then re-run the same sweep: the fix is real only if the scaling curve straightens and benchstat shows a significant win at high -cpu (→ go-perf-benchmarking-statistics).

6. Atomics Don't Make It Disappear

A common reflex is "it's already an atomic.Uint64, so concurrency is handled." Atomics give you correctness, not layout. A contended atomic write still acquires the line exclusively — so two atomics on the same line bounce exactly as two plain writes would, and the lock-free counter you sharded to avoid a mutex can be slower than the mutex was. Padding is orthogonal to the atomic/lock choice (→ go-perf-atomics-vs-locks for that cost model); you generally need both: an atomic (or per-shard mutex) for correctness and cache-line separation for scaling.

7. Quick Diagnosis

1. Does it scale backward? Benchmark -cpu=1,2,4,8; flat or negative throughput as cores rise is the signature (§5).
2. Is the CPU profile innocent? If pprof shows no obvious hot instruction yet the wall-clock is high under load, suspect off-CPU coherence stalls.
3. Confirm the line. perf c2c reports HITM (hit-modified) events and the exact line/offset two cores fight over (→ go-perf-os-tooling).
4. Check the layout. Are independently-written, per-core fields within one CacheLinePadSize of each other (unsafe.Sizeof/Offsetof)?
5. Pad and re-sweep. Add cpu.CacheLinePad; the fix is real only if the scaling curve straightens and benchstat shows a win at high -cpu.
6. Watch for over-correction. If padding helps nothing, the bottleneck was elsewhere (a genuinely shared variable, a real lock, GC) — revert the bytes and look again (§8).

8. When NOT to Pad

Padding is not free — it bloats every instance and pushes data apart, hurting the cache locality that go-perf-struct-layout works to gain. Pad only the genuinely hot, independently-written, per-core fields, and only after measurement:

• Read-mostly or read-only data never false-shares. Coherence only ping-pongs on writes; a line that all cores only read stays Shared in every cache. Don't pad immutable config.
• The allocator may already separate you. Go's per-P p struct dropped its padding: "Padding is no longer needed. False sharing is now not a worry because p is large enough that its size class is an integer multiple of the cache line size." (src/runtime/runtime2.go). A large heap object whose size rounds to a multiple of the line already starts each instance on a line boundary — check before adding bytes.
• One cold field next to a hot one is cheaper to move (reorder it out of the hot line, a go-perf-struct-layout job) than to pad around.
• Low concurrency or low write rate rarely justifies the bytes. A counter touched by two goroutines a few times a second will never generate enough coherence traffic to matter; padding it is cargo-culting.

Over-padding every struct is the opposite mistake: it wastes memory and cache footprint for contention that a profile would show never existed. The discipline is the same as everywhere in this plugin — pad in response to a measured non-scaling curve, never on the suspicion that a field "might" be hot (→ go-perf-methodology).

9. Routing to Related Skills

This plugin (go-perf-*):

• go-perf-struct-layout — field ordering, alignment, fieldalignment, unsafe.Sizeof; packing to shrink (the inverse of this skill). Padding decisions that aren't about cross-core contention live there.
• go-perf-contention-and-sharding — how to shard state into N buckets; this skill only fixes the layout of the shards once you have them.
• go-perf-atomics-vs-locks — cost of the atomic/mutex inside each shard; contended atomics also bounce lines.
• go-perf-block-mutex-profiles — diagnosing logical lock/channel contention (distinct from false sharing's accidental, lock-free contention).
• go-perf-os-tooling — perf c2c, cache-miss counters, flame graphs for confirming the coherence stalls.
• go-perf-benchmarking-statistics — b.RunParallel, -cpu sweeps, benchstat across core counts.
• go-perf-allocator-internals — size classes and span layout; the runtime's "size class is a multiple of the line" reasoning (§8) for when allocation already separates instances.
• go-perf-methodology — the measure-first gate; never pad on a hunch.

Sibling marketplace (go-* correctness):

• go-sync-primitives — atomic/Mutex correctness and copylock; go-race-and-memory-model — happens-before of the atomics you are padding.
• go-performance — the measure-first overview this plugin deepens; start there for the loop before chasing a cache line.

10. Don't

• Don't pack per-core counters adjacently. Eight atomic.Uint64 per 64-byte line means eight "independent" shards fight one line (src/runtime/mgc.go).
• Don't blame the algorithm when a parallel structure won't scale. Flat/negative scaling with an innocent CPU profile is the false-sharing signature — check the layout and perf c2c first (§5).
• Don't hardcode [64]byte. arm64 lines are 128 bytes; use cpu.CacheLinePad so the pad tracks GOARCH (src/internal/cpu/cpu.go).
• Don't pad read-only or read-mostly data. Coherence only ping-pongs on writes; padding immutable data is pure waste (§6).
• Don't pad everything by reflex. Over-padding bloats memory and cache footprint; pad only profile-proven hot per-core fields (§6).
• Don't claim a false-sharing fix from a single-goroutine benchmark. It cannot see the problem; sweep -cpu and confirm the scaling curve straightens (§5).
• Don't reach for internal/cpu. It is not importable; the public package is golang.org/x/sys/cpu.
• Don't assume an atomic or a per-shard mutex removes the need to pad. Correctness is not layout; two contended atomics on one line bounce exactly like two plain writes (§6).
• Don't pad a low-traffic field. Coherence cost scales with write frequency; a rarely-written counter never generates enough traffic to repay the bytes (§8).

11. Reference Files

False-sharing anti-patterns in LLM-generated Go, each wrong/right with citations:

references/common-mistakes.md

Source provenance for every claim in this skill:

references/sources.yaml