# Go Perf Contention And Sharding

> Reduces Go lock contention by sharding state into N lock-striped shards keyed by hash; shard-count and padding guidance; sync.Map's two optimized use cases vs a sharded map[K]V+Mutex for the general case; per-P counters summed on read; batching to amortize a lock; singleflight to collapse duplicate work. Fires on "reduce lock contention", "shard this map", "sync.Map vs mutex map", "hot mutex", "lock striping". Routes sync.Map API to go-sync-primitives, padding to go-perf-false-sharing, measuring to go-perf-block-mutex-profiles.

- Skill: `ctoth/go-perf-contention-and-sharding` (Agent Skill, multi-file: 3 files)
- Install (CLI): `npx skillmds@latest add ctoth/go-perf-contention-and-sharding`
- Raw SKILL.md: https://api.skillmd.com/api/skills/ctoth/go-perf-contention-and-sharding/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Integrations & APIs
- Author: ctoth (https://skillmd.com/u/ctoth)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/ctoth/go-perf-contention-and-sharding

---


# Go Contention & Sharding

> "The Map type is optimized for two common use cases ... In these two cases, use of a Map may significantly reduce lock contention compared to a Go map paired with a separate Mutex or RWMutex."
> — [pkg.go.dev/sync](https://pkg.go.dev/sync#Map)

A single mutex guarding shared state is a **serialization point**: while one goroutine holds it, every other core that wants that state stalls, so throughput stops scaling and can *fall* as cores are added. This skill owns the **performance** fix — splitting one lock into many so independent operations rarely collide. The *correctness* rules of the primitives — `sync.Map`'s two cases, the no-copy Mutex rule, typed atomics — belong to **`go-sync-primitives`**; read it first. Always **prove the contention exists** before sharding: a block/mutex profile (**`go-perf-block-mutex-profiles`**) tells you which lock is hot, and benchstat (**`go-perf-benchmarking-statistics`**) tells you the shard actually helped.

All Go below builds clean under `gofmt`, `go vet`, and `go test` on Go 1.26.

---

## 1. The Core Idea — One Hot Lock Serializes Every Core

When a profile shows goroutines spending wall-clock time *waiting* on one `Mutex`, the lock — not the work it guards — is the bottleneck. The CPU profile is blind to this (it only sees on-CPU time); the **mutex profile** records it, but is **off by default** and records nothing until you call `runtime.SetMutexProfileFraction` — "controls the fraction of mutex contention events that are reported in the mutex profile ... To turn off profiling entirely, pass rate 0" ([runtime](https://pkg.go.dev/runtime#SetMutexProfileFraction)) — and `runtime.SetBlockProfileRate` for the block profile ([runtime](https://pkg.go.dev/runtime#SetBlockProfileRate)). Enablement and reading these are owned by **`go-perf-block-mutex-profiles`**.

The fix is **lock striping**: replace the one lock + one map with *N* shards, each its own lock guarding its own slice of the keyspace. Hash the key to a shard; two goroutines touching keys in different shards never contend. Contention drops roughly *N×* when keys spread evenly, because at most ~1/N of the traffic lands on any one lock.

---

## 2. A Sharded Map — Lock Striping by Key Hash

The pattern: a fixed array of shards, each a `Mutex` (or `RWMutex` when reads dominate — see `go-perf-atomics-vs-locks`) over its own `map`, selected by hashing the key. This is exactly how `orcaman/concurrent-map` works — it shards "the map with minimal time spent waiting for locks," and is pitched for "something more like in-memory db" with frequent writes, *unlike* `sync.Map` which it notes "is designed for append-only scenarios" ([orcaman/concurrent-map](https://github.com/orcaman/concurrent-map)).

```go
// shardCount is fixed and a power of two so key%shardCount is a mask, not a divide.
// orcaman/concurrent-map ships SHARD_COUNT = 32 as its default ([orcaman/concurrent-map]).
const shardCount = 32

type shard struct {
	mu sync.RWMutex
	m  map[string]int
	// Pad so two shards never share a 64-byte cache line; see go-perf-false-sharing.
	_ [64 - (unsafe.Sizeof(sync.RWMutex{})+unsafe.Sizeof(map[string]int(nil)))%64]byte
}

type ShardedMap struct{ shards [shardCount]shard }

func NewShardedMap() *ShardedMap {
	s := &ShardedMap{}
	for i := range s.shards {
		s.shards[i].m = make(map[string]int)
	}
	return s
}

// shard() spreads keys with a real hash — FNV-1a, as orcaman/concurrent-map does (full
// fnv32 in references/common-mistakes.md) — then masks to a shard since shardCount is a power of two.
func (s *ShardedMap) shard(key string) *shard { return &s.shards[fnv32(key)%shardCount] }

func (s *ShardedMap) Load(key string) (int, bool) {
	sh := s.shard(key)
	sh.mu.RLock()
	defer sh.mu.RUnlock()
	v, ok := sh.m[key]
	return v, ok
}

func (s *ShardedMap) Store(key string, v int) {
	sh := s.shard(key)
	sh.mu.Lock()
	sh.m[key] = v
	sh.mu.Unlock()
}
```

Two failure modes: a **bad hash** that clusters keys recreates a single hot shard (use a real hash, not `len(key)` or a sequential id's low bits); and **unpadded shards** sharing a cache line, so independent locks still bounce the line between cores — false sharing, owned by **`go-perf-false-sharing`**.

---

## 3. Shard Count — ≈ GOMAXPROCS or a Power of Two

The shard count sets the contention ceiling. Guidance:

- **Floor it at `GOMAXPROCS`.** With fewer shards than cores, cores still queue on shared locks. A small multiple of `GOMAXPROCS` (or a fixed power of two comfortably above it, like `orcaman`'s 32) leaves headroom as the schedulable parallelism grows. Note Go 1.25 makes `GOMAXPROCS` **container-aware** (it reads the cgroup CPU limit), so the effective core count under a CPU quota may be lower than the host's ([Go 1.25](https://go.dev/doc/go1.25)).
- **Prefer a power of two**, so `hash % shardCount` lowers to a bitmask instead of a division on the hot path.
- **Don't over-shard.** Each shard is a live map with its own bucket overhead; thousands of shards waste memory and cache for no contention win past the core count. And pad each shard to a cache line, or adjacent shard locks false-share — route to **`go-perf-false-sharing`**.

---

## 4. `sync.Map` vs a Sharded Map — Match the Access Pattern

`sync.Map` is **not** a general "concurrent map" — its own doc is explicit: "The Map type is specialized. Most code should use a plain Go map instead, with separate locking or coordination, for better type safety and to make it easier to maintain other invariants along with the map content" ([sync](https://pkg.go.dev/sync#Map)). It is "optimized for two common use cases: (1) when the entry for a given key is only ever written once but read many times, as in caches that only grow, or (2) when multiple goroutines read, write, and overwrite entries for disjoint sets of keys. In these two cases, use of a Map may significantly reduce lock contention" ([sync](https://pkg.go.dev/sync#Map)). The decision (its *API/correctness* depth lives in `go-sync-primitives`):

| Access pattern | Reach for |
|---|---|
| Write-once, read-many; cache that only grows (case 1) | `sync.Map` |
| Goroutines own **disjoint** key sets (case 2) | `sync.Map` |
| General read-**write** churn, overlapping keys, typed values, multi-field invariants | **sharded `map[K]V` + Mutex** (§2) |

For the general read-write case a sharded plain map is usually faster *and* type-safe (no `any` boxing, no per-read type assertion) — and it lets you hold an invariant across the map under one shard's lock, which `sync.Map` cannot. Reach for `sync.Map` only when the pattern is genuinely one of its two documented cases.

---

## 5. Counters — Stripe and Sum, Don't Hammer One Atomic

A single `atomic.Int64` incremented by every core is itself a contention point: each `Add` must win exclusive ownership of that one cache line, so the line ping-pongs between cores under load. The fix mirrors map sharding — **N per-core counter cells, each on its own cache line, summed on read**:

```go
type cell struct {
	n atomic.Int64
	_ [64 - 8]byte // one cell per cache line; padding owned by go-perf-false-sharing
}

type StripedCounter struct{ cells [shardCount]cell }

// id is any cheap per-goroutine spreader (a worker index, a hashed key).
func (c *StripedCounter) Add(id uint64, delta int64) { c.cells[id%shardCount].n.Add(delta) }

func (c *StripedCounter) Sum() int64 {
	var total int64
	for i := range c.cells {
		total += c.cells[i].n.Load()
	}
	return total
}
```

This trades an exact snapshot read (`Sum` is not instantaneously consistent) for write throughput that scales with cores — the right trade for high-frequency metrics. The atomic-vs-mutex *cost* model is **`go-perf-atomics-vs-locks`**; padding is **`go-perf-false-sharing`**.

---

## 6. Batch Under the Lock — Amortize the Acquisition

Lock acquisition has a fixed cost paid *per acquisition*. Acquiring once per item turns N items into N lock/unlock round-trips; acquiring once for the whole batch pays it once.

```go
// WRONG — one lock acquisition per item; the lock cost dominates the loop
for _, item := range items {
	mu.Lock()
	queue = append(queue, item)
	mu.Unlock()
}

// RIGHT — one acquisition amortized over the whole batch
mu.Lock()
queue = append(queue, items...)
mu.Unlock()
```

Hold the lock only for the shared mutation; do per-item *computation* before `Lock` so the critical section stays short (the small-critical-section rule is `go-sync-primitives`). Batching and lock striping compose: shard first, then batch within each shard's lock.

---

## 7. `singleflight` — Collapse Duplicate Concurrent Work

When many goroutines independently request the same expensive thing at once (a cache miss stampede, a thundering herd on one DB row), the contention is *duplicated work*, not a lock. `golang.org/x/sync/singleflight` provides "a duplicate function call suppression mechanism": `Group.Do` "executes and returns the results of the given function, making sure that only one execution is in-flight for a given key at a time. If a duplicate comes in, the duplicate caller waits for the original to complete and receives the same results" ([singleflight](https://pkg.go.dev/golang.org/x/sync/singleflight)).

```go
var g singleflight.Group

func (c *Cache) Fetch(ctx context.Context, key string) (*Row, error) {
	v, err, _ := g.Do(key, func() (any, error) {
		return loadFromDB(ctx, key) // exactly one DB load per key, however many callers race
	})
	if err != nil {
		return nil, err
	}
	return v.(*Row), nil
}
```

The third return, `shared`, "indicates whether v was given to multiple callers"; `DoChan` gives a `select`-able channel result, and `Forget` drops a key so the next call re-runs rather than reusing a stale in-flight result ([singleflight](https://pkg.go.dev/golang.org/x/sync/singleflight)). One caveat: `Do` couples callers — a slow or failed shared call delays everyone waiting on that key.

---

## 8. Don't

- **Don't shard before a profile proves the lock is hot.** Enable the mutex profile (`SetMutexProfileFraction`, off by default ([runtime](https://pkg.go.dev/runtime#SetMutexProfileFraction))) and confirm the wait — depth in `go-perf-block-mutex-profiles`.
- **Don't assume `sync.Map` is faster.** It is "specialized" and for the general read-write case a sharded `map`+`Mutex` usually wins ([sync](https://pkg.go.dev/sync#Map)).
- **Don't leave shards unpadded.** Adjacent shard locks share a cache line and false-share; pad to a cache line (`go-perf-false-sharing`).
- **Don't shard on a weak hash.** `len(key)` or sequential-id low bits cluster keys into one hot shard; use a real hash like FNV-1a ([orcaman/concurrent-map](https://github.com/orcaman/concurrent-map)).
- **Don't hammer one global atomic counter** from every core, or take the lock per item — stripe the counter (§5) and batch under one acquisition (§6).
- **Don't let a stampede duplicate work** that `singleflight` would collapse ([singleflight](https://pkg.go.dev/golang.org/x/sync/singleflight)).
- **Don't reach for a channel-as-mutex** to "fix" contention where sharding is the real lever — it imports a channel's full failure surface to do a lock's job (`go-sync-primitives`).

---

## 9. Routing to Related Skills

**This plugin (`go-perf-*` depth):**

- `go-perf-block-mutex-profiles` — **measure the contention first**: enabling and reading block/mutex/goroutine profiles.
- `go-perf-atomics-vs-locks` — the atomic-vs-Mutex / RWMutex cost model behind a shard's lock and a striped counter's cell.
- `go-perf-false-sharing` — padding shards and counter cells to a cache line (`cpu.CacheLinePad`); the non-scaling symptom.
- `go-perf-maps` — single-map performance (Swiss Tables, prealloc, key cost) that each shard inherits.
- `go-perf-benchmarking-statistics` — proving the sharded version is actually faster (n≥10, benchstat, `b.RunParallel`).

**Parent / sibling marketplace (`go-*` correctness):**

- `go-sync-primitives` — `sync.Map`'s two cases, the no-copy Mutex/atomic rules, small critical sections, `sync.Pool` — the **API and correctness** this skill's perf depth sits on.
- `go-race-and-memory-model` — why unsynchronized shard access is still a data race; happens-before.
- `go-version-feature-map` — version floors (container-aware `GOMAXPROCS` 1.25; typed atomics 1.19).

---

## 10. Reference Files

Contention/sharding anti-patterns in LLM-generated Go, each with wrong/right code and citations:

[references/common-mistakes.md](references/common-mistakes.md)

Source provenance for every claim in this skill:

[references/sources.yaml](references/sources.yaml)