Go Perf Worker Pools Throughput

Go Worker Pools & Throughput

"Goroutines are cheap, but not free." — dgryski/go-perfbook

Channel correctness — directionality, who closes, comma-ok, unbuffered-vs-buffered semantics — belongs to go-channels-select; goroutine lifetime and leaks (a stage that stops consuming blocks its senders forever, "a resource leak" — Go Pipelines) belong to go-concurrency-goroutines. Read those first for will it work. This skill owns only will it go fast: how many workers, how to bound them, and what the queue between them costs.

The one-line thesis: bound concurrency to the bottleneck resource, then batch to amortize the per-item synchronization. More workers past the saturated resource only add scheduling and contention overhead — whether a given count helps is a measurement question (route to go-perf-methodology).

All Go below builds clean under gofmt, go vet, and go test on Go 1.26.

1. The Core Anti-Pattern: One Unbounded Goroutine Per Item

The default LLM move — for _, item := range items { go work(item) } — has no ceiling. The Go blog names the cost directly: an implementation that "starts a new goroutine for each file … may allocate more memory than is available on the machine," and the fix is to "limit these allocations by bounding the number of files read in parallel … by creating a fixed number of goroutines" (Go Pipelines). Distributing work across "a fixed number of digester goroutines that receive file names from paths" is the worker pool — fan-out is "a way to distribute work amongst a group of workers to parallelize CPU use and I/O" (Go Pipelines).

Unbounded fan-out fails three ways at once: memory blows up (each goroutine's stack plus its captured data), the scheduler thrashes across thousands of runnable goroutines, and any downstream resource (DB connections, file descriptors, a rate-limited API) is hit by N simultaneous callers with no governor. Bounding fixes all three with one number.

2. Pool Sizing — CPU-Bound vs I/O-Bound

The size of the pool is a function of what the worker waits on.

• CPU-bound work (hashing, compression, parsing, pure compute): the parallelism ceiling is the number of CPUs that can run Go code simultaneously. GOMAXPROCS "sets the maximum number of CPUs that can be executing simultaneously," defaulting from "the number of logical CPUs on the machine, the process's CPU affinity mask, and, on Linux, the process's … cgroup CPU quota" (runtime.GOMAXPROCS). Spawning more than ~`GOMAXPROCSCPU-bound workers buys **no** extra parallelism — they queue for the same cores — and adds context-switch and cache cost. Size CPU-bound pools at or nearGOMAXPROCS`.
• I/O-bound work (network calls, disk, RPC, DB): each worker spends most of its time blocked, off-CPU, so a handful of CPUs can host hundreds of in-flight requests. The right count is higher than GOMAXPROCS and is set by the downstream limit (connection-pool size, API rate limit) and Little's Law, not by core count.

There is no universal constant: "I/O-bound = higher" is a starting hypothesis to tune by measuring throughput against worker count and watching where it plateaus (route the A/B to go-perf-methodology; the GMP scheduler and container-aware GOMAXPROCS to go-perf-goroutines-scheduler). The plateau is the bottleneck resource; workers past it are waste.

3. Bounding Primitive A — errgroup with SetLimit

golang.org/x/sync/errgroup is the idiomatic bounded group with error propagation. A zero Group "is valid, has no limit on the number of active goroutines, and does not cancel on error" — so an unconfigured group is still unbounded fan-out. SetLimit is what bounds it: "SetLimit limits the number of active goroutines in this group to at most n. A negative value indicates no limit. A limit of zero will prevent any new goroutines from being added" (errgroup).

The backpressure is built in: with a limit set, Go "blocks until the new goroutine can be added without the number of goroutines in the group exceeding the configured limit" (errgroup). The producer loop self-throttles — no manual semaphore needed.

func processAll(ctx context.Context, items []Item, limit int) error {
	g, ctx := errgroup.WithContext(ctx) // ctx cancels on first error
	g.SetLimit(limit)                   // at most `limit` workers live at once
	for _, it := range items {
		it := it
		g.Go(func() error { // blocks here once `limit` are active — backpressure
			return work(ctx, it)
		})
	}
	return g.Wait() // first non-nil error, after all return
}

Two rules from the docs: WithContext gives a context "canceled the first time a function passed to Go returns a non-nil error or the first time Wait returns" — so siblings stop on the first failure (errgroup). And "the limit must not be modified while any goroutines in the group are active" — call SetLimit once, up front (errgroup). When you'd rather drop work than block, TryGo "calls the given function in a new goroutine only if the number of active goroutines in the group is currently below the configured limit" and "reports whether the goroutine was started" (errgroup).

errgroup lifecycle/correctness (when to use it at all, the captured-loop-variable trap pre-1.22) → go-concurrency-goroutines.

4. Bounding Primitive B — semaphore.Weighted

When some tasks cost more of the resource than others, golang.org/x/sync/semaphore lets each acquisition carry a weight: Weighted "provides a way to bound concurrent access to a resource. The callers can request access with a given weight," and NewWeighted(n) "creates a new weighted semaphore with the given maximum combined weight for concurrent access" (semaphore). A uniform-cost pool is just weight 1 per task with n = pool size.

Acquire is the context-aware blocker: it "acquires the semaphore with a weight of n, blocking until resources are available or ctx is done. On success, returns nil. On failure, returns ctx.Err() and leaves the semaphore unchanged" (semaphore). Pair every Acquire with a Release of the same weight — Release "releases the semaphore with a weight of n" — and defer it so a panic can't strand capacity.

sem := semaphore.NewWeighted(int64(maxInFlight))
for _, job := range jobs {
	if err := sem.Acquire(ctx, job.cost); err != nil { // blocks; respects ctx
		break // ctx canceled — stop launching
	}
	go func(j Job) {
		defer sem.Release(j.cost)
		handle(j)
	}(job)
}

Choose by need: errgroup.SetLimit when tasks are uniform and you want error/cancel propagation for free; semaphore.Weighted when tasks have unequal cost (e.g. weight by payload size) or you need TryAcquire (non-blocking) to shed load. Mutex/atomic correctness of the underlying primitives → go-sync-primitives.

5. The Classic Fixed-N Channel Worker Pool

errgroup/semaphore spawn one goroutine per task (bounded). The alternative — N long-lived workers draining a shared jobs channel — avoids per-task goroutine churn entirely and is the canonical shape when the same workers run for the lifetime of a stage. It is the Go blog's pattern: "a fixed number of digester goroutines that receive file names from paths and send results" (Go Pipelines).

func pool[T, R any](ctx context.Context, n int, in <-chan T, fn func(T) R) <-chan R {
	out := make(chan R, n) // buffered: workers don't block handing off results
	var wg sync.WaitGroup
	wg.Add(n)
	for range n { // N workers share `in` (fan-out) and share `out` (fan-in)
		go func() {
			defer wg.Done()
			for job := range in { // each worker pulls until `in` is closed
				select {
				case out <- fn(job):
				case <-ctx.Done():
					return
				}
			}
		}()
	}
	go func() { wg.Wait(); close(out) }() // close out once all workers exit
	return out
}

The workers share one input channel — that is the fan-out; they share one output channel — that is the fan-in. Because "multiple goroutines are sending on a shared channel," no single worker may close out; a separate goroutine closes it after wg.Wait (Go Pipelines). Who-closes / range-ends-on-close semantics → go-channels-select; the WaitGroup lifecycle → go-concurrency-goroutines. Reuse this pool for many jobs; it amortizes goroutine creation to once per stage instead of once per item.

6. The Channel-as-Queue Has Real Cost — Batch to Amortize

A buffered channel is not free plumbing. Every <- and <--send takes the channel's internal lock and copies the element (go-perfbook; src/runtime/chan.go). At low rates this is invisible; at high throughput, one channel op per item can dominate the actual work. The lever is batching: hand a []Item across the channel instead of one Item, so a slice of 256 costs one lock+copy instead of 256.

// WRONG (high throughput): one channel send + one wake per item
for _, it := range items {
	jobs <- it // lock + copy + potential goroutine wake, every iteration
}

// RIGHT: amortize the per-send overhead over a batch
const batch = 256
for i := 0; i < len(items); i += batch {
	end := min(i+batch, len(items))
	jobs <- items[i:end] // one send carries `batch` items; worker ranges the slice
}

Batching also amortizes the worker side: one wake processes N items, not one. The same logic applies to a mutex-guarded queue (do N items under one Lock) — contention reduction at depth is go-perf-contention-and-sharding; "reduce the shared or reduce the mutable" (go-perfbook). Whether a channel pipeline or a sharded mutex queue wins is workload-specific — measure (→ go-perf-methodology); don't assume channels are the cheap option.

Also avoid the unbuffered-handoff ping-pong: an unbuffered channel forces the sender to block until a receiver is ready on every item, serializing the two goroutines into a rendezvous. For a throughput pipeline give the channel a buffer so producer and consumer overlap (unbuffered-vs-buffered semantics → go-channels-select).

7. Backpressure Is the Point of a Bounded Queue

A bounded queue is not just a memory cap — it is the backpressure signal. When the buffer fills, sends block, which slows the producer to the rate consumers can sustain. An unbounded queue (or unbounded goroutine fan-out) removes that signal: the producer races ahead, the queue grows without limit, and memory climbs until the process is killed — the same "more memory than is available" failure as §1 (Go Pipelines).

So the bound (the channel buffer size, SetLimit(n), or NewWeighted(n)) does double duty: it caps resource use and it couples producer speed to consumer speed. Size it to the smallest number that keeps the bottleneck resource busy — large enough to absorb jitter, small enough to apply backpressure before memory grows. And always honor cancellation on the blocking path (Acquire(ctx, …), errgroup.WithContext) so a slow consumer doesn't wedge producers forever; context propagation depth → go-context.

8. Routing to Related Skills

This plugin (go-perf-* depth):

• go-perf-goroutines-scheduler — goroutine cost, GMP/work-stealing, container-aware GOMAXPROCS (1.25), schedtrace; the "what does a goroutine cost / how many cores do I really have" layer beneath pool sizing.
• go-perf-contention-and-sharding — when the queue is a contended mutex/sync.Map: shard it, batch under one lock.
• go-perf-methodology — the measure-first gate: is more concurrency the bottleneck, and did adding workers actually raise throughput (benchstat, not eyeballing).
• go-perf-block-mutex-profiles — find the contended channel/lock the pool is stalling on.

Sibling marketplace (go-* correctness):

• go-channels-select — channel directionality, who closes, comma-ok, select, unbuffered-vs-buffered semantics.
• go-concurrency-goroutines — goroutine lifetime/leaks, WaitGroup, errgroup correctness, "keep libraries synchronous."
• go-sync-primitives — Mutex/atomic correctness, copylock.
• go-context — cancellation/deadline propagation through the pool.

9. Don't

• Don't spawn one goroutine per item with no ceiling — bound it (SetLimit/semaphore); unbounded fan-out "may allocate more memory than is available" (Go Pipelines).
• Don't size a CPU-bound pool far above GOMAXPROCS — extra workers queue for the same cores and only add switch/cache cost (runtime.GOMAXPROCS).
• Don't pick a worker count by guessing — "I/O-bound = higher" is a hypothesis; measure the throughput plateau (→ go-perf-methodology).
• Don't leave an errgroup unconfigured and call it bounded — a zero Group "has no limit on the number of active goroutines" (errgroup).
• Don't call SetLimit while workers are running — "the limit must not be modified while any goroutines in the group are active" (errgroup).
• Don't Acquire without a matching Release (defer it) — a stranded weight permanently shrinks the pool (semaphore).
• Don't send one item per channel op on a hot path — batch a slice per send to amortize the lock+copy (go-perfbook).
• Don't use an unbounded queue — you lose backpressure and grow memory until OOM; the bound is the throttle (Go Pipelines).

10. Reference Files

Worker-pool/throughput anti-patterns in LLM-generated Go, each with wrong/right contrast and citations:

references/common-mistakes.md

Source provenance for every claim in this skill:

references/sources.yaml