Go Perf Zerocopy And Syscalls

Go Zero-Copy & Syscall Reduction

Whether to reach this low is a methodology call — go-perf-methodology and the measure-first loop come first. Most services should use bufio + io.Copy and never touch a raw syscall. Reach here only when a CPU profile or strace -c shows the cost is in read/write/copy at the OS boundary. The good news: the biggest win — sendfile — is something io.Copy already does for you, if you don't defeat it.

All Go below builds clean under gofmt, go vet, and go test on Go 1.25 / 1.26.

1. io.Copy Is Already Zero-Copy — If You Keep the Fast Path

io.Copy(dst, src) is not a dumb userspace read/write loop. Its contract: "If src implements WriterTo, the copy is implemented by calling src.WriteTo(dst). Otherwise, if dst implements ReaderFrom, the copy is implemented by calling dst.ReadFrom(src)" (io.Copy). And critically for allocation: in CopyBuffer, "If either src implements WriterTo or dst implements ReaderFrom, buf will not be used to perform the copy" (io.CopyBuffer) — the staging buffer is skipped entirely.

*net.TCPConn and *os.File implement these interfaces with kernel zero-copy on Linux. The dispatch is explicit in the standard library — (*TCPConn).readFrom tries three paths in order (src/net/tcpsock_posix.go):

func (c *TCPConn) readFrom(r io.Reader) (int64, error) {
	if n, err, handled := spliceFrom(c.fd, r); handled { // socket→socket: splice(2)
		return n, err
	}
	if n, err, handled := sendFile(c.fd, r); handled {    // file→socket: sendfile(2)
		return n, err
	}
	return genericReadFrom(c, r)                           // fallback: userspace copy
}

So io.Copy(tcpConn, file) becomes a sendfile(2) syscall — data goes disk → socket inside the kernel, never round-tripping through your process's memory:

// file → socket: this is sendfile(2). No read()/write() loop, no userspace buffer.
n, err := io.Copy(tcpConn, file) // tcpConn is *net.TCPConn, file is *os.File

sendFile "copies the contents of r to c using the sendfile system call to minimize copies" and bottoms out in poll.SendFile → syscall.Sendfile (src/net/sendfile.go, src/internal/poll/sendfile_unix.go).

The conditions that keep the fast path (this is where LLM code fails):

• Pass the concrete types, unwrapped. sendFile requires r.(syscall.Conn) to succeed — an *os.File qualifies (sendfile.go). Wrap the file in a bufio.Reader and that assertion fails → you fall back to genericReadFrom, a userspace copy. A bufio.Writer around the conn likewise hides the *TCPConn, so ReadFrom is never the TCP one.
• io.LimitedReader is fine — it's unwrapped. sendFile/spliceFrom peel off a *io.LimitedReader and honor its N, so io.Copy(conn, io.LimitReader(file, n)) still uses sendfile (sendfile.go).
• io.MultiReader, io.TeeReader, a custom wrapper, or a []byte source defeat it — none implement syscall.Conn, so you drop to the generic loop.
• net/http already does this. http.ServeContent/ServeFile and io.Copy(w, file) where w is the server's http.ResponseWriter use ReadFrom/sendfile.

2. socket→socket (splice) and file→file (copy_file_range)

The same ReaderFrom machinery covers two more kernel fast paths — you keep the concrete types, you never name the syscall:

• socket → socket = splice(2) (the proxy/tunnel hot path). io.Copy(dstConn, srcConn) between two TCP (or stream Unix) connections goes through spliceFrom → poll.Splice, moving data via a kernel pipe "to minimize copies of data from and to userspace … src and dst must both be stream-oriented sockets"; it only engages for *TCPConn/stream *UnixConn sources (src/internal/poll/splice_linux.go, src/net/splice_linux.go).
• file → file = copy_file_range(2). io.Copy(dstFile, srcFile) with two *os.Files goes through (*File).readFrom, which tries copyFileRange first, then spliceToFile (src/os/zero_copy_linux.go). copy_file_range is gated on kernel ≥ 5.3 (older kernels are buggy) and falls back cleanly otherwise (src/internal/poll/copy_file_range_linux.go).
• file → socket via (*os.File).WriteTo also lands on sendfile, but only when the destination is a TCP/Unix stream socket (zero_copy_linux.go).

O_APPEND destination files disable the zero-copy path — neither copy_file_range nor splice supports O_APPEND dsts, so os skips them (zero_copy_linux.go).

// proxy: client conn → backend conn becomes splice(2) in each direction.
go func() { io.Copy(backend, client) }() // both *net.TCPConn
io.Copy(client, backend)

3. net.Buffers — Batch N Writes Into One writev(2)

When you have several discrete byte slices to send (header + body + trailer, or framed messages), N separate conn.Write calls are N write(2) syscalls. net.Buffers coalesces them into one writev(2):

"Buffers contains zero or more runs of bytes to write. On certain machines, for certain types of connections, this is optimized into an OS-specific batch write operation (such as 'writev')." — net.Buffers

Buffers is type Buffers [][]byte; its WriteTo implements io.WriterTo (net.Buffers). On a *net.TCPConn/*net.UnixConn it reaches (*netFD).writeBuffers → (*poll.FD).Writev, which builds an iovec array and issues writev — up to 1024 iovecs per syscall (src/net/writev_unix.go, src/internal/poll/writev.go).

parts := net.Buffers{header, body, trailer}
_, err := parts.WriteTo(conn) // one writev(2), not three write(2) — conn is *net.TCPConn

Caveats from the source: WriteTo "modifies the slice v as well as v[i]" — it consumes the Buffers as it drains, so don't reuse the value after a partial write without resetting it (net.Buffers). The fast path only exists for unix stream connections; elsewhere it degrades to sequential writes. For small slices, a single bufio.Writer + Flush is simpler and just as few syscalls; net.Buffers wins when the slices are large and already separate (avoiding the copy into a buffer). Buffer sizing and Flush discipline → go-perf-buffered-io.

4. mmap — Random Access to Large Read-Mostly Files

For a large file you index into randomly (a search index, a memory-mapped database, a column store), mmap maps the file into your address space so reads become page faults, not pread syscalls — and the page cache is shared, not duplicated into a Go buffer. golang.org/x/exp/mmap is the safe wrapper:

r, err := mmap.Open("index.dat") // memory-maps the file for reading
if err != nil { /* ... */ }
defer r.Close()
b := r.At(off)                       // single byte, no syscall
n, err := r.ReadAt(buf, off)         // io.ReaderAt, no syscall on a resident page
_ = r.Len()                          // file length

Open "memory-maps the named file for reading"; ReaderAt "reads a memory-mapped file" and implements io.ReaderAt (x/exp/mmap).

Caveats — mmap is not free and not always a win:

• Concurrency rule, from the docs: "clients can execute parallel ReadAt calls, but it is not safe to call Close and reading methods concurrently" (mmap). A read after Close (unmap) is a segfault, not a Go panic — the lifetime is yours.
• Page faults aren't free. Cold pages are major faults (disk I/O) that block the OS thread, stalling the scheduler differently than an async read. For sequential streaming, bufio + read readahead often beats mmap.
• Not for small files — mmap/munmap + page-table setup dwarfs a couple of read calls.
• Not for actively-written / truncatable files — a truncate under you makes the next touch SIGBUS.

For raw control (anonymous maps, MAP_SHARED, madvise), syscall.Mmap / golang.org/x/sys/unix expose the bytes directly with the same hazards and no wrapper safety.

5. Reducing Syscalls Generally

The unifying principle: a syscall has fixed overhead (mode switch, ~hundreds of ns) that a small payload can't amortize. Levers, cheapest first:

1. Buffer so one syscall carries many operations — bufio.Writer turns a byte-at-a-time loop into one write per buffer fill. The default answer → go-perf-buffered-io.
2. Batch discrete sends with net.Buffers/writev (§3) instead of N writes.
3. Let the kernel copy with sendfile/splice/copy_file_range via io.Copy (§1–2) so the bytes never enter your process at all.
4. Drop to raw syscall / golang.org/x/sys/unix only when the stdlib can't express the call (readv, recvmmsg, SO_REUSEPORT, O_DIRECT). Last resort: you lose portability and poller integration and must handle EINTR/EAGAIN yourself. Profile first (strace -c → go-perf-os-tooling).

A typical HTTP/gRPC service spends almost nothing in raw syscalls relative to allocation and serialization — confirm the cost is real before reaching below bufio + io.Copy.

6. Routing to Related Skills

• go-perf-buffered-io — bufio Reader/Writer/Scanner sizing, Flush, io.Copy/ io.CopyBuffer basics, ReaderFrom/WriterTo fast-path overview. The default I/O layer.
• go-perf-strings-bytes-zerocopy — unsafe.String/unsafe.Slice zero-copy []byte↔string conversion and its read-only/lifetime rules. (Different "zero copy.")
• go-perf-os-tooling — strace -c, perf, flame graphs to prove the cost is in syscalls/copies before you optimize here.
• go-perf-methodology — should-I-optimize, the measure-first loop, picking the diagnostic.
• go-perf-worker-pools-throughput — concurrency around I/O (many conns, fan-in/out).

7. Don't

• Don't hand-roll a read/write loop to send a file over a socket. io.Copy(conn, file) is already sendfile(2) (src/net/sendfile.go).
• Don't wrap the source *os.File in a bufio.Reader before io.Copy to a conn — the syscall.Conn assertion fails and you silently lose sendfile (sendfile.go).
• Don't issue N Write calls for header+body+trailer — use net.Buffers for one writev(2) (net.Buffers).
• Don't reuse a net.Buffers value after a partial WriteTo without resetting it — it's consumed in place (net.Buffers).
• Don't mmap a small or sequentially-streamed file — page-fault and setup overhead lose to bufio + read; mmap is for large random-access, read-mostly data.
• Don't read or ReadAt an mmap.ReaderAt after Close — that's a segfault, not a panic (mmap).
• Don't drop to raw syscall.Mmap/writev/readv before profiling proves syscall cost — most services never need to ([go-perf-os-tooling]).
• Don't assume zero-copy on every OS. sendfile/splice/copy_file_range/writev are Linux/Unix fast paths; on other platforms the same code is correct but falls back to a copy.

8. Reference Files

references/common-mistakes.md — wrong/right zero-copy and syscall patterns with citations.

references/sources.yaml — source provenance for every claim.