Linux Sysctl Tuning

Sysctl Performance Tuning

Distro support

One-family-pair skill: the sysctl interface, the /etc/sysctl.d/ drop-in directory, and sysctl --system are identical on Debian/Ubuntu and the RHEL family (Fedora, RHEL, CentOS Stream, Rocky, Alma, Oracle). The kernel exposes the same net.* and vm.* keys on both. Only the package that ships a given module differs.

[GROUNDING-GAP: BBR/swappiness/TCP-buffer perf tuning — thin/absent in the RHCSA grounding corpus; authored from kernel networking docs (Documentation/ networking, Documentation/admin-guide/sysctl) and tcp(7)/sysctl(8) man pages. Deepen with Systems Performance 2e (Brendan Gregg).]

Use When

• Raising socket buffers / TCP window for high-throughput (10GbE+) transfers.
• Scaling a server to many concurrent connections (somaxconn, syn backlog).
• Enabling BBR congestion control with the fq qdisc.
• Tuning memory writeback / swap behaviour (swappiness, dirty_ratio).

Do Not Use When

• The goal is SECURITY hardening (rp_filter, tcp_syncookies, kptr_restrict, redirects). That sysctl lives in linux-server-hardening (references/sysctl-reference.md) — cross-reference it, do not duplicate it.
• The bottleneck is per-service CPU/memory share; use linux-service-priority (cgroup limits) instead.
• You are diagnosing, not tuning; profile first with linux-perf-profiling.

Required Inputs

• The workload class (throughput-bound transfer, connection-heavy server, or balanced general server).
• The NIC speed and expected concurrent-connection count.
• Whether a maintenance window applies (most keys are live, but verify).

Capability Contract

Reading sysctls and drop-ins is read-only. Live writes, drop-in edits, module loading, load generation, and rollback require explicit authority. Security sysctls route to linux-server-hardening.

Degraded Mode

Without representative load testing, return candidate keys and an experiment design only. If a key/algorithm is unavailable, do not force it. If rollback cannot be observed, do not apply the change.

Decision Rules

Workflow

1. Measure first — profile with linux-perf-profiling so you tune the real bottleneck, not a guess.

2. Show the current live value of each candidate key (sysctl <key>).

3. Write the delta to a drop-in in /etc/sysctl.d/ (never edit /etc/sysctl.conf or vendor files in /usr/lib/sysctl.d/).

4. Apply with sudo sysctl --system and re-read the keys to confirm.

5. Load-test, then keep or roll back by deleting the one drop-in.

6. Stop when target improvement is absent or a guardrail regresses; recover by removing the owned drop-in, restoring recorded live values, and rerunning the baseline test.

Quality Standards

• One owned drop-in file, high number prefix (e.g. 60-/99-) so it wins.
• Set only keys you can justify against a measured bottleneck.
• Confirm BBR is available before selecting it (tcp_available_congestion_control).

Anti-Patterns

• Copying a tuning blob. Fix: link every key to measured saturation and a hypothesis.

• Editing vendor/global config. Fix: use one owned drop-in with documented precedence.

• Claiming improvement without comparable load. Fix: repeat baseline and candidate under identical workload.

• Forcing unavailable BBR or qdisc support. Fix: inspect kernel features before selection.

• Keeping a regressing change. Fix: enforce guardrails and restore prior effective values.

• Copy-pasting a giant "ultimate sysctl" blob without measuring.

• Editing /etc/sysctl.conf directly (use a drop-in — easy diff/rollback).

• Setting vm.swappiness=0 (can trigger OOM under pressure; 10 is the safe low).

• Mixing security and performance keys in one file (split ownership).

• Claiming improvement without a comparable load test. Correction: repeat baseline and candidate runs under the same workload.

• Ignoring config precedence. Correction: inspect effective values and every supplying drop-in before choosing the filename.

Outputs

Evidence Produced

Capture kernel, feature availability, config provenance, before/after keys, load-test parameters and metrics, errors, guardrails, and rollback result.

Worked Example

After profiling shows a connection-accept queue bottleneck, record current somaxconn, application backlog, and saturation, test one bounded increase under the same load, then persist only if latency improves without memory or error regression.

• The drop-in file written and the keys it changed.
• Before/after values for each tuned key.
• The load-test result that justified (or reverted) the change.

sysctl basics

sysctl net.core.somaxconn                 # read one key
sysctl -a | grep tcp_congestion           # search
sudo sysctl -w vm.swappiness=10           # transient (lost on reboot)

# Persist: a drop-in is read at boot and by `sysctl --system`.
# Files are read in lexical order across /usr/lib, /run, /etc — last wins.
sudo tee /etc/sysctl.d/60-perf.conf >/dev/null <<'EOF'
net.core.somaxconn = 65535
vm.swappiness = 10
EOF
sudo sysctl --system                      # apply ALL drop-ins

High-throughput networking

# Socket buffer ceilings (bytes) — needed for large bandwidth-delay products.
net.core.rmem_max = 134217728
net.core.wmem_max = 134217728
# Per-socket TCP autotuning: min default max
net.ipv4.tcp_rmem = 4096 87380 67108864
net.ipv4.tcp_wmem = 4096 65536 67108864

# BBR congestion control — pair it with the fq qdisc (BBR's pacing needs fq).
net.core.default_qdisc = fq
net.ipv4.tcp_congestion_control = bbr

# Verify BBR is usable BEFORE setting it:
sysctl net.ipv4.tcp_available_congestion_control     # must list 'bbr'
sudo modprobe tcp_bbr                                 # load if absent

Connection scaling

net.core.somaxconn = 65535          # accept-queue depth (also raise app backlog!)
net.ipv4.tcp_max_syn_backlog = 8192 # half-open (SYN_RECV) queue
net.core.netdev_max_backlog = 16384 # per-CPU packet ingress queue
net.ipv4.ip_local_port_range = 1024 65535   # ephemeral ports for outbound
net.ipv4.tcp_tw_reuse = 1           # reuse TIME_WAIT sockets for new outbound

somaxconn only caps the kernel queue; the application's listen() backlog must be raised too (e.g. nginx backlog=, the app's listen() argument).

Memory behaviour

vm.swappiness = 10            # 0..200; lower = avoid swap, keep page cache (NOT 0)
vm.dirty_ratio = 20          # % RAM dirty before a writing process blocks
vm.dirty_background_ratio = 5 # % RAM dirty before kernel starts async flush
vm.overcommit_memory = 0     # 0=heuristic, 1=always, 2=strict (with overcommit_ratio)

Cross-reference: security sysctl

SECURITY-focused net.*/kernel.* tunables (rp_filter, tcp_syncookies, kptr_restrict, accept_redirects, core-dump hygiene) are owned by the linux-server-hardening skill — see its references/sysctl-reference.md. Keep security keys in that skill's drop-in and performance keys in this one; do not move or duplicate either set.

Optional fast path (when sk-* scripts are installed)

Scripts

This skill installs the following scripts to /usr/local/bin/. To install:

sudo install-skills-bin linux-sysctl-tuning

References

• references/sysctl-tuning-reference.md