gpu-host-tuning
Host-side tuning + audit for Linux GPU inference servers. Sits beneath any
inference framework (vLLM, sglang, TensorRT-LLM, llama.cpp).
Assumes a working host. Driver, nvidia-fabricmanager, and NVLink/NVSwitch
must already be healthy — if nvidia-smi doesn't list every GPU, or
fabricmanager isn't active (running), that is a bring-up problem, not a
tuning one: use nvidia-datacenter-bringup (same inference-host plugin)
first. Tuning a half-initialised host produces numbers that describe the fault,
not the hardware.
Three modes:
- Audit — read-only snapshot
- Bench — ground-truth pinned-host↔GPU memcpy ceiling
- Tune — apply individual levers from the cheat-sheet
This file is a pointer map. The actual logic lives in scripts/ and the
authoritative references in references/.
Quick start
# From the skill directory — typically ~/.claude/skills/gpu-host-tuning
# (personal) or .claude/skills/gpu-host-tuning (project install).
# Audit (read-only, ~60s)
./scripts/collect.sh
# Audit + pinned-memcpy bench (needs torch + CUDA, ~5 min)
./scripts/collect.sh --bench
The script prompts for the output parent dir on first interactive run and
remembers the choice. Override via --out <dir> or HOST_AUDIT_DIR=<dir>.
Default snapshot dirname is gpu-host-tuning-<host>-<UTC>.
What the snapshot captures
One file per probe, numbered by section. See
references/probe-interpretation.md
for the full file-by-file decoder.
| Section |
What |
00-09 meta |
collector version, run timestamp, args |
10-19 system + firmware |
dmidecode (BIOS, CPU, memory DIMMs), lshw, /sys/class/dmi |
20-29 CPU + power + C-states |
governor, EPP, intel_pstate / amd_pstate, cpuidle states + disable mask, turbostat 5s residency, microcode, vulnerabilities, thermal zones |
30-39 memory + NUMA |
numactl -H, /proc/meminfo, THP, numa_balancing, vm tunables, hugepages |
40-49 kernel + limits |
uname, /etc/os-release, /proc/cmdline, sysctl -a, ulimit, /sys/devices/system/cpu/vulnerabilities, dmesg, IRQ affinity, env vars in vllm processes |
50-59 PCIe |
lspci tree + verbose, AER counters, link width/speed for every NVIDIA device |
60-69 GPU |
nvidia-smi -q full, topo -m, nvlink --status, clocks/power/ECC, dmon 5s, dcgmi diag |
70-79 network |
NICs, IB (ibstat / ibv_devinfo), ethtool ring sizes, RDMA links |
80-89 storage |
lsblk, NVMe id-ctrl, smartctl, mount flags, io scheduler |
90-99 container runtime |
containerd version, CDI specs, cgroup v2, kubelet config, RKE2 config |
Three modes — what each maps to
| Mode |
What |
Reference |
| Audit |
./scripts/collect.sh writes the snapshot directory |
references/probe-interpretation.md decodes each numbered file |
| Bench |
./scripts/collect.sh --bench adds the pinned-memcpy CSV |
references/session-findings.md lists baselines per chassis |
| Tune |
No script — apply individual levers from the cheat-sheet |
references/recommended-tunings.md (lever-by-lever) and references/tuned-profiles.md (apply via tuned-adm) |
When to use which reference
| Goal |
Read |
| Apply NVIDIA's stock DGX tunings |
references/tuned-profiles.md |
| See exactly what NVIDIA's settings packages flip (per-platform JSON, GRUB drop-ins, sysctl, units) |
references/nvidia-dgx-config-decoder.md |
| Run a proper bring-up flow |
references/bringup-recipe.md |
| Find the lever the audit flagged |
references/recommended-tunings.md |
| Decode an audit output file |
references/probe-interpretation.md |
| Tune a Dell XE9680 (H100/H200, SPR/EMR) |
references/dell-xe9680.md |
| Tune a Dell XE9780 / XE9780L (B200/B300, Granite Rapids) |
references/dell-xe9780.md |
| Understand why cpufreq/cpuidle is empty inside a cloud VM |
references/virt-and-cloud-quirks.md |
| See measured baselines from real boxes |
references/session-findings.md |
Comparing two snapshots
Two snapshots on the same host (e.g., pre-tune and post-tune) can be
compared with diff -ruN snap_pre/ snap_post/. For a structured impact
ranking, use references/probe-interpretation.md to interpret deltas.
Companion skills
1---2name: gpu-host-tuning3description: Audit AND tune Linux/GPU inference hosts — read-only host snapshot (CPU power state, C-states, NUMA topology, PCIe link state, GPU settings, kernel boot params, sysctl, ulimits, IRQ affinity, container runtime), optional pinned-host↔GPU memcpy bench (torch + numactl), and per-lever cheat-sheets to flip settings (governor, EPP, cpuidle, persistence, ECC, hugepages, intel_iommu, NCCL env, tuned-adm profiles, Dell/Supermicro/HPE BIOS guidance). Sits beneath any inference framework (vLLM, sglang, TensorRT-LLM) — about the host, not the framework.4---56# gpu-host-tuning78Host-side tuning + audit for Linux GPU inference servers. Sits *beneath* any9inference framework (vLLM, sglang, TensorRT-LLM, llama.cpp).1011**Assumes a working host.** Driver, `nvidia-fabricmanager`, and NVLink/NVSwitch12must already be healthy — if `nvidia-smi` doesn't list every GPU, or13fabricmanager isn't `active (running)`, that is a bring-up problem, not a14tuning one: use `nvidia-datacenter-bringup` (same `inference-host` plugin)15first. Tuning a half-initialised host produces numbers that describe the fault,16not the hardware.1718Three modes:19201. **Audit** — read-only snapshot212. **Bench** — ground-truth pinned-host↔GPU memcpy ceiling223. **Tune** — apply individual levers from the cheat-sheet2324This file is a pointer map. The actual logic lives in `scripts/` and the25authoritative references in `references/`.2627## Quick start2829```bash30# From the skill directory — typically ~/.claude/skills/gpu-host-tuning31# (personal) or .claude/skills/gpu-host-tuning (project install).3233# Audit (read-only, ~60s)34./scripts/collect.sh3536# Audit + pinned-memcpy bench (needs torch + CUDA, ~5 min)37./scripts/collect.sh --bench38```3940The script prompts for the output parent dir on first interactive run and41remembers the choice. Override via `--out <dir>` or `HOST_AUDIT_DIR=<dir>`.42Default snapshot dirname is `gpu-host-tuning-<host>-<UTC>`.4344## What the snapshot captures4546One file per probe, numbered by section. See47[`references/probe-interpretation.md`](references/probe-interpretation.md)48for the full file-by-file decoder.4950| Section | What |51|---|---|52| `00-09` meta | collector version, run timestamp, args |53| `10-19` system + firmware | dmidecode (BIOS, CPU, memory DIMMs), lshw, /sys/class/dmi |54| `20-29` CPU + power + C-states | governor, EPP, intel_pstate / amd_pstate, cpuidle states + disable mask, turbostat 5s residency, microcode, vulnerabilities, thermal zones |55| `30-39` memory + NUMA | numactl -H, /proc/meminfo, THP, numa_balancing, vm tunables, hugepages |56| `40-49` kernel + limits | uname, /etc/os-release, /proc/cmdline, sysctl -a, ulimit, /sys/devices/system/cpu/vulnerabilities, dmesg, IRQ affinity, env vars in vllm processes |57| `50-59` PCIe | lspci tree + verbose, AER counters, link width/speed for every NVIDIA device |58| `60-69` GPU | nvidia-smi -q full, topo -m, nvlink --status, clocks/power/ECC, dmon 5s, dcgmi diag |59| `70-79` network | NICs, IB (ibstat / ibv_devinfo), ethtool ring sizes, RDMA links |60| `80-89` storage | lsblk, NVMe id-ctrl, smartctl, mount flags, io scheduler |61| `90-99` container runtime | containerd version, CDI specs, cgroup v2, kubelet config, RKE2 config |6263## Three modes — what each maps to6465| Mode | What | Reference |66|---|---|---|67| **Audit** | `./scripts/collect.sh` writes the snapshot directory | `references/probe-interpretation.md` decodes each numbered file |68| **Bench** | `./scripts/collect.sh --bench` adds the pinned-memcpy CSV | `references/session-findings.md` lists baselines per chassis |69| **Tune** | No script — apply individual levers from the cheat-sheet | `references/recommended-tunings.md` (lever-by-lever) and `references/tuned-profiles.md` (apply via `tuned-adm`) |7071## When to use which reference7273| Goal | Read |74|---|---|75| Apply NVIDIA's stock DGX tunings | [`references/tuned-profiles.md`](references/tuned-profiles.md) |76| See exactly what NVIDIA's settings packages flip (per-platform JSON, GRUB drop-ins, sysctl, units) | [`references/nvidia-dgx-config-decoder.md`](references/nvidia-dgx-config-decoder.md) |77| Run a proper bring-up flow | [`references/bringup-recipe.md`](references/bringup-recipe.md) |78| Find the lever the audit flagged | [`references/recommended-tunings.md`](references/recommended-tunings.md) |79| Decode an audit output file | [`references/probe-interpretation.md`](references/probe-interpretation.md) |80| Tune a Dell XE9680 (H100/H200, SPR/EMR) | [`references/dell-xe9680.md`](references/dell-xe9680.md) |81| Tune a Dell XE9780 / XE9780L (B200/B300, Granite Rapids) | [`references/dell-xe9780.md`](references/dell-xe9780.md) |82| Understand why cpufreq/cpuidle is empty inside a cloud VM | [`references/virt-and-cloud-quirks.md`](references/virt-and-cloud-quirks.md) |83| See measured baselines from real boxes | [`references/session-findings.md`](references/session-findings.md) |8485## Comparing two snapshots8687Two snapshots on the same host (e.g., pre-tune and post-tune) can be88compared with `diff -ruN snap_pre/ snap_post/`. For a structured impact89ranking, use `references/probe-interpretation.md` to interpret deltas.9091## Companion skills9293- [`vllm-nvidia-hardware`](../vllm-nvidia-hardware/) — per-SKU specs (HBM, TDP, NVLink, PCIe gen)94- [`vllm-deployment`](../vllm-deployment/) — K8s manifest authoring, cache mounts, probes95- [`vllm-performance-tuning`](../vllm-performance-tuning/) — vLLM-side knobs (above this skill's layer)
air-gapped