Gpu Computing Standards

GPU computing standards

Criteria verified as of August 2026. Re-verify on the web before committing to anything (§8).

1. Scope and triggers

Applies when the GPU is an infrastructure resource that has to be installed, shared, measured, powered, cooled and paid for — whether it serves inference, training, scientific computing, simulation or transcoding:

• The stack and its coupled versions: driver ↔ CUDA runtime ↔ framework.
• Installation on Linux: open modules (nvidia-open) vs. proprietary, DKMS, Secure Boot and module signing, nouveau, nvidia-persistenced.
• Containers with GPU: NVIDIA Container Toolkit, nvidia-ctk, CDI; on Kubernetes, device plugin, GPU Operator and the DRA driver.
• Sharing the GPU: MIG, MPS, time-slicing, and what isolation each one gives.
• Monitoring: nvidia-smi, DCGM, dcgm-exporter, XID, ECC, throttling.
• Physical and sizing: TDP, power, cooling, rack density.
• Cost: buy vs. rent, utilisation as a FinOps metric, job queues.
• Alternatives to CUDA: AMD ROCm/HIP, and their real maturity.
• Reliability: XID, GPU fallen off the bus, ECC, RMA.

Not applicable: see local-inference-standards (serving models: engine choice, quantisation, KV cache sizing, OpenAI endpoint and its security — if the server has no GPU (CPU, Apple Silicon), it is still theirs and not this skill's; here rules when the problem is the GPU, there when the problem is the model), libvirt-kvm-standards and proxmox-ve-standards (GPU passthrough to a VM: THEIRS, not here — VFIO, vfio-pci, IOMMU groups, intel_iommu=on/amd_iommu=on, vBIOS dumping, card reset and the loss of live migration are covered in their §3.8; this skill goes up to the hypervisor boundary and does not cross it: what does belong here is the driver inside the guest, non-virtualised sharing and everything in §5-§6), kubernetes-standards (manifests, Helm, GitOps, admission policies — here only the criterion of which GPU resource is requested and how), container-runtime-security-standards (seccomp, container escape, --privileged — here only NVIDIA's toolkit-specific surface, §5.2), podman-systemd-containers-standards (--device nvidia.com/gpu=all in a Quadlet unit), onprem-standards (platform umbrella: rack, redundant power, UPS, OOB management plane, hardware life cycle — this skill is a layer inside their §1.2 and respects their §1.3 invariants; the GPU does not exempt you from telemetry, tested backup or fencing), homelab-standards (GPU at home: there budget, noise, consumption and proportionality rule — a second-hand 3090 in a lab is not sized with data centre criteria), linux-administration-standards (systemd, kernel, packages), rhel-fedora-standards (dnf module, akmods, rpm-ostree with a driver), linux-hardening-standards (CIS baseline, modprobe.d, sysctl), selinux-standards (device node contexts and policies), observability-standards (Prometheus, PromQL, alerts — here only which GPU metric matters and why), sre-practice-standards (SLOs, capacity as a practice), iac-standards and cicd-standards (automating installation and version pinning), vulnerability-management-standards (triage and SLA for the CVEs in §5), bcdr-standards and backup-recovery-standards (continuity of a GPU cluster), networking-standards (the cluster's data network), high-speed-interconnect-standards (the compute network is theirs, and it is not designed with the data network's criteria: InfiniBand, RoCE v2, iWARP, opensm and P_Key partitions, UCX, NCCL/RCCL and its network backend, GPUDirect RDMA, NVMe over Fabrics, and the criterion for when NVMe/TCP is enough. Here the accelerator and its consumption of that network; there the network), firewall-policy-standards, identity-access-management-standards, grc-compliance-standards, datacenter-facilities-standards (the room: density per rack, electrical distribution and liquid cooling — CDU, loop, aisle — are theirs; here the TDP and the thermal requirement of the accelerator handed to them as a datum), python-standards, julia-standards and r-standards (code that uses the GPU), fortran-standards (reciprocal, already declared from their §1: the !$acc/!$omp target written in the .f90 and its correctness are theirs; the kernel, the occupancy and the GPU programming model, here), cpp-standards and c-standards (CUDA/HIP kernels are C++ and that closeness is confusing: the GPU programming model — thread hierarchy, shared memory, occupancy, streams, coalescing — belongs here; the host C++ — standard, RAII, dependency management, clang-tidy, tests — is theirs), green-it-standards (rack density, TDP, liquid cooling and consumption as a physical limit belong here; their footprint accounting — energy and embodied — and the criteria for reporting it, theirs), webgl-webgpu-standards (they are two different GPUs: here the server's, which is provisioned, shared, monitored and paid for — driver, CUDA/ROCm, MIG, DCGM, rack density —; there the client's, seen through the browser, with a permission and context-loss model that does not exist on the server), assembly-standards (PTX and SASS are assembly; the criterion for when you drop to that level, how it is justified with a measurement and how it is maintained — including its expiry when the microarchitecture changes — is theirs), llm-app-engineering-standards, rag-standards, ai-agents-standards, mcp-standards (AI application layer), and claude-api (canonical reference for Anthropic's API: the alternative to buying a GPU is not buying it — no Claude model datum, price or limit is asserted from memory).

Also: mlops-standards (training, experiments and model life cycle — the hardware is here, the pipeline is there), llm-evaluation-standards, mlsecops-standards (model artifact supply chain), ai-governance-standards.

2. Default decisions

Verify the latest version on the web before fixing it in a real project (§8).

2.1 The stack and its coupled versions

It is the number one source of "it doesn't work". Four layers that are versioned separately and break together:

kernel driver (nvidia.ko / nvidia-open)
   └── CUDA driver API (libcuda.so — shipped by the DRIVER, not the toolkit)
        └── CUDA runtime / toolkit (libcudart, cuBLAS, cuDNN, NCCL)
             └── framework (PyTorch, JAX, TensorRT, inference engine)

What must be clear and not confused:

• The driver is backward compatible: an application compiled against an old CUDA keeps working with a newer driver. The converse is not true.
• Minor version compatibility (text verified in the release notes of the toolkit in force, Aug 2026): CUDA 13.x requires driver ≥ 580; 12.x, ≥ 525; 11.x, ≥ 450. That is, within a major branch you do not need to raise the driver on every toolkit update.
• The toolkit in force as of August 2026 is CUDA 13.3 Update 1; the current production driver branch published as open kernel modules is 610.43.03 (Jul 2026), with earlier branches 595 and 580 still active.
• Since CUDA 13.1 the Windows driver is no longer bundled with the toolkit. On Linux the separation was always conceptually like that: install the driver through the distribution's package manager and the toolkit separately (or, better, inside the container).
• PyTorch ships its own CUDA runtime in the wheel (torch-2.13.0+cu130, +cu132, +rocm7, +xpu according to the official index as of August 2026). From the system it only needs the driver. That is why the classic error "I have CUDA 12 installed and PyTorch wants 13" is almost always a badly chosen wheel, not a toolkit problem.

Golden rule: on the host, only the driver. Everything else, inside the container. It is what turns "update the framework" into an image change instead of an intervention on the host. On the host you install driver + nvidia-container-toolkit and nothing else.

Everything is pinned: driver version, CUDA base image by digest, framework version, NCCL version. The exact combination is recorded as a versioned artifact (iac-standards). A driver update is a platform change with a window and a rollback, not a Tuesday dnf update.

2.2 Installation on Linux (NVIDIA)

Number one operational risk: the kernel updates and the module does not build. A new kernel can break the DKMS build of the installed driver. Mandatory mitigation:

1. Pinned kernel on GPU nodes, with a deliberate and tested update, not an automatic one.
2. Boot gate: the node does not return to service until nvidia-smi responds correctly and a CUDA test load passes (§4).
3. Be able to go back: the previous kernel present in the boot manager.
4. In automated patching (linux-hardening-standards, unattended-upgrades/dnf-automatic), explicitly exclude kernel and driver on GPU nodes.

2.3 Containers with GPU

Criterion: on Kubernetes, GPU Operator + device plugin today; DRA as a pilot, with a planned migration, not an improvised one. Outside Kubernetes, CDI with nvidia-ctk cdi generate and --device nvidia.com/gpu=….

2.4 Sharing the GPU

GPUs are not oversubscribed the way CPUs are. "Sharing" means three very different things:

Rules:

• Real multi-tenancy ⇒ MIG, or a whole GPU. Time-slicing and MPS are not security boundaries and must not be presented as such.
• MPS and MIG do not combine (verify in the GPU Operator version in force).
• The static MIG profile wastes: if the smallest profile is 20 GB and your model takes 12, you throw away 8 GB per instance. The partition is planned with workload data, and is reviewed.
• Sharing does not create capacity. If the GPU is already saturated, splitting it only splits the queue.

2.5 AMD ROCm and alternatives — honestly

• Versions (Aug 2026): the official compatibility doc publishes ROCm 7.14.0; at the same time there are releases of the 7.2.x line (7.2.4, May 2026). There is more than one release stream (production vs. technology preview) and the numbers are not comparable with each other: verify which is the production stream before pinning. PyTorch publishes +rocm7 wheels.
• Where it works today: PyTorch + vLLM/SGLang on Instinct (MI300X/MI355X and family) is the viable combination. HBM capacity per card is a real structural advantage: large models with less tensor parallelism.
• Where it does not: there is no equivalent to TensorRT-LLM or FlashAttention 3; custom PTX kernels have to be ported; hipify translates CUDA code but does not translate calls to CUDA libraries (cuDNN, cuBLAS, TensorRT). Consumer cards are well behind the Instinct ones in kernel maturity.
• The real cost is not performance, it is ecosystem: when CUDA fails there is a decade of published answers; when ROCm fails, there are GitHub issues. Your team pays that cost in hours, and it has to be budgeted.
• Criterion: ROCm is a legitimate and evaluable option if (a) your stack is PyTorch + vLLM with no custom kernels, (b) you have people willing to debug, and (c) the cost or memory advantage is measurable in your workload. Outside that, CUDA remains the default, and the premium you pay buys ecosystem, not just FLOPs. ❌ Choosing ROCm out of ideology or list price without a measured pilot.
• Intel (+xpu): PyTorch publishes wheels; treat it as evaluable, not as a default. Maturity not verified in this pass (§8).

3. Structure and conventions

• GPU nodes are a separate node class: labelled (node-feature-discovery), with taints so that non-GPU workloads do not sneak onto them, pinned kernel and their own patching window. Mixing them with the general fleet guarantees that a routine patching run breaks them.
• Inventory: GPU model, VRAM, compute capability (SM), serial number, driver version, VBIOS version and ECC status — as code, alongside the rest of the fleet (onprem-standards).
• A single source of versions: a versioned file with driver, toolkit, framework and container toolkit for each environment. Without it, "it works on my node" is unarguable.
• Rebuildable from scratch: a GPU node is re-provisioned from code, including driver installation and module signing. ❌ Artisanal nodes.
• No compiling inside the production node: the image with the framework is built in CI (cicd-standards) and promoted by digest.

4. Quality and gates

Validation of a GPU node before putting it into service (and after every kernel or driver update) — in order of increasing cost:

1. nvidia-smi responds and lists all the expected GPUs, with the expected driver version. If one is missing, the node does not enter.
2. Module loaded and signed: the module loads with Secure Boot active; DKMS/akmod reports installed for the running kernel and for the previous one.
3. Health status: ECC enabled and no uncorrectable errors; no GPU in a degraded mode; PCIe link at the expected width and generation (an x16 negotiated at x4 explains mysterious performance problems); no recent XID in the journal.
4. Functional compute test: a container with the toolkit runs a real CUDA workload and checks the result. nvidia-smi inside the container is not enough: it tests visibility, not compute.
5. Stress and thermal test: sustained load for minutes, watching temperature, power and throttling reasons. Power and cooling failures only show up under sustained load, never in a short test.
6. Multi-GPU: GPU-to-GPU bandwidth test (NVLink/PCIe) and an NCCL collective test. A misdetected topology turns an expensive cluster into a slow one.
7. DCGM diagnostics (dcgmi diag) at the appropriate level, as the final step before accepting the node or returning it after an incident.

CI/CD gates that break the build or the deployment:

• Driver/toolkit/framework versions declared and pinned by digest; drift detected against the inventory breaks the pipeline.
• The image does not include the host driver (classic error); it does include the runtime and is tested against the minimum supported driver version.
• CVE scanning of the CUDA base image and of the container toolkit (vulnerability-management-standards).
• No manifest requests nvidia.com/gpu without a limit or without a toleration; no GPU pod runs as root or with --privileged (container-runtime-security-standards).

5. Security

5.1 Driver surface

• The GPU driver is a huge kernel module, partly proprietary, with a wide ioctl interface exposed to unprivileged processes through /dev/nvidia*. It is first-order local escalation surface, and the record confirms it.
• Verified in NVD (Aug 2026): CVE-2026-24187 (CVSS 8.8, use-after-free in the Linux driver, with S:C — scope change — and consequences that include code execution), CVE-2026-24199 (4.7, race condition in the kernel module, DoS). The associated bulletin (May 2026) patches branches R595, R580 and R535, and leaves R570 as EOL with no fix: if your nvidia-smi says 570.x, there is no patch, there is a branch migration. Verify the bulletin in force before acting (§8).
• Rule: NVIDIA's bulletins are subscribed to and triaged like the kernel's (vulnerability-management-standards). The GPU driver is not "a desktop component".
• Hardening: restrictive permissions on /dev/nvidia*, confinement of the process that can issue ioctls against the driver (selinux-standards), and unload the module on hosts that do not use the GPU.

5.2 Container Toolkit — a documented history of escapes

The NVIDIA Container Toolkit has had real and repeated container escapes. Verified in NVD:

Derived criterion:

• Prefer CDI over the legacy hooks mode: it is the declarative mechanism and several of the flaws are specific to the hooks.
• The toolkit is patched fast and pinned by version: it is the component with the worst record in the stack and it is on the critical path of every GPU container.
• Giving a container a GPU widens its surface, not just its compute. Containers that run untrusted workloads do not get a GPU without an additional layer (container-runtime-security-standards).
• ❌ --privileged "to make the GPU work": if it is needed, the toolkit's configuration is wrong.

5.3 Tenant isolation on a shared GPU

• Time-slicing and MPS do not isolate GPU memory. In MPS the processes share address space and the limits are applied at the API layer, not in hardware: a hostile process can read or corrupt another's GPU memory. FORBIDDEN to use them as a boundary between tenants.
• MIG is the only hardware-backed partition. Even so, it is not a hypervisor: for truly hostile tenants, a dedicated GPU or passthrough to a VM (libvirt-kvm-standards, proxmox-ve-standards).
• GPU memory is not guaranteed to clear itself between jobs. With sensitive data, an explicit reset/clean policy between jobs of different tenants, and verify it — do not assume it.
• Passthrough has its own risk and its own owner: a passed-through device does DMA towards the host, mediated by the IOMMU; with a badly configured IOMMU or with ACS overrides, the guest can write host memory. That is governed by libvirt-kvm-standards §3.8.

6. Performance, observability and physical operation

6.1 Monitoring: what each metric really measures

• nvidia-smi's utilization.gpu (and DCGM_FI_DEV_GPU_UTIL) is misleading. It measures the fraction of time in which at least one kernel was resident, not how much work is done. A trivial kernel on 1 of 132 SMs reads 100%. In LLM decode this is the normal state: 100% "utilisation" with the SMs almost empty. FORBIDDEN to use utilization.gpu as a capacity, sizing or FinOps metric.
• What is actually measured (DCGM, profiling metrics):
  • DCGM_FI_PROF_SM_ACTIVE — cycles with an active SM / total cycles.
  • DCGM_FI_PROF_SM_OCCUPANCY — fill at the warp level.
  • DCGM_FI_PROF_PIPE_TENSOR_ACTIVE versus DCGM_FI_PROF_DRAM_ACTIVE — the useful diagnosis: tensor high ⇒ compute bound; DRAM high and tensor low ⇒ memory bound (the normal situation in decode); both low with GPU_UTIL at 100% ⇒ small kernels, launch overhead or data starvation.
  • Memory used versus memory reserved (an inference engine reserves almost all the VRAM by design: the reservation metric says nothing about saturation).
  • Temperature, power, and clocks_throttle_reasons — thermal or power throttling is the most frequent explanation of "it's slower than yesterday".
  • ECC: correctable errors (trend) and uncorrectable ones (immediate alert); retired pages / row remapping.
  • Link: negotiated PCIe generation and width; NVLink errors.
• Collection caveats: profiling metrics require privileges (the nv-hostengine runs as superuser), not all of them can be collected at once on all hardware (DCGM multiplexes by sampling), and there are known issues with anomalous values on MIG devices. Do not build a critical alert on a metric you have not seen behave on your hardware.
• Tools (Aug 2026): DCGM 4.6.0 (Jul 2026) and dcgm-exporter 4.6.0-4.8.3 (Jul 2026) towards Prometheus. On Kubernetes it is deployed by the GPU Operator. nvidia-smi is for interactive diagnosis, not for continuous monitoring.
• The rest (retention, cardinality, alert design, dashboards as code) belongs to observability-standards.

6.2 Reliability: XID and company

• XIDs are the driver's error channel. They appear in the journal as NVRM: Xid (PCI:0000:xx:00): <n>, …. They are collected, correlated and alerted on; they are not log noise.
• XID 79 — "GPU has fallen off the bus": the GPU disconnects from the PCIe bus. Typical pattern: visible and fine at idle, drops when load is applied. Usual causes in order of probability: insufficient or unstable power, riser/adapter (OCuLink, expansion chassis, eGPU), temperature, connector seating, faulty hardware — and only after that, software. Diagnosis: collect the nvidia-bug-report.sh before unloading the module, review PCIe AER in the journal, and test the card in another slot and in another machine. If after a reboot the GPU does not appear, assume hardware.
• "GPU Reset Required" XID: the node must be drained and the GPU or the host restarted. A node in that state does not accept jobs: automate the cordon/drain instead of discovering it through failing jobs.
• Uncorrectable ECC ⇒ the node leaves service. You do not "keep an eye on it". Increasing retired pages ⇒ RMA candidate.
• Every long job must have checkpointing: in a GPU fleet, a card failing during a days-long training run is not a hypothesis, it is a statistical certainty.
• Mandatory runbook per failure family (XID, ECC, throttling, NCCL drop) with the action and the RMA criterion (incident-management-standards, onprem-standards).

6.3 Sizing and physical operation — the GPU changes the data centre design

• The GPU is not sized like a server. A traditional server rack moves in the 5-15 kW range; current GPU platforms are an order of magnitude above that: rack-scale Blackwell-generation systems are specified at around ~120-140 kW per rack (the GB200 NVL72 reference is documented up to 132 kW), and the following generations point higher. Verify the figure for the specific model in its data sheet (§8).
• Consequences that are not negotiable:
  • Direct-to-chip liquid cooling stops being optional above ~40-50 kW per rack: the air does not reach, and rear-door heat exchangers do not cover that density. That drags in a CDU, a facility water loop, flow rate and inlet temperature.
  • Dedicated three-phase power sized for full load, with matching PDUs and protections (onprem-standards).
  • Weight and structure: a GPU-scale rack weighs around a tonne. The floor and the access matter.
  • Design with headroom: rack density has been doubling every 18-24 months. A data centre specified exactly to the current generation is born obsolete.
  • In a homelab all of this translates into: idle consumption, noise, and whether the house's electrical circuit copes. homelab-standards rules.
• Power limit as a lever: lowering the power ceiling usually costs little performance and saves a lot of energy and heat. It is measured on your workload, not assumed.

6.4 Cost: buying, renting and the metric that matters

• A GPU's FinOps metric is utilisation, and not nvidia-smi's (§6.1): it is the fraction of time with useful work against the amortised time. A GPU at 15% is burnt money — and worse, it is invisibly burnt money, because the dashboard reads "100% GPU util".
• Buying is justified by: sustained high utilisation, a 2-3 year horizon, data that cannot leave, and the capacity to operate the hardware. Renting (cloud or bare metal by the hour) is justified by: bursty load, uncertainty about the model or the size, training peaks, and not wanting to buy an architecture that goes stale.
• Total cost of ownership: purchase + electricity (24×7, and with PUE, not just the card's TDP) + cooling + space + network + warranty/RMA + operation. The operation line is the one that gets forgotten and the one that grows most.
• Job queues to squeeze the hardware — without them utilisation collapses:
  • Slurm (gres.conf, --gres=gpu:N, partitions, QoS, preemption, fairshare) for batch workloads, HPC and training: it is the standard of the scientific world and it does well what Kubernetes does poorly (priority queues and backfill).
  • Kubernetes when the load is service-shaped (inference) or the rest of the platform is already there; for batch, with a queueing scheduler on top because the default scheduler has no queues and no gang scheduling — the specific choice (Kueue, Volcano) and its criteria belong to kubernetes-standards §6, which has them verified.
  • Rule: do not run both for the same class of workload. That a batch scheduler and a service scheduler coexist is normal in a real cluster — and the criterion for that split belongs to hpc-standards §2.2, which rules here —; what is not done is duplicating the route of the same workload just in case, because then nobody knows where to look when a GPU is missing.
• Chargeback/showback per team with the correct metric: without it, nobody releases a GPU reserved "just in case", and the idle reservation is the biggest cost sink in any GPU fleet.

7. Long-term sustainability and prohibitions

• Cadence: review NVIDIA's security bulletins and Container Toolkit CVEs monthly; review the driver branch and its EOL quarterly (an EOL branch receives no patch: you have to migrate); review the driver/CUDA/framework matrix before every framework upgrade.
• Driver branch policy: use supported branches (LTS/production) and plan the exit from a branch before its EOL, not on discovering that there is no patch for a critical CVE.
• Updating driver and kernel at the same time is asking for a double incident: they are done separately, with the §4 validation in between.
• Hardware life cycle: the previous GPU generation is not thrown away, it is demoted to development, to less demanding inference or to a lab. But the electricity consumption of old hardware can make it more expensive than replacing it: it is calculated, not assumed.

FORBIDDEN

• ❌ Using utilization.gpu / DCGM_FI_DEV_GPU_UTIL as a capacity, saturation or cost metric.
• ❌ Presenting time-slicing or MPS as isolation between tenants.
• ❌ Disabling ECC in production to gain VRAM.
• ❌ Installing the driver with the .run on a package-managed server.
• ❌ Automatic kernel updates on a GPU node, or patching with no subsequent validation gate (nvidia-smi + a real CUDA workload).
• ❌ Leaving nouveau unblacklisted and expecting the driver to load.
• ❌ Disabling Secure Boot "because the module does not load", with no written decision and without evaluating MOK signing.
• ❌ --privileged to access the GPU from a container.
• ❌ An unpatched Container Toolkit, or the legacy hooks mode when CDI is available.
• ❌ Giving a GPU to a container that runs untrusted workloads.
• ❌ Returning to service a node with recent XIDs or uncorrectable ECC errors without diagnosis.
• ❌ Long jobs without checkpointing in a GPU fleet.
• ❌ Installing the full CUDA toolkit on the host when the framework ships it in the container.
• ❌ Mixing driver versions within the same scheduling pool.
• ❌ Designing the room, power or cooling with CPU rack assumptions.
• ❌ Buying a GPU without a calculation of expected utilisation and without a queue to fill it.
• ❌ Adopting ROCm (or any alternative to CUDA) without a measured pilot on your workload.
• ❌ Covering passthrough/VFIO here: it belongs to libvirt-kvm-standards and proxmox-ve-standards.
• ❌ Asserting from memory a driver, CUDA or ROCm version, a TDP or a rack density figure.

8. Mandatory web verification

Before fixing any version, number or name:

1. Versions, via api.github.com/.../releases/latest or via the releases.atom feed (never via the HTML of the releases page: the summariser invents the year). Checked that way in August 2026: nvidia-container-toolkit v1.19.1 (May 2026) with v1.20.0-rc.1 (Jul 2026), GPU Operator 26.3.3 (Jun 2026), k8s-device-plugin v0.19.3 (Jun 2026), DCGM 4.6.0 (Jul 2026), dcgm-exporter 4.6.0-4.8.3 (Jul 2026), open-gpu-kernel-modules 610.43.03 (Jul 2026), ROCm 7.14.0 (Jul 2026) and 7.2.4 (May 2026).
2. CUDA: version in force and the minor version compatibility table in the official release notes. Verified verbatim (Aug 2026): toolkit 13.3 Update 1; 13.x ⇒ driver ≥ 580, 12.x ⇒ ≥ 525, 11.x ⇒ ≥ 450.
3. Driver ↔ CUDA ↔ framework matrix: PyTorch's wheel index (download.pytorch.org/whl/torch/) says exactly which builds exist. Verified (Aug 2026): torch 2.13.0 with +cu126, +cu129, +cu130, +cu132, +rocm7, +xpu.
4. Module flavour and supported architectures: NVIDIA's installation guide and the README of the specific branch. Careful, there is a discrepancy between official sources (§ gap below).
5. NVIDIA security bulletins (driver, vGPU, Container Toolkit, TensorRT-LLM) and NVD for the detail and the CVSS. Verified in NVD (Aug 2026): CVE-2026-24187 (8.8), CVE-2026-24199 (4.7), CVE-2026-24260 (8.5), CVE-2025-23266 (9.0), CVE-2025-23267 (8.5), CVE-2025-23359 (8.3), CVE-2024-0132 (9.0).
6. EOL of the driver branch you use: the May 2026 bulletin left R570 with no fix. Check your branch's status before you need it.
7. DRA status for GPUs in your Kubernetes version and NVIDIA driver (GA in the core since 1.34; the GPU driver, technology preview as of August 2026).
8. Exact DCGM field names in the installed version and in the dcgm-exporter doc.
9. XID codes: NVIDIA's canonical list (docs.nvidia.com/deploy/xid-errors).
10. TDP, rack density and cooling requirements of the specific model, in the manufacturer's data sheet. Not from a blog.

Declared gaps (not verified in this pass, do not fill from memory)

• Module flavour by architecture: official sources in conflict. The data centre installation guide says the open modules are "only for Turing and newer architectures" and that the proprietary ones are needed for "older GPUs from the Maxwell, Pascal, or Volta architectures"; the README of recent branches indicates that the proprietary flavour covers Turing–Hopper and that Blackwell and later are open only, and that branch 580 was the last with Maxwell/Pascal/Volta support. Consult the README of the exact branch you are going to install; the general guide may be out of date.
• ROCm: which is the production release stream. Numbering schemes coexist (7.2.x and 7.1x.x) that appear to correspond to different streams (production vs. technology preview/TheRock). Not verified which one should be pinned.
• ROCm performance figures against CUDA: the published ranges go from "37-66% of an H100" to "90-95% parity", depending on the source, the model and the tuning effort. None independently verified. Do not cite a number without your own pilot.
• Maturity of the Intel XPU backend: not evaluated.
• Status of Kueue / Volcano — gap CLOSED: verified in kubernetes-standards §6 (version, API, governance and licence). Re-verify there, not here.
• MIG detail: maximum number of instances, profiles and architectures supported today cited from secondary sources. Verify in NVIDIA's MIG doc before planning a partition.
• Rack density: the figures in §6.3 are from reference rack-scale platforms, not from your server. The datum that rules is the data sheet of the model you buy.
• InfiniBand / RoCE with no owner — gap CLOSED: it belongs to high-speed-interconnect-standards, which covers IB, RoCE v2, iWARP, subnet manager, GPUDirect RDMA and NVMe-oF. Do not improvise the compute network here: load it.
• Module signing with MOK and its automation (kmodsign, per-distro procedure): criterion fixed, specific commands not verified in this pass.

If the web contradicts this document, the web wins — flag the discrepancy.