DOCA GPUNetIO ib_write_bw
Where to start: This is a tool skill for the GPUNetIO-
flavored ib_write_bw benchmark shipped under
doca/tools/gpunetio_ib_write_bw/ (a client + server pair,
built from source against the installed DOCA via meson).
It measures sustained RDMA WRITE bandwidth when the WRs are
posted from a CUDA kernel through the doca-gpunetio
device-side surface, with the GPU on the data path. Open
TASKS.md and start at
## configure for the GPU-NIC
pairing precondition and the build pattern; jump to
## run for the smoke-before-bulk flow.
Open CAPABILITIES.md when the question
is what this tool actually measures, how the result
decomposes (GPU occupancy vs NIC issue rate vs link
saturation), or how the result reads against the GPI
sister tool and the upstream CPU-initiated perftest
ib_write_bw. If DOCA is not installed yet, route to
doca-setup first; if the
user is still deciding between the GPI and GPUNetIO
programming surfaces, the picture in
../../libs/doca-gpunetio/CAPABILITIES.md#capabilities-and-modes
and
../../libs/doca-gpi/CAPABILITIES.md#capabilities-and-modes
is the first stop.
Example questions this skill answers well
The CLASSES of doca-gpunetio-ib-write-bw questions this
skill is built to answer, each with one worked example. The
class is the load-bearing piece; the worked example is one
instance.
- "What sustained RDMA-WRITE bandwidth can the GPUNetIO
path deliver on this GPU-NIC pair?" — worked example:
"measure sustained WRITE BW between two hosts with an
H100 + ConnectX-7 on each side". Answered by the
GPU-NIC pairing precondition in
CAPABILITIES.md ## Capabilities and modes
- the bring-up flow in
TASKS.md ## configure +
TASKS.md ## run. The same shape
answers "measure GPUNetIO-driven WRITE BW between a
host GPU and a BlueField DPU".
- "Where is the bottleneck — GPU compute occupancy, NIC
issue rate, or link saturation?" — worked example:
"I see 120 Gbit/s on a 200 Gbit/s link; is the NIC
saturated, am I CPU-bound on the client, or is the CUDA
kernel not driving enough WRs in flight?". Answered by
the throughput-decomposition rules in
CAPABILITIES.md ## Observability
- the eval-loop overlay in
TASKS.md ## test.
- "How does the result differ from the classic CPU-
initiated
perftest ib_write_bw?" — worked example:
"my team has a CPU-initiated WRITE BW number on this
same NIC; should I expect the GPUNetIO number to match
or be different?". Answered by the "GPU-initiated
path adds (or removes) overhead vs the CPU-initiated
path" rule in
CAPABILITIES.md ## Capabilities and modes.
- "Is the doca-gpunetio path the right surface for my
sustained-throughput workload class?" — worked example:
"my application streams sensor data from GPU memory at
line rate to a remote consumer". Answered by the
"when GPUNetIO is the right surface vs GPI vs CPU-
initiated" rule in
CAPABILITIES.md ## Capabilities and modes
- the use-side decision in
TASKS.md ## use.
- "My BW number swings between runs. What do I check
before quoting it?" — worked example: "three runs at
the same flags gave 145, 187, and 160 Gbit/s; is the
benchmark noisy or is my platform inconsistent?".
Answered by the measurement-soundness rules in
CAPABILITIES.md ## Error taxonomy
layer 5 + the steady-state guidance in
TASKS.md ## test.
- "What version of DOCA + CUDA Toolkit do I need for this
binary to build and run?" — worked example: "my
install has DOCA at one semver and CUDA at another; will
the ToT-shipped
gpunetio_ib_write_bw even link?".
Answered by the version overlay in
CAPABILITIES.md ## Version compatibility
which cross-links the canonical detection chain in
doca-version.
Audience
This skill serves external developers and performance
engineers who need a reproducible measurement of sustained
RDMA WRITE bandwidth when the WRs are posted from a CUDA
kernel through doca-gpunetio, on the user's actual install
and GPU-NIC pair. Concretely:
- A developer comparing the GPUNetIO path against the GPI
path or the host-initiated
perftest-style path before
committing an application design to one of them.
- A platform operator validating a tuning change (NUMA
pinning, GPU PCIe placement, IB device choice, GID
index, NIC firmware burn) by re-running this benchmark
against the new state.
- An SRE / performance engineer producing a "this is the
GPUNetIO-driven WRITE BW on this GPU-NIC pair today"
artifact downstream consumers can cite.
- An AI agent answering "is the doca-gpunetio path a win
for my sustained-throughput workload class" honestly —
with a measured number, the build + invocation that
produced it, and the GPU + NIC + DOCA version that
scopes it — rather than guessing from datasheet
headlines.
It is not for users debugging the doca-gpunetio
library itself (route to
../../libs/doca-gpunetio/SKILL.md),
and not a substitute for the perftest upstream
ib_write_bw (which measures CPU-initiated WRITE BW).
Language scope
The doca-gpunetio-ib-write-bw tool is shipped as C plus
a CUDA .cu translation unit under
doca/tools/gpunetio_ib_write_bw/, split into a client/
subtree and a server/ subtree. The verified surface (per
client/{main.c,common.h,common.c,kernel.cu,perftest.c} and
server/{main.c,common.h,common.c,perftest.c}): host-side
build via meson against the installed DOCA pkg-config
modules (doca-gpunetio, doca-rdma, doca-common); the
device-side build via nvcc against the DOCA GPU NetIO
device-side header set; the OOB descriptor exchange via a
TCP socket between client and server. There is no Python /
Rust / Go binding — the tool is a pair of CLI binaries.
The skill's job is to keep the operator-side workflow
language-neutral; the device-side CUDA surface is not
wrappable in another language.
When to load this skill
Load this skill when the user is — or the agent needs to —
build and run the gpunetio_ib_write_bw client + server on
real hosts with DOCA installed plus a CUDA Toolkit matched
to the DOCA install, and a GPU + IB device pair on the
host's PCIe topology. Concretely:
- Measuring sustained kernel-initiated RDMA WRITE
bandwidth between two hosts (or a host and a BlueField
DPU) with the GPUNetIO surface.
- Deciding whether the GPUNetIO path is the right runtime
surface for a class of workload vs the GPI programming
surface (the
doca-gpi
library — doca/tools/ ships no GPI benchmark binary) or
the classic CPU-initiated perftest path.
- Capturing a documented baseline (build + invocation +
DOCA version + GPU + NIC + as-deployed environment +
numbers) for later regression hunts.
- Diagnosing a build / link / run failure that surfaces
the GPUNetIO + RDMA bring-up sequence under this tool's
shipped scaffolding.
Do not load this skill for general DOCA orientation,
library API work, or installation. For those, use
doca-public-knowledge-map,
../../libs/doca-gpunetio/SKILL.md,
or doca-setup. Do not load
it for application-level end-to-end throughput either —
this benchmark measures the WR-submission path through
GPUNetIO, not the user's full pipeline.
What this skill provides
This is a thin loader. Substantive material lives in
two companion files:
CAPABILITIES.md — what the tool measures (the
sustained-WRITE-BW primitive driven by a client-side
CUDA kernel through doca-gpunetio), the
runtime-surface selection rule (GPUNetIO vs GPI vs
CPU-initiated), the GPU-NIC pairing precondition, the
throughput-decomposition guide (GPU compute occupancy
vs NIC issue rate vs link saturation), the version
overlay (DOCA .pc PLUS CUDA Toolkit), the layered
error taxonomy (config-syntax / build-time / GPU-NIC-
pairing / GPUNetIO-lifecycle / RDMA-connection /
measurement-soundness / version / cross-cutting), the
observability surface (stdout report, DOCA log levels,
OOB-socket exchange), and the safety overlay (the
"GPU-side handle is a credential" rule from
doca-gpunetio; the cross-cutting hardware-safety
meta-policy).
TASKS.md — step-by-step workflows for the in-scope
task verbs: install (preconditions — DOCA install,
CUDA Toolkit, GPU + NIC pair, OOB connectivity),
configure (build-tree under
doca/tools/gpunetio_ib_write_bw/ and the meson
build wrapping the shipped DOCA), build (the
meson setup + meson compile pattern from the
public DOCA build documentation), modify (do not
patch the shipped tool source; modify the invocation
and the surrounding environment instead), run (smoke-
before-bulk; client + server bring-up order; reading
the per-iteration report), test (the eval loop —
steady-state, NUMA placement, NIC saturation cross-
check), debug (walk the error taxonomy layer by
layer), use (how a BW result feeds a class-of-
workload decision), plus a Deferred task verbs
block routing out-of-scope questions.
The skill assumes a host where DOCA is already installed,
a CUDA Toolkit matched to the install is present, and the
operator has whatever privileges the public install profile
expects for binding a doca_dev, a doca_gpu, and an OOB
TCP socket.
What this skill deliberately does not ship
This skill is agent guidance, not a samples or scripts
bundle. To keep the boundary clean, it deliberately does
not contain — and pull requests should not add:
- Specific flag strings or expected throughput numbers
beyond what the tool's shipped
--help and main.c ARGP
registration establish. The flag surface is small
(device name, GPU PCIe address, GID index, server IP on
the client side); the agent re-reads the binary's
--help on the installed version before quoting flag
strings. Throughput numbers are device-, firmware-,
version-, and topology-specific.
- Pre-written DOCA GPUNetIO or CUDA kernel source code
that would compete with the shipped tool tree. The
shipped
client/{main.c,kernel.cu,perftest.c,common.{c,h}}
and server/{main.c,perftest.c,common.{c,h}} files are
the verified worked example; the agent's job is to
route the user there and prescribe minimum-diff
modification per the universal modify-a-sample workflow
in
doca-programming-guide.
- Wrappers, parsers, or scripts in any language that
consume the tool's stdout. The output format is small
and documented in
CAPABILITIES.md ## Observability;
if the user wants to script against it, the right
answer is "read the live source, write the parser
against your installed binary".
- A
samples/, bindings/, or reference/ subtree.
This is a thin loader for a shipped tool tree;
substantive material lives in the source tree and in
the GPUNetIO library docs.
Loading order
- Read this
SKILL.md first to confirm the user's
question is in scope (the user actually wants to
measure sustained kernel-initiated WRITE BW through
GPUNetIO, not learn GPUNetIO as a library or do a
CPU-initiated measurement).
- For what the tool measures, the surface-selection
rule against the GPI sister tool and the CPU-initiated
perftest, the throughput-decomposition guide, the
version overlay, the error taxonomy, the observability
surface, and the safety overlay, see
CAPABILITIES.md.
- For step-by-step workflows —
install, configure,
build, modify, run, test, debug, use — see
TASKS.md.
Related skills
../../libs/doca-gpunetio/SKILL.md —
the library this tool wraps. The per-GPU doca_gpu
context, the GPU-visible doca_gpu_eth_* and RDMA-side
handles, the CUDA-side persistent-kernel pattern, the
dual capability-discovery rule (DOCA cap-query AND
cudaGetDeviceProperties), and the env preconditions
(nvidia_peermem loaded, CUDA buffers registered with
DOCA) live there.
../../libs/doca-rdma/SKILL.md —
the underlying RDMA library. The RDMA queue this tool
binds is created and connected via doca-rdma; the
queue lifecycle, transport type (RC vs UC vs UD),
permission matrix, and connection method are owned
there.
../../libs/doca-verbs/SKILL.md —
the raw-verbs escape hatch beneath doca-rdma /
doca-gpunetio. This tool stays on the higher-level
surfaces; doca-verbs is the right place only if the
user needs a specific WR flag / QP attribute the
GPUNetIO + RDMA surfaces do not expose.
../doca-gpunetio-ib-write-lat/SKILL.md —
the latency analog of this tool. Same physical
operation; same runtime framework; different metric
class (BW vs latency). The two together carry the
full GPUNetIO-side throughput / latency picture.
doca-gpi — the GPI
programming surface (CUDA-kernel-initiated RDMA), the
alternative runtime framework for the same physical
operation. doca/tools/ ships no GPI ib_write_lat /
ib_write_bw benchmark binary, so the GPI comparison is
against the library surface, not a sibling tool. The
selection rule in
CAPABILITIES.md ## Capabilities and modes
is the decision aid.
doca-version — the
canonical version-detection chain, four-way match rule,
NGC container semantics, and headers-win-over-docs
rule. The ## Version compatibility section in this
skill is a thin overlay; the body lives there.
doca-setup — env
preparation, install verification, GPU + CUDA Toolkit
pairing, nvidia_peermem load, hugepages, NUMA, and
the I have no install yet path with the public NGC
DOCA container.
doca-public-knowledge-map —
routing to the public DOCA documentation set (DOCA GPU
NetIO, DOCA RDMA pages on docs.nvidia.com) and the
docs.nvidia.com/cuda/ pointer for the CUDA Toolkit.
doca-debug — the
cross-cutting debug ladder. The tool surfaces its own
error taxonomy; when the cause is below DOCA, the
taxonomy hands off here.
doca-hardware-safety —
the bundle-wide hardware-safety meta-policy. The
## Safety policy overlay cross-links it.
1---2license: Apache-2.03name: doca-gpunetio-ib-write-bw4description: >5 Use this skill when the user is building, running, or interpreting6 the doca/tools/gpunetio_ib_write_bw client+server benchmark — a CUDA7 kernel on the client posts RDMA WRITE work requests through the8 doca-gpunetio device-side surface to measure sustained GPU-driven9 WRITE bandwidth on a GPU+IB-device pair. Trigger even when the user10 does not explicitly mention "doca-gpunetio-ib-write-bw" or11 "GPUNetIO" — typical implicit phrasings include "measure WRITE BW12 when the GPU posts the WRs", "BW swings between runs on the same13 flags", "is the NIC saturated or am I CPU-bound on the CUDA14 kernel", "meson compile fails for the GPUNetIO bw tool",15 "nvidia_peermem isn't picking up my GPU buffer", or "GPU-initiated16 WRITE throughput vs CPU-initiated perftest". Refuse and route17 elsewhere for general doca-gpunetio library work, DOCA install, the18 GPU-initiated WRITE latency analog, the CPU-initiated upstream19 perftest, or application-level end-to-end throughput — those belong20 to other skills.21metadata:22 source: https://skillmd.com/skills/nvidia/doca-gpunetio-ib-write-bw23 kind: tool24compatibility: >25 Requires DOCA SDK on Linux with a BlueField DPU or ConnectX NIC,26 NVIDIA GPU, CUDA toolkit and nvcc, loaded `nvidia_peermem`, and an27 InfiniBand RNIC paired with the GPU. Uses `pkg-config` for28 doca-gpunetio, doca-rdma, and doca-common, plus the installed29 gpunetio_ib_write_bw sources. Run only on a trusted, non-shared IB30 fabric during the benchmark window.31---3233# DOCA GPUNetIO ib_write_bw3435**Where to start:** This is a tool skill for the GPUNetIO-36flavored `ib_write_bw` benchmark shipped under37`doca/tools/gpunetio_ib_write_bw/` (a client + server pair,38built from source against the installed DOCA via `meson`).39It measures sustained RDMA WRITE bandwidth when the WRs are40posted **from a CUDA kernel through the doca-gpunetio41device-side surface**, with the GPU on the data path. Open42[`TASKS.md`](TASKS.md) and start at43[`## configure`](TASKS.md#configure) for the GPU-NIC44pairing precondition and the build pattern; jump to45[`## run`](TASKS.md#run) for the smoke-before-bulk flow.46Open [`CAPABILITIES.md`](CAPABILITIES.md) when the question47is *what this tool actually measures*, *how the result48decomposes (GPU occupancy vs NIC issue rate vs link49saturation)*, or *how the result reads against the GPI50sister tool and the upstream CPU-initiated `perftest`51`ib_write_bw`*. If DOCA is not installed yet, route to52[`doca-setup`](../../doca-setup/SKILL.md) first; if the53user is still deciding between the GPI and GPUNetIO54programming surfaces, the picture in55[`../../libs/doca-gpunetio/CAPABILITIES.md#capabilities-and-modes`](../../libs/doca-gpunetio/CAPABILITIES.md#capabilities-and-modes)56and57[`../../libs/doca-gpi/CAPABILITIES.md#capabilities-and-modes`](../../libs/doca-gpi/CAPABILITIES.md#capabilities-and-modes)58is the first stop.5960## Example questions this skill answers well6162The CLASSES of `doca-gpunetio-ib-write-bw` questions this63skill is built to answer, each with one worked example. The64class is the load-bearing piece; the worked example is one65instance.6667- **"What sustained RDMA-WRITE bandwidth can the GPUNetIO68 path deliver on this GPU-NIC pair?"** — worked example:69 *"measure sustained WRITE BW between two hosts with an70 H100 + ConnectX-7 on each side"*. Answered by the71 GPU-NIC pairing precondition in72 [`CAPABILITIES.md ## Capabilities and modes`](CAPABILITIES.md#capabilities-and-modes)73 + the bring-up flow in74 [`TASKS.md ## configure`](TASKS.md#configure) +75 [`TASKS.md ## run`](TASKS.md#run). The same shape76 answers *"measure GPUNetIO-driven WRITE BW between a77 host GPU and a BlueField DPU"*.78- **"Where is the bottleneck — GPU compute occupancy, NIC79 issue rate, or link saturation?"** — worked example:80 *"I see 120 Gbit/s on a 200 Gbit/s link; is the NIC81 saturated, am I CPU-bound on the client, or is the CUDA82 kernel not driving enough WRs in flight?"*. Answered by83 the throughput-decomposition rules in84 [`CAPABILITIES.md ## Observability`](CAPABILITIES.md#observability)85 + the eval-loop overlay in86 [`TASKS.md ## test`](TASKS.md#test).87- **"How does the result differ from the classic CPU-88 initiated `perftest` `ib_write_bw`?"** — worked example:89 *"my team has a CPU-initiated WRITE BW number on this90 same NIC; should I expect the GPUNetIO number to match91 or be different?"*. Answered by the *"GPU-initiated92 path adds (or removes) overhead vs the CPU-initiated93 path"* rule in94 [`CAPABILITIES.md ## Capabilities and modes`](CAPABILITIES.md#capabilities-and-modes).95- **"Is the doca-gpunetio path the right surface for my96 sustained-throughput workload class?"** — worked example:97 *"my application streams sensor data from GPU memory at98 line rate to a remote consumer"*. Answered by the99 *"when GPUNetIO is the right surface vs GPI vs CPU-100 initiated"* rule in101 [`CAPABILITIES.md ## Capabilities and modes`](CAPABILITIES.md#capabilities-and-modes)102 + the use-side decision in [`TASKS.md ## use`](TASKS.md#use).103- **"My BW number swings between runs. What do I check104 before quoting it?"** — worked example: *"three runs at105 the same flags gave 145, 187, and 160 Gbit/s; is the106 benchmark noisy or is my platform inconsistent?"*.107 Answered by the measurement-soundness rules in108 [`CAPABILITIES.md ## Error taxonomy`](CAPABILITIES.md#error-taxonomy)109 layer 5 + the steady-state guidance in110 [`TASKS.md ## test`](TASKS.md#test).111- **"What version of DOCA + CUDA Toolkit do I need for this112 binary to build and run?"** — worked example: *"my113 install has DOCA at one semver and CUDA at another; will114 the ToT-shipped `gpunetio_ib_write_bw` even link?"*.115 Answered by the version overlay in116 [`CAPABILITIES.md ## Version compatibility`](CAPABILITIES.md#version-compatibility)117 which cross-links the canonical detection chain in118 [`doca-version`](../../doca-version/SKILL.md).119120## Audience121122This skill serves **external developers and performance123engineers who need a reproducible measurement of sustained124RDMA WRITE bandwidth when the WRs are posted from a CUDA125kernel through doca-gpunetio**, on the user's actual install126and GPU-NIC pair. Concretely:127128- A developer comparing the GPUNetIO path against the GPI129 path or the host-initiated `perftest`-style path before130 committing an application design to one of them.131- A platform operator validating a tuning change (NUMA132 pinning, GPU PCIe placement, IB device choice, GID133 index, NIC firmware burn) by re-running this benchmark134 against the new state.135- An SRE / performance engineer producing a *"this is the136 GPUNetIO-driven WRITE BW on this GPU-NIC pair today"*137 artifact downstream consumers can cite.138- An AI agent answering *"is the doca-gpunetio path a win139 for my sustained-throughput workload class"* honestly —140 with a measured number, the build + invocation that141 produced it, and the GPU + NIC + DOCA version that142 scopes it — rather than guessing from datasheet143 headlines.144145It is **not** for users debugging the `doca-gpunetio`146library itself (route to147[`../../libs/doca-gpunetio/SKILL.md`](../../libs/doca-gpunetio/SKILL.md)),148and **not** a substitute for the `perftest` upstream149`ib_write_bw` (which measures CPU-initiated WRITE BW).150151## Language scope152153The `doca-gpunetio-ib-write-bw` tool is shipped as **C plus154a CUDA `.cu` translation unit** under155`doca/tools/gpunetio_ib_write_bw/`, split into a `client/`156subtree and a `server/` subtree. The verified surface (per157`client/{main.c,common.h,common.c,kernel.cu,perftest.c}` and158`server/{main.c,common.h,common.c,perftest.c}`): host-side159build via `meson` against the installed DOCA `pkg-config`160modules (`doca-gpunetio`, `doca-rdma`, `doca-common`); the161device-side build via `nvcc` against the DOCA GPU NetIO162device-side header set; the OOB descriptor exchange via a163TCP socket between client and server. There is no Python /164Rust / Go binding — the tool is a pair of CLI binaries.165The skill's job is to keep the operator-side workflow166language-neutral; the device-side CUDA surface is not167wrappable in another language.168169## When to load this skill170171Load this skill when the user is — or the agent needs to —172build and run the `gpunetio_ib_write_bw` client + server on173real hosts with DOCA installed plus a CUDA Toolkit matched174to the DOCA install, and a GPU + IB device pair on the175host's PCIe topology. Concretely:176177- Measuring sustained kernel-initiated RDMA WRITE178 bandwidth between two hosts (or a host and a BlueField179 DPU) with the GPUNetIO surface.180- Deciding whether the GPUNetIO path is the right runtime181 surface for a class of workload vs the GPI programming182 surface (the [`doca-gpi`](../../libs/doca-gpi/SKILL.md)183 library — `doca/tools/` ships no GPI benchmark binary) or184 the classic CPU-initiated `perftest` path.185- Capturing a documented baseline (build + invocation +186 DOCA version + GPU + NIC + as-deployed environment +187 numbers) for later regression hunts.188- Diagnosing a build / link / run failure that surfaces189 the GPUNetIO + RDMA bring-up sequence under this tool's190 shipped scaffolding.191192Do **not** load this skill for general DOCA orientation,193library API work, or installation. For those, use194[`doca-public-knowledge-map`](../../doca-public-knowledge-map/SKILL.md),195[`../../libs/doca-gpunetio/SKILL.md`](../../libs/doca-gpunetio/SKILL.md),196or [`doca-setup`](../../doca-setup/SKILL.md). Do not load197it for *application-level* end-to-end throughput either —198this benchmark measures the WR-submission path through199GPUNetIO, not the user's full pipeline.200201## What this skill provides202203This is a **thin loader**. Substantive material lives in204two companion files:205206- `CAPABILITIES.md` — what the tool measures (the207 sustained-WRITE-BW primitive driven by a client-side208 CUDA kernel through doca-gpunetio), the209 runtime-surface selection rule (GPUNetIO vs GPI vs210 CPU-initiated), the GPU-NIC pairing precondition, the211 throughput-decomposition guide (GPU compute occupancy212 vs NIC issue rate vs link saturation), the version213 overlay (DOCA `.pc` PLUS CUDA Toolkit), the layered214 error taxonomy (config-syntax / build-time / GPU-NIC-215 pairing / GPUNetIO-lifecycle / RDMA-connection /216 measurement-soundness / version / cross-cutting), the217 observability surface (stdout report, DOCA log levels,218 OOB-socket exchange), and the safety overlay (the219 *"GPU-side handle is a credential"* rule from220 doca-gpunetio; the cross-cutting hardware-safety221 meta-policy).222- `TASKS.md` — step-by-step workflows for the in-scope223 task verbs: `install` (preconditions — DOCA install,224 CUDA Toolkit, GPU + NIC pair, OOB connectivity),225 `configure` (build-tree under226 `doca/tools/gpunetio_ib_write_bw/` and the `meson`227 build wrapping the shipped DOCA), `build` (the228 `meson setup` + `meson compile` pattern from the229 public DOCA build documentation), `modify` (do not230 patch the shipped tool source; modify the invocation231 and the surrounding environment instead), `run` (smoke-232 before-bulk; client + server bring-up order; reading233 the per-iteration report), `test` (the eval loop —234 steady-state, NUMA placement, NIC saturation cross-235 check), `debug` (walk the error taxonomy layer by236 layer), `use` (how a BW result feeds a class-of-237 workload decision), plus a `Deferred task verbs`238 block routing out-of-scope questions.239240The skill assumes a host where DOCA is already installed,241a CUDA Toolkit matched to the install is present, and the242operator has whatever privileges the public install profile243expects for binding a `doca_dev`, a `doca_gpu`, and an OOB244TCP socket.245246## What this skill deliberately does not ship247248This skill is **agent guidance**, not a samples or scripts249bundle. To keep the boundary clean, it deliberately does250not contain — and pull requests should not add:251252- **Specific flag strings or expected throughput numbers**253 beyond what the tool's shipped `--help` and `main.c` ARGP254 registration establish. The flag surface is small255 (device name, GPU PCIe address, GID index, server IP on256 the client side); the agent re-reads the binary's257 `--help` on the installed version before quoting flag258 strings. Throughput numbers are device-, firmware-,259 version-, and topology-specific.260- **Pre-written DOCA GPUNetIO or CUDA kernel source code**261 that would compete with the shipped tool tree. The262 shipped `client/{main.c,kernel.cu,perftest.c,common.{c,h}}`263 and `server/{main.c,perftest.c,common.{c,h}}` files are264 the verified worked example; the agent's job is to265 route the user there and prescribe minimum-diff266 modification per the universal modify-a-sample workflow267 in268 [`doca-programming-guide`](../../doca-programming-guide/SKILL.md).269- **Wrappers, parsers, or scripts** in any language that270 consume the tool's stdout. The output format is small271 and documented in272 [`CAPABILITIES.md ## Observability`](CAPABILITIES.md#observability);273 if the user wants to script against it, the right274 answer is *"read the live source, write the parser275 against your installed binary"*.276- **A `samples/`, `bindings/`, or `reference/` subtree.**277 This is a thin loader for a shipped tool tree;278 substantive material lives in the source tree and in279 the GPUNetIO library docs.280281## Loading order2822831. Read this `SKILL.md` first to confirm the user's284 question is in scope (the user actually wants to285 measure sustained kernel-initiated WRITE BW through286 GPUNetIO, not learn GPUNetIO as a library or do a287 CPU-initiated measurement).2882. **For what the tool measures, the surface-selection289 rule against the GPI sister tool and the CPU-initiated290 `perftest`, the throughput-decomposition guide, the291 version overlay, the error taxonomy, the observability292 surface, and the safety overlay, see293 [CAPABILITIES.md](CAPABILITIES.md).**2943. **For step-by-step workflows — `install`, `configure`,295 `build`, `modify`, `run`, `test`, `debug`, `use` — see296 [TASKS.md](TASKS.md).**297298## Related skills299300- [`../../libs/doca-gpunetio/SKILL.md`](../../libs/doca-gpunetio/SKILL.md) —301 the library this tool wraps. The per-GPU `doca_gpu`302 context, the GPU-visible `doca_gpu_eth_*` and RDMA-side303 handles, the CUDA-side persistent-kernel pattern, the304 dual capability-discovery rule (DOCA cap-query AND305 `cudaGetDeviceProperties`), and the env preconditions306 (`nvidia_peermem` loaded, CUDA buffers registered with307 DOCA) live there.308- [`../../libs/doca-rdma/SKILL.md`](../../libs/doca-rdma/SKILL.md) —309 the underlying RDMA library. The RDMA queue this tool310 binds is created and connected via `doca-rdma`; the311 queue lifecycle, transport type (RC vs UC vs UD),312 permission matrix, and connection method are owned313 there.314- [`../../libs/doca-verbs/SKILL.md`](../../libs/doca-verbs/SKILL.md) —315 the raw-verbs escape hatch beneath `doca-rdma` /316 `doca-gpunetio`. This tool stays on the higher-level317 surfaces; `doca-verbs` is the right place only if the318 user needs a specific WR flag / QP attribute the319 GPUNetIO + RDMA surfaces do not expose.320- [`../doca-gpunetio-ib-write-lat/SKILL.md`](../doca-gpunetio-ib-write-lat/SKILL.md) —321 the latency analog of this tool. Same physical322 operation; same runtime framework; different metric323 class (BW vs latency). The two together carry the324 full GPUNetIO-side throughput / latency picture.325- [`doca-gpi`](../../libs/doca-gpi/SKILL.md) — the GPI326 programming surface (CUDA-kernel-initiated RDMA), the327 alternative runtime framework for the same physical328 operation. `doca/tools/` ships no GPI `ib_write_lat` /329 `ib_write_bw` benchmark binary, so the GPI comparison is330 against the library surface, not a sibling tool. The331 selection rule in332 [`CAPABILITIES.md ## Capabilities and modes`](CAPABILITIES.md#capabilities-and-modes)333 is the decision aid.334- [`doca-version`](../../doca-version/SKILL.md) — the335 canonical version-detection chain, four-way match rule,336 NGC container semantics, and headers-win-over-docs337 rule. The `## Version compatibility` section in this338 skill is a thin overlay; the body lives there.339- [`doca-setup`](../../doca-setup/SKILL.md) — env340 preparation, install verification, GPU + CUDA Toolkit341 pairing, `nvidia_peermem` load, hugepages, NUMA, and342 the *I have no install yet* path with the public NGC343 DOCA container.344- [`doca-public-knowledge-map`](../../doca-public-knowledge-map/SKILL.md) —345 routing to the public DOCA documentation set (DOCA GPU346 NetIO, DOCA RDMA pages on `docs.nvidia.com`) and the347 `docs.nvidia.com/cuda/` pointer for the CUDA Toolkit.348- [`doca-debug`](../../doca-debug/SKILL.md) — the349 cross-cutting debug ladder. The tool surfaces its own350 error taxonomy; when the cause is below DOCA, the351 taxonomy hands off here.352- [`doca-hardware-safety`](../../doca-hardware-safety/SKILL.md) —353 the bundle-wide hardware-safety meta-policy. The354 `## Safety policy` overlay cross-links it.