DOCA debug
Where to start: This skill is the canonical layered-ladder
reference both doca-setup ## debug
and doca-programming-guide ## debug
escalate to. If the symptom does not fit cleanly in env-class or
program-class, start at TASKS.md ## debug — the
full layered ladder lives there.
Example questions this skill answers well
The CLASSES of debug questions this skill is built to answer, each
with one worked example.
- "My DOCA build / link / runtime / program failed — which layer is
it?" — worked example: "
ld says undefined reference to doca_flow_init — which layer?" Answered by the canonical layered
ladder in TASKS.md ## debug (layer 4 = Link).
- "How do I turn up verbosity for any DOCA library or tool?" —
worked example: "My DOCA Flow program is silent — where do I get
more log output?" Answered by the trace-flavor / log-level surface
in
CAPABILITIES.md ## Observability
- the run-with-verbosity workflow in
TASKS.md ## run.
- "How do I capture state for a forum question or bug report?" —
worked example: "I'd like to file a forum question with reproducible
context — what should I include?" Answered by the
capture-a-reproducible-state workflow in
TASKS.md ## test.
- "What does this DOCA tool's output mean / which tool answers
this?" — worked example: "
doca_caps returned nothing for RDMA
— does that mean unsupported?" Answered by the
tool-vs-capability decision tree in
CAPABILITIES.md ## Capabilities and modes
- "I'm debugging inside the NGC container — what's
observable?" — worked example: "
hugepages is empty inside the
container; is that a real problem or a container thing?"
Answered by the container-specific debug constraints in
CAPABILITIES.md ## Safety policy
TASKS.md ## debug layer 5 (Runtime).
- "Where do I ask for help with this?" — worked example: "Where
is the customer-facing DOCA forum and what should the post
contain?" Answered by the Developer-Forum routing rule in
TASKS.md ## debug plus
doca-public-knowledge-map.
If the symptom is purely env-class, route to
doca-setup ## debug; if purely
program-class, route to
doca-programming-guide ## debug.
This skill is the cross-cutting layer both call into.
When to load this skill
Load this skill when the user is debugging anything DOCA-related — a build that won't compile, a link step that can't resolve a doca_* symbol, a runtime call that returns DOCA_ERROR_*, a packet that does not appear on the wire, a service that won't start, or a tool that returns no useful output. Concretely:
- The user reports a symptom and needs to find the layer that caused it (install / version / build / link / runtime / program).
- The user asks "how do I get more logs?" or "how do I turn up the verbosity?" for any DOCA library or tool.
- The user wants to capture state for a forum question or an internal bug report (the bundle does not own the internal-bug-report channel; it routes to the public DOCA Developer Forum).
- The user is reading a stack trace, a
valgrind output, or a core dump from a DOCA program and wants to know where to look first.
- The user is debugging inside the NGC DOCA container and needs to know what is and is not observable from inside it.
Do not load this skill for:
- "What is
DOCA_ERROR_BAD_STATE?", "what error codes does DOCA return?" — that is the cross-library error taxonomy, owned by doca-programming-guide CAPABILITIES.md ## Error taxonomy. This skill consumes that taxonomy; it does not redefine it.
- "My
pkg-config cannot find doca-flow", "hugepages are not mounted", "my representor isn't visible" — those are env-class symptoms, owned by doca-setup ## debug. This skill is the canonical pointer that env-class debug ladder redirects to once the symptom escalates beyond install / version / build prerequisites.
- Library-specific debugging (Flow pipe trace, RDMA queue-pair state, Comch channel statistics) — those live in the matching library skill (e.g.
doca-flow ## debug). This skill provides the cross-cutting debug ladder; library skills layer their library-specific debug surface on top of it.
What this skill provides
This is a thin loader. The body keeps only the orientation needed to pick the right next file. The substantive debug material lives in two companion files:
- CAPABILITIES.md — what kinds of debug surface DOCA exposes: the layered debug model (install / version / build / link / runtime / program), the read-only-first stance, version-availability of debug tools (e.g.
doca_caps since DOCA 2.6.0), the cross-library error taxonomy (cross-link only — owned by doca-programming-guide), the observability primitives DOCA emits (stderr logs, --sdk-log-level, the doca-<lib>-trace build flavor, library counters), and the safety constraints on debug actions (read-only first, don't mutate install tree mid-investigation).
- TASKS.md — the actual debug workflows:
## configure (set up env for high-verbosity debug), ## test (capture a reproducible state), ## debug (the canonical layered ladder, the universal entry point that every library ## debug redirects to), and the Where to ask for help routing (NVIDIA DOCA Developer Forum). Three other anchors (build, modify, run) exist for lint compliance and route to doca-programming-guide, which owns those verbs after the env / program split.
Loading order
- Read this
SKILL.md first to confirm the user's symptom is cross-cutting debug (not env-class only, not program-class only, not library-internal only).
- For the layered debug model, the read-only stance, the version-availability of debug tools, and the observability surface DOCA emits, see CAPABILITIES.md.
- For the canonical layered debug ladder and the capture-and-report workflow, see TASKS.md. The
build, modify, and run anchors in TASKS.md are stubs that route to doca-programming-guide; their substance lives there.
- If the user's symptom turns out to be env-class (install / build prerequisites), hand off to
doca-setup ## debug. If program-class, hand off to doca-programming-guide ## debug. If library-internal, hand off to the matching library skill's ## debug (e.g. doca-flow ## debug).
The two companion files cross-link to each other and to doca-public-knowledge-map whenever the right answer is "look it up in the public docs or the installed package layout" rather than "debug-specific guidance".
Related skills
doca-public-knowledge-map — public DOCA documentation routing and the on-disk layout of an installed DOCA package. This skill defers all "where is X documented", "where on disk is Y", and "how do I check the installed version" questions to the knowledge-map.
doca-setup — env-class debug (install / build prerequisites): pkg-config failures, missing hugepages, representors not visible, header-vs-runtime version mismatches. doca-setup ## debug is the env-class layered ladder; this skill is the cross-cutting debug ladder both env and program ladders escalate to.
doca-programming-guide — program-class debug (lifecycle order, DOCA_ERROR_* interpretation, doca_error_get_descr() use, the validate-before-commit rule). doca-programming-guide ## debug is the program-class layered ladder; this skill picks up where it leaves off when the symptom involves cross-library tooling (gdb, valgrind, container introspection, core dumps).
- Library skills (e.g.
doca-flow, doca-dms, doca-caps) — library-specific debug overlays. Each library's ## debug builds on the cross-cutting ladder defined here, then adds its own counters, traces, and inspector tools.
1---2name: doca-debug3description: Use this skill when the user is debugging any DOCA symptom — a build that won't compile, a link step that can't resolve a doca_* symbol, a runtime call returning DOCA_ERROR_*, a silent service or tool, or a stack trace / valgrind / core dump — and needs the layered ladder (install → version → build → link → runtime → program → driver), verbosity controls (--sdk-log-level, DOCA_LOG_LEVEL, the doca-{lib}-trace flavor), container-debug constraints, or how to capture state for a Developer Forum post. Trigger even when the user does not say "DOCA debug" — implicit phrasings include "undefined reference to doca_*", "how do I get more logs", "packets aren't reaching the wire", "doca_caps returned nothing", or "hugepages empty in the container". Refuse and route elsewhere for library-specific debug (Flow pipe trace, RDMA QP, Comch stats), env-class pkg-config or hugepages symptoms, the DOCA_ERROR_* taxonomy and lifecycle interpretation, and performance or incident-response work — those belong to other skills.4license: Apache-2.05---67# DOCA debug89**Where to start:** This skill is the canonical layered-ladder10reference both [`doca-setup ## debug`](../doca-setup/TASKS.md#debug)11and [`doca-programming-guide ## debug`](../doca-programming-guide/TASKS.md#debug)12escalate to. If the symptom does not fit cleanly in env-class or13program-class, start at [`TASKS.md ## debug`](TASKS.md#debug) — the14full layered ladder lives there.1516## Example questions this skill answers well1718The CLASSES of debug questions this skill is built to answer, each19with one worked example.2021- **"My DOCA build / link / runtime / program failed — which layer is22 it?"** — worked example: *"`ld` says `undefined reference to23 doca_flow_init` — which layer?"* Answered by the canonical layered24 ladder in [`TASKS.md ## debug`](TASKS.md#debug) (layer 4 = Link).25- **"How do I turn up verbosity for any DOCA library or tool?"** —26 worked example: *"My DOCA Flow program is silent — where do I get27 more log output?"* Answered by the trace-flavor / log-level surface28 in [`CAPABILITIES.md ## Observability`](CAPABILITIES.md#observability)29 + the run-with-verbosity workflow in30 [`TASKS.md ## run`](TASKS.md#run).31- **"How do I capture state for a forum question or bug report?"** —32 worked example: *"I'd like to file a forum question with reproducible33 context — what should I include?"* Answered by the34 capture-a-reproducible-state workflow in35 [`TASKS.md ## test`](TASKS.md#test).36- **"What does this DOCA tool's output mean / which tool answers37 this?"** — worked example: *"`doca_caps` returned nothing for RDMA38 — does that mean unsupported?"* Answered by the39 tool-vs-capability decision tree in40 [`CAPABILITIES.md ## Capabilities and modes`](CAPABILITIES.md#capabilities-and-modes)41 + cross-link to [doca-caps](../tools/doca-caps/SKILL.md).42- **"I'm debugging inside the NGC container — what's43 observable?"** — worked example: *"`hugepages` is empty inside the44 container; is that a real problem or a container thing?"*45 Answered by the container-specific debug constraints in46 [`CAPABILITIES.md ## Safety policy`](CAPABILITIES.md#safety-policy)47 + [`TASKS.md ## debug`](TASKS.md#debug) layer 5 (Runtime).48- **"Where do I ask for help with this?"** — worked example: *"Where49 is the customer-facing DOCA forum and what should the post50 contain?"* Answered by the Developer-Forum routing rule in51 [`TASKS.md ## debug`](TASKS.md#debug) plus52 [doca-public-knowledge-map](../doca-public-knowledge-map/SKILL.md).5354If the symptom is purely env-class, route to55[`doca-setup ## debug`](../doca-setup/TASKS.md#debug); if purely56program-class, route to57[`doca-programming-guide ## debug`](../doca-programming-guide/TASKS.md#debug).58This skill is the cross-cutting layer both call into.5960## When to load this skill6162Load this skill when the user is debugging anything DOCA-related — a build that won't compile, a link step that can't resolve a `doca_*` symbol, a runtime call that returns `DOCA_ERROR_*`, a packet that does not appear on the wire, a service that won't start, or a tool that returns no useful output. Concretely:6364- The user reports a symptom and needs to find the layer that caused it (install / version / build / link / runtime / program).65- The user asks "how do I get more logs?" or "how do I turn up the verbosity?" for any DOCA library or tool.66- The user wants to capture state for a forum question or an internal bug report (the bundle does not own the internal-bug-report channel; it routes to the public DOCA Developer Forum).67- The user is reading a stack trace, a `valgrind` output, or a core dump from a DOCA program and wants to know where to look first.68- The user is debugging *inside* the NGC DOCA container and needs to know what is and is not observable from inside it.6970Do **not** load this skill for:7172- *"What is `DOCA_ERROR_BAD_STATE`?", "what error codes does DOCA return?"* — that is the cross-library *error taxonomy*, owned by [`doca-programming-guide CAPABILITIES.md ## Error taxonomy`](../doca-programming-guide/CAPABILITIES.md#error-taxonomy). This skill consumes that taxonomy; it does not redefine it.73- *"My `pkg-config` cannot find `doca-flow`", "hugepages are not mounted", "my representor isn't visible"* — those are env-class symptoms, owned by [`doca-setup ## debug`](../doca-setup/TASKS.md#debug). This skill is the canonical pointer that env-class debug ladder redirects to once the symptom escalates beyond install / version / build prerequisites.74- *Library-specific debugging* (Flow pipe trace, RDMA queue-pair state, Comch channel statistics) — those live in the matching library skill (e.g. [`doca-flow ## debug`](../libs/doca-flow/TASKS.md#debug)). This skill provides the cross-cutting debug ladder; library skills layer their library-specific debug surface on top of it.7576## What this skill provides7778This is a **thin loader**. The body keeps only the orientation needed to pick the right next file. The substantive debug material lives in two companion files:7980- [CAPABILITIES.md](CAPABILITIES.md) — what *kinds of debug surface* DOCA exposes: the layered debug model (install / version / build / link / runtime / program), the read-only-first stance, version-availability of debug tools (e.g. `doca_caps` since DOCA 2.6.0), the cross-library error taxonomy (cross-link only — owned by `doca-programming-guide`), the observability primitives DOCA emits (`stderr` logs, `--sdk-log-level`, the `doca-<lib>-trace` build flavor, library counters), and the safety constraints on debug actions (read-only first, don't mutate install tree mid-investigation).81- [TASKS.md](TASKS.md) — the actual debug workflows: `## configure` (set up env for high-verbosity debug), `## test` (capture a reproducible state), `## debug` (the canonical layered ladder, the universal entry point that every library `## debug` redirects to), and the *Where to ask for help* routing (NVIDIA DOCA Developer Forum). Three other anchors (`build`, `modify`, `run`) exist for lint compliance and route to [`doca-programming-guide`](../doca-programming-guide/SKILL.md), which owns those verbs after the env / program split.8283## Loading order84851. Read this `SKILL.md` first to confirm the user's symptom is *cross-cutting* debug (not env-class only, not program-class only, not library-internal only).862. **For the layered debug model, the read-only stance, the version-availability of debug tools, and the observability surface DOCA emits, see [CAPABILITIES.md](CAPABILITIES.md).**873. **For the canonical layered debug ladder and the *capture-and-report* workflow, see [TASKS.md](TASKS.md).** The `build`, `modify`, and `run` anchors in `TASKS.md` are stubs that route to [`doca-programming-guide`](../doca-programming-guide/SKILL.md); their substance lives there.884. If the user's symptom turns out to be env-class (install / build prerequisites), hand off to [`doca-setup ## debug`](../doca-setup/TASKS.md#debug). If program-class, hand off to [`doca-programming-guide ## debug`](../doca-programming-guide/TASKS.md#debug). If library-internal, hand off to the matching library skill's `## debug` (e.g. [`doca-flow ## debug`](../libs/doca-flow/TASKS.md#debug)).8990The two companion files cross-link to each other and to [`doca-public-knowledge-map`](../doca-public-knowledge-map/SKILL.md) whenever the right answer is *"look it up in the public docs or the installed package layout"* rather than *"debug-specific guidance"*.9192## Related skills9394- [`doca-public-knowledge-map`](../doca-public-knowledge-map/SKILL.md) — public DOCA documentation routing and the on-disk layout of an installed DOCA package. This skill defers all *"where is X documented"*, *"where on disk is Y"*, and *"how do I check the installed version"* questions to the knowledge-map.95- [`doca-setup`](../doca-setup/SKILL.md) — env-class debug (install / build prerequisites): `pkg-config` failures, missing hugepages, representors not visible, header-vs-runtime version mismatches. `doca-setup ## debug` is the env-class layered ladder; this skill is the cross-cutting debug ladder both env and program ladders escalate to.96- [`doca-programming-guide`](../doca-programming-guide/SKILL.md) — program-class debug (lifecycle order, `DOCA_ERROR_*` interpretation, `doca_error_get_descr()` use, the validate-before-commit rule). `doca-programming-guide ## debug` is the program-class layered ladder; this skill picks up where it leaves off when the symptom involves cross-library tooling (`gdb`, `valgrind`, container introspection, core dumps).97- Library skills (e.g. [`doca-flow`](../libs/doca-flow/SKILL.md), [`doca-dms`](../services/doca-dms/SKILL.md), [`doca-caps`](../tools/doca-caps/SKILL.md)) — library-specific debug overlays. Each library's `## debug` builds on the cross-cutting ladder defined here, then adds its own counters, traces, and inspector tools.