Crash Debug
Use for RTX Remix Toolkit crash investigations. Keep runs separate, preserve provenance, classify the likely failing
layer only when the evidence supports it, and avoid unsupported root-cause claims.
Workflow
Identify the run: time, run id, trigger, outcome, project/capture names, app path, launch cwd, process id, Toolkit
commit, HdRemix/DXVK version, driver version, GPU, and any dirty repo state.
Draft a dev repro guide early. Include exact launch method, project/capture inputs, UI steps, expected timing,
observed crash signature, and artifact/run ids. Keep updating it as evidence changes.
Check docs_dev/internal/qa-steps.md before using open_project.bat. Use it for quick Runtime bug repros or rapid
testing of specific Runtime issues; do not use it for smoke checks, feature validation, or user-flow bugs because it
bypasses normal UI validation.
Gather artifacts: Kit logs, DXVK/HdRemix logs, minidumps, Aftermath dumps, crash metadata, configs, screenshots,
repro notes, and relevant Windows events. Preserve build/version details for Toolkit, HdRemix/DXVK, GPU driver, and
OS when available.
Bundle artifacts with:
tools\packman\python.bat tools\crash_debug\bundle_crash_artifacts.py --help
Organize bundles under one case root outside the repo by default. Use a user temp directory such as
%TEMP%\lightspeed-kit-crash-artifacts\<case-name> on Windows or $TMPDIR/lightspeed-kit-crash-artifacts/<case-name>
on Linux/macOS. Do not place crash bundles under the repo root unless the user explicitly asks for a repo-local
bundle.
%TEMP%\lightspeed-kit-crash-artifacts\<case-name>\
CASE_MANIFEST.md
RUN-001_<short-title>/
RUN_MANIFEST.md
Preserve minidump UUIDs in the run manifest.
Use a new --run-id for each bundle invocation; the bundler refuses existing run ids to avoid rewriting evidence.
When a case has multiple runs, create or update CASE_MANIFEST.md using example-crash-manifest.md as the table
format. Keep root-cause ids stable across runs (RC-01, RC-02, ...), keep unknowns explicit, and do not add
placeholder rows for artifacts that do not exist.
Do not mix logs or dumps from multiple app copies. If multiple Toolkit processes are open, prove which process wrote
each artifact before drawing conclusions.
Classification
Use module names, exception codes, stack frames, log tails, crash metadata, and repro timing together.
- Kit/USD/Sdf/Tf: Python traceback, Kit crash metadata, USD/Sdf/Tf modules, extension lifecycle, stage/load state.
- DXVK/HdRemix/d3d9:
d3d9.dll, HdRemix.dll, remixapi_*, QueryFeatureVersion, remix-dxvk.log.
- GPU/driver:
VK_ERROR_DEVICE_LOST, NRC EndFrame, WER LiveKernelEvent 141, nvlddmkm, .nv-gpudmp/.gpudmp.
- Extension code: Python traceback, extension logs, recent extension events, tests that isolate the extension.
lastCommands is supporting evidence only. A visible UI or command event can be last while the failing layer is
elsewhere.
Aftermath
Track Kit-owned and DXVK/HdRemix Aftermath separately:
- Kit AF:
NVIDIA Aftermath Status: ...
- DXVK/HdRemix AF:
dxvk.enableAftermath = True and Aftermath enabled in remix-dxvk.log
Kit crash metadata such as aftermath_status describes Kit-owned AF only. It does not prove DXVK/HdRemix AF state.
Aftermath is instrumentation, not a workaround. Missing .nv-gpudmp files do not prove AF setup failed unless the run
actually reached a GPU/device-lost path that AF can report. Do not add AF toggles to crash isolation by default or infer
AF changes behavior without paired runs that isolate AF state as the changed variable.
Reporting
Lead with:
Classification: likely <layer>
Confidence: high/medium/low
Why: <evidence>
Not proven: <remaining ambiguity>
Repro guide: <path or current steps>
Next test: <one-variable test>
Workaround: <only if supported>
1---2name: crash-debug3description: Use when investigating RTX Remix Toolkit crashes, Kit minidumps, DXVK/HdRemix logs, Aftermath dumps, GPU device-lost failures, d3d9 crashes, capture-switch repros, or crash handoff bundles.4---56# Crash Debug78Use for RTX Remix Toolkit crash investigations. Keep runs separate, preserve provenance, classify the likely failing9layer only when the evidence supports it, and avoid unsupported root-cause claims.1011## Workflow12131. Identify the run: time, run id, trigger, outcome, project/capture names, app path, launch cwd, process id, Toolkit14 commit, HdRemix/DXVK version, driver version, GPU, and any dirty repo state.152. Draft a dev repro guide early. Include exact launch method, project/capture inputs, UI steps, expected timing,16 observed crash signature, and artifact/run ids. Keep updating it as evidence changes.173. Check `docs_dev/internal/qa-steps.md` before using `open_project.bat`. Use it for quick Runtime bug repros or rapid18 testing of specific Runtime issues; do not use it for smoke checks, feature validation, or user-flow bugs because it19 bypasses normal UI validation.204. Gather artifacts: Kit logs, DXVK/HdRemix logs, minidumps, Aftermath dumps, crash metadata, configs, screenshots,21 repro notes, and relevant Windows events. Preserve build/version details for Toolkit, HdRemix/DXVK, GPU driver, and22 OS when available.235. Bundle artifacts with:2425 ```batch26 tools\packman\python.bat tools\crash_debug\bundle_crash_artifacts.py --help27 ```28296. Organize bundles under one case root outside the repo by default. Use a user temp directory such as30 `%TEMP%\lightspeed-kit-crash-artifacts\<case-name>` on Windows or `$TMPDIR/lightspeed-kit-crash-artifacts/<case-name>`31 on Linux/macOS. Do not place crash bundles under the repo root unless the user explicitly asks for a repo-local32 bundle.3334 ```text35 %TEMP%\lightspeed-kit-crash-artifacts\<case-name>\36 CASE_MANIFEST.md37 RUN-001_<short-title>/38 RUN_MANIFEST.md39 ```40417. Preserve minidump UUIDs in the run manifest.428. Use a new `--run-id` for each bundle invocation; the bundler refuses existing run ids to avoid rewriting evidence.439. When a case has multiple runs, create or update `CASE_MANIFEST.md` using `example-crash-manifest.md` as the table44 format. Keep root-cause ids stable across runs (`RC-01`, `RC-02`, ...), keep unknowns explicit, and do not add45 placeholder rows for artifacts that do not exist.4610. Do not mix logs or dumps from multiple app copies. If multiple Toolkit processes are open, prove which process wrote47 each artifact before drawing conclusions.4849## Classification5051Use module names, exception codes, stack frames, log tails, crash metadata, and repro timing together.5253- Kit/USD/Sdf/Tf: Python traceback, Kit crash metadata, USD/Sdf/Tf modules, extension lifecycle, stage/load state.54- DXVK/HdRemix/d3d9: `d3d9.dll`, `HdRemix.dll`, `remixapi_*`, `QueryFeatureVersion`, `remix-dxvk.log`.55- GPU/driver: `VK_ERROR_DEVICE_LOST`, `NRC EndFrame`, WER `LiveKernelEvent 141`, `nvlddmkm`, `.nv-gpudmp`/`.gpudmp`.56- Extension code: Python traceback, extension logs, recent extension events, tests that isolate the extension.5758`lastCommands` is supporting evidence only. A visible UI or command event can be last while the failing layer is59elsewhere.6061## Aftermath6263Track Kit-owned and DXVK/HdRemix Aftermath separately:6465- Kit AF: `NVIDIA Aftermath Status: ...`66- DXVK/HdRemix AF: `dxvk.enableAftermath = True` and `Aftermath enabled` in `remix-dxvk.log`6768Kit crash metadata such as `aftermath_status` describes Kit-owned AF only. It does not prove DXVK/HdRemix AF state.6970Aftermath is instrumentation, not a workaround. Missing `.nv-gpudmp` files do not prove AF setup failed unless the run71actually reached a GPU/device-lost path that AF can report. Do not add AF toggles to crash isolation by default or infer72AF changes behavior without paired runs that isolate AF state as the changed variable.7374## Reporting7576Lead with:7778```text79Classification: likely <layer>80Confidence: high/medium/low81Why: <evidence>82Not proven: <remaining ambiguity>83Repro guide: <path or current steps>84Next test: <one-variable test>85Workaround: <only if supported>86```
nvidiagameworks