# Nanousd Renderer Scene Extraction

> Load USD scenes through NanoUSD and convert composed USD prims into renderer-ready geometry, material, light, camera, transform, instancing, texture, and diagnostic representations. Use when implementing, debugging, or validating NanoUSD-based scene ingestion, renderer scene structs, material/light extraction, texture path resolution, or loader parity against USD/OpenUSD semantics.

- Skill: `nvidia-omniverse/nanousd-renderer-scene-extraction` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add nvidia-omniverse/nanousd-renderer-scene-extraction`
- Raw SKILL.md: https://api.skillmd.com/api/skills/nvidia-omniverse/nanousd-renderer-scene-extraction/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: NVIDIA-Omniverse (https://skillmd.com/u/nvidia-omniverse)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/nvidia-omniverse/nanousd-renderer-scene-extraction

---


# NanoUSD Renderer Scene Extraction

## Purpose

Turn composed USD data into a renderer-owned scene representation without losing authored intent. Preserve paths, transforms, visibility, material bindings, texture identity, light units, instancing, and unsupported-feature diagnostics until the renderer deliberately consumes or rejects them.

## Workflow

1. Define the extraction contract first.
   - Name the output structs and ownership rules: geometry buffers, material table, texture table, light table, camera table, prototype/instance table, path maps, warnings, and unsupported features.
   - Record coordinate convention, matrix layout, time code, up-axis policy, units policy, material model coverage, and renderer capability gaps.
   - Keep CPU scene ingestion separate from GPU upload. Upload only after counts, bounds, indices, material bindings, and light records pass invariants.

2. Open and validate the stage.
   - Use `nanousd_open`, `nanousd_isvalid`, `nanousd_error`, and `nanousd_close`; for borrowed handles, make ownership explicit and never close caller-owned stages.
   - Capture root layer, layer stack, `upAxis`, `metersPerUnit`, `timeCodesPerSecond`, frame range, and current evaluation time.
   - Treat load/composition warnings as scene diagnostics, not stdout noise. Include counts for prims, meshes, materials, textures, lights, curves, instances, skipped prims, and unsupported features.

3. Traverse composed prims deliberately.
   - Iterate `nanousd_nprims` / `nanousd_prim` for the composed stage, then free each prim handle with `nanousd_freeprim`.
   - Skip inactive, undefined, abstract, invisible, guide, and proxy data unless the renderer explicitly requests those purposes.
   - Visibility is inherited: walk ancestors before deciding a child mesh is visible. Keep USD paths in every emitted record for debugging, selection, AOVs, and stable IDs.

4. Build transforms once and document conventions.
   - Use `nanousd_get_local_transform(prim, time, local, &resetXformStack)` and compose parent transforms while honoring `resetXformStack`.
   - The Vulkan loader currently uses row-major, row-vector transforms: `P_world = P_local * leaf * parent * ... * root`.
   - Apply `upAxis` conversion and `metersPerUnit` exactly once at the scene boundary, and apply it consistently to mesh bounds, curve widths, light positions/extents, cameras, and instance transforms.
   - Re-evaluate transforms when the renderer time changes; do not reuse default-time matrices for animated xformOps.

5. Extract mesh geometry with interpolation intact.
   - Read `points`, `faceVertexCounts`, and `faceVertexIndices`; prefer NanoUSD zero-copy array readers for speed, but copy data the renderer must retain because later NanoUSD calls can invalidate cached array data.
   - Validate index ranges before triangulation. Triangulate polygons deterministically and preserve the source path and face/primitive mapping if picking or material subsets need it.
   - Handle normals, `primvars:normals`, `primvars:st`, alternate UV names, `displayColor`, `displayOpacity`, orientation, double-sidedness, subdivision scheme, and interpolation metadata.
   - Distinguish authored opinions from USD schema fallbacks. For visibility-critical fields such as Mesh `doubleSided`, record whether the value was authored or defaulted, and validate primary, shadow, and instance traversal semantics with a reduced fixture before tuning light intensity or denoising.
   - Face-varying UVs or normals require vertex expansion. Generate smooth or flat normals only when absent, and label generated data as generated.
   - Curves, points, volumes, subdivision surfaces, and unsupported prim types must produce either a converted renderer record or a structured unsupported-feature record; do not silently drop them.

6. Preserve instancing and prototypes.
   - Use NanoUSD instance/prototype APIs (`nanousd_isinstance`, `nanousd_prototype`, `nanousd_ninstances`, `nanousd_instance`) and `PointInstancer` attributes (`prototypes`, `protoIndices`, `positions`, `orientations`, `scales`) to build prototype and instance tables.
   - Store prototype geometry once, then store per-instance transforms, visibility, material overrides, and stable IDs.
   - USD `quatf`/`quath` arrays read into NanoUSD float arrays as `(x, y, z, w)` in the current Vulkan loader pattern; verify ordering before composing instance transforms.
   - Keep prototype-only prims hidden from normal draw lists while still available for instance expansion.

7. Resolve materials and textures from USD graphs.
   - Resolve `material:binding` relationships on the prim and ancestors. If material subsets are supported, preserve face ranges and subset binding strength.
   - Follow shader connections with `nanousd_nconnections` / `nanousd_connection`; strip output suffixes to locate upstream shader or texture prims with `nanousd_primpath`.
   - Support renderer-known models explicitly, such as UsdPreviewSurface, OmniPBR, OmniGlass, OmniSurface, and sidecar MaterialX. Unknown nodegraphs should become unsupported material diagnostics, not valid white defaults.
   - Map base color, opacity, opacity threshold, metallic, roughness, IOR, clearcoat, normal, normal scale, emissive, occlusion/ORM, specular workflow, subsurface, transmission, texture transforms, and fallback values.
   - Deduplicate textures by resolved identity plus color-space/sampler semantics. Mark base color and emissive textures as sRGB; mark normal, roughness, metallic, AO, opacity, and ORM data textures as linear data.
   - Resolve asset paths relative to the authored layer or stage context, not process `cwd`. For sidecar scans, bound the search to the scene directory and make expensive scans opt-in or cached.

8. Extract lights and cameras as first-class scene data.
   - Handle DomeLight, RectLight, DistantLight, SphereLight, and any renderer-supported Disk/Cylinder variants. Count known unsupported light types.
   - Preserve `inputs:color`, `inputs:intensity`, `inputs:exposure`, `inputs:normalize`, dimensions, radius, angle, shaping controls, visibility, texture/HDR paths, and dome rotation.
   - Derive light world frames from the same transform pipeline as geometry. For RectLight, store center, normal, and half-extent axes; for DistantLight, store normalized direction and angle; for SphereLight, store radius and world position.
   - Extract authored cameras when needed: transform, projection, focal length, apertures, clipping range, focus distance, and active-camera selection. If auto-framing is a fallback, report it as a fallback.

9. Emit a renderer-ready scene with diagnostics.
   - Return geometry buffers, material/light/texture arrays, curves, camera records, bounds, path-to-index maps, prototype/instance maps, and unsupported-feature diagnostics.
   - Include stable path hashes or IDs for meshes, lights, materials, emitters, and instances so renderer AOVs and logs can tie pixels back to USD.
   - Track load timing and memory by stage open, material load, mesh extraction, instancing, curves, lights, texture decode, GPU upload, and acceleration-structure build.

10. Validate with focused fixtures before full assets.
   - Add reduced scenes for resetXformStack, up-axis plus metersPerUnit, face-varying UV seams, inherited visibility, material binding, texture connections, point instancing, DomeLight HDR, direct lights, emissive materials, and unsupported prims.
   - Compare against OpenUSD/usdview/Hydra or a known reference renderer when parity matters. Match camera, exposure, tone mapping, color space, light/environment, sample count, and seed.
   - Use AOVs to isolate ingest bugs: geometry IDs, normals, UVs, base color, material ID, roughness, metallic, opacity, emissive, direct light, shadow visibility, indirect, and depth.

## Reference Contract and Cross-Cutting Conventions

These are the parts an agent cannot infer and that have silently broken existing
backends. Treat the renderers' `scene.h` as the reference *shape*; do not freeze a
field list here — match the canonical struct. The durable content is the invariant
and the failure it prevents.

**Scene struct shape (point at `scene.h`, preserve these groups).** Every renderer
needs, and must keep until the backend consumes them: geometry (flat positions plus
optional normals/displayColor/UVs and triangulated indices — copy what the renderer
retains, since later NanoUSD calls can invalidate zero-copy arrays); identity (the
USD path on *every* record, so AOVs, picking, and logs tie pixels back to USD);
transform (one world matrix per record in a single documented convention);
bounds (world-space *and* object-space — object-space lets you bound expanded
instances without re-transforming every shared prototype vertex); material binding
(an index into a material table, not inlined material data); instancing (a prototype
index plus a "prototype-only" flag, so PointInstancer prototype subtrees are
materialized via instances and never drawn standalone); and scene-level data (up-axis,
dome/IBL metadata, light table, structured unsupported-feature list). Do not flatten
instancing or drop paths at ingest "to simplify" — that information cannot be
recovered later.

**Vertex layout (example — verify against the header).** An interleaved per-vertex
layout such as 48-byte / 12-float `[pos.xyz, normal.xyz, pad.xyz, uv.xy, matID]` is
one workable choice. The durable rule: the host writer and every shader/pipeline that
reads the buffer derive stride and offsets from ONE definition, and any later pass
that touches uv/matID must use the same stride. A mismatch yields garbage attributes
with no error.

**Transform convention and the silent translation-drop bug (invariant + signature).**
Pick one convention and apply one named conversion at the GPU boundary. The NanoUSD
loaders use row-vector USD (translation in the matrix's last row). A flat element copy
across a row-vector/column-vector boundary silently drops translation: the object
renders at the origin and per-frame transform updates look like no-ops. Verify with a
translate-and-re-render test — a static identity-vs-identity test passes tautologically
and hides exactly this. This bug shipped in more than one backend.

**Host/shader struct sync.** Any struct shared between host and shader (per-segment
geometry, push constants, material records) has two definitions in two languages and
they must change together. Keep them adjacent or generated, and note the duplication at
each site; a silent mismatch mis-decodes every record. See `renderer-backend-port` for
the cross-language alignment/padding trap.

## NanoUSD Entry Points

- Stage: `nanousd_open`, `nanousd_close`, `nanousd_isvalid`, `nanousd_error`, `nanousd_stage_get_root_layer_path`, `nanousd_stage_n_layers`, `nanousd_stage_layer_path`.
- Metadata: `nanousd_metadatas`, `nanousd_metadatad`, `nanousd_timecodes_per_second`, `nanousd_frames_per_second`, `nanousd_start_time`, `nanousd_end_time`.
- Traversal: `nanousd_nprims`, `nanousd_prim`, `nanousd_primpath`, `nanousd_defaultprim`, `nanousd_parent`, `nanousd_nchildren`, `nanousd_child`.
- Prim state: `nanousd_path`, `nanousd_name`, `nanousd_typename`, `nanousd_kind`, `nanousd_isactive`, `nanousd_isdefined`, `nanousd_isabstract`, `nanousd_isa`, `nanousd_hasapi`.
- Attributes: `nanousd_hasattrib`, `nanousd_attribtype`, scalar/vector readers, `nanousd_attribarraylen`, `nanousd_attribarrayf`, `nanousd_attribarrayi`, `nanousd_arraydataf`, `nanousd_arraydatai`, `nanousd_attrib_token`, `nanousd_attribasset`, `nanousd_attrib_interpolation`, `nanousd_attrib_authored`.
- Time samples: `nanousd_hassamples`, `nanousd_nsamplekeys`, `nanousd_samplekey`, `nanousd_sample*`.
- Relationships/connections: `nanousd_hasrel`, `nanousd_nreltargets`, `nanousd_reltarget`, `nanousd_hasconnections`, `nanousd_nconnections`, `nanousd_connection`.
- Transforms: `nanousd_get_local_transform`, `nanousd_mul_m4d`.
- Instancing: `nanousd_isinstance`, `nanousd_isprototype`, `nanousd_isinprototype`, `nanousd_prototype`, `nanousd_ninstances`, `nanousd_instance`.

## Failure Signatures

- Exploded or mirrored scene: transform order, reset stack, quaternion order, up-axis, or units applied twice.
- White or default-gray scene: material binding not inherited, shader connections not followed, unknown material treated as valid, or texture path resolved from `cwd`.
- Texture looks washed out or normal map is wrong: sRGB applied to data textures, missing scale/bias, channel swizzle mismatch, or ORM channels mapped incorrectly.
- Missing shelves, chess pieces, or repeated assets: prototype-only prims drawn or hidden incorrectly, PointInstancer expansion skipped, or native instances flattened without prototype lookup.
- Lights visible but no shadows or no direct light: light transform, intensity/exposure/normalize, visibility, unsupported light type, alpha cutout, or TLAS membership mismatch.
- Black glass, opaque cutouts, dark roof openings, or missing shadow bands: opacity, opacity threshold, transmission, IOR, authored/schema-fallback sidedness, or shadow opacity policy diverges between raster, primary RT, instance RT, and shadow rays.
- Correct first frame but wrong animation: default-time values cached despite time-varying transforms, geometry, materials, or visibility.

## Handoff

When finishing loader work, report the scene path, time code, root layer, counts, bounds, material/light/texture totals, instancing totals, unsupported features, key diagnostics, validation command, and any renderer-degraded semantics that remain.