Spg

Sensor Processing Graphs

When to Use

Any task where the user's own GPU code has to run inside the renderer, over the images or sensor data the renderer produces. Trigger vocabulary: SPG, .cu.lua, .slang.lua, info:spg:sourceAsset, subIdentifier, omni:rtx:aov, orderedVars, cuda.kernel, slang.dispatch, spg:rtx.spg.stdlib/..., "post-process the AOV", "custom kernel on the render output", "my SPG AOV is black".

Not this skill: reading a finished AOV back on the host. Use reading-render-output.

Task → Sections

Read The Model, Instructions and The Traps for any task. Then only what the row names.

Prerequisites

• ovrtx 0.4 or later. SPG is enabled by default.
• The user can already render: a stage, a RenderProduct with a camera, and a step loop.
• Slang nodes need the renderer on Vulkan, which is the ovrtx default on Linux and Windows.
• Proficiency with CUDA kernels or Slang/HLSL compute shaders, and with USD authoring, is assumed. This skill maps those onto SPG rather than teaching them.

Inputs

The user's scene file, their kernel if they have one, and which backend they want. If they have not said, choose by what the node needs: Choosing a Backend.

Every examples/ and docs/ path below is relative to the ovrtx public root: rendering/ovrtx/public/ in the kit repository, the package root in an installed ovrtx.

Precedence when facts conflict: the code under examples/python/spg-* first, then the pages under docs/spg/, then this skill. Every code reference below names a snippet marker in a runnable example. Read the file between the markers rather than relying on memory; this skill is not the source of truth for API shape. Markers are paired comments carrying the name:

// [snippet:grayscale-kernel-template]   ... // [/snippet:grayscale-kernel-template]

// in .cu and .slang, -- in .lua, # in .py and .usda. Conventions in skills/README.md.

The Model

SPG runs a directed graph of GPU nodes inside the renderer's frame, over its Arbitrary Output Variables (AOVs). Everything stays on the GPU.

A node is three files that share a name: the GPU code, Kernel.cu or Kernel.slang; the launch script beside it, Kernel.cu.lua or Kernel.slang.lua, which validates inputs, describes outputs and returns the launch configuration; and the USD shader definition, Kernel.usda, which declares the ports and points at the GPU source.

Three facts drive most mistakes:

1. The launch script is found by appending .lua to the source asset path, and cannot be redirected. GrayscaleKernel.cu pairs with GrayscaleKernel.cu.lua.
2. The launch script runs once per rendered frame, on the CPU, before the GPU work, and sees resource descriptors — shape, dtype, rank — never pixel or point data.
3. Execution is driven backwards from what is asked for. A node whose output nothing consumes and which publishes no RenderVar never runs at all.

Instructions

1. Decide built-in or custom. If a stdlib node covers the operation, author only a USD Shader prim and write no code: Built-In Nodes. Do not invent node IDs.
2. Write the GPU code, one entry point, named the same in all three files and extern "C" on CUDA so the symbol survives.
3. Write the launch script, one global Lua function of that name taking (inputs, outputs), assigning a descriptor to every declared output and returning a launch configuration.
4. Write the shader definition: info:implementationSource = "sourceAsset", info:spg:sourceAsset at the GPU file, subIdentifier naming the entry point, and one opaque inputs:/outputs: port per resource. Details in The Three Files.
5. Wire it into the scene: Wiring the Graph.
6. Verify by content, not by exit code. A node that never ran leaves an AOV of zeros, so a clean run proves nothing. Assert something only your node could have produced, as every spg-* example's README.md does. If you cannot run anything, hand the user the command and the check rather than reporting the work as verified.

The smallest complete node, in all three files:

Source (kernel): examples/python/spg-grayscale/GrayscaleKernel.cu snippet grayscale-kernel-template Source (launch script): examples/python/spg-grayscale/GrayscaleKernel.cu.lua snippet grayscale-launch-template Source (shader definition): examples/python/spg-grayscale/GrayscaleKernel.usda snippet shader-definition-template Source (scene): examples/python/spg-grayscale/grayscale_scene.usda snippet render-graph

The Three Files

The entry-point name appears three times and must match exactly: the GPU source's function, the launch script's global function, and subIdentifier in USD, which falls back to the Shader prim's own name when absent.

Ports are declared in USD and keyed by name in Lua, with the inputs: scope stripped, so inputs:LdrColor arrives as inputs["LdrColor"]. A resource port is opaque inputs:X or opaque outputs:Y and carries an AOV or a buffer another node produced. A value input is a typed attribute, float inputs:strength, carrying a number, vector, matrix, token or asset.

A resource is described by shape, rank and dtype. A dtype is a value taken from the cuda or slang table, not a type name: cuda.uchar4 is four unsigned 8-bit numbers, so one RGBA pixel. It compares, inputs["Image"].dtype == cuda.uchar4, and it is callable to wrap a value, cuda.int(42).

Shapes are height-first. shape[1] is height and shape[2] is width, while cuda.image(width, height, dtype) takes width first. That reversal is the most common cause of a transposed or wrongly proportioned result.

Every declared output must be assigned a descriptor before the launch script returns, or it is never backed and the node has nothing to write into.

Choosing a Backend

CUDA and Slang are the same shape: same three files, same ports, same launch-script contract. They differ in how the GPU code receives things.

The same node in both, to compare directly:

Source (CUDA): examples/python/spg-pipeline/InvertKernel.cu.lua snippet invert-launch Source (Slang): examples/python/spg-pipeline/InvertKernel.slang.lua snippet invert-slang-launch

Values

Declare a typed attribute on the shader definition; a scene instance may override it.

Source (USD): examples/python/spg-pipeline/InvertKernel.usda snippet invert-shader-definition

A value input is a wrapper, not a number. Arithmetic on inputs["strength"] raises. Pass it through a dtype constructor to hand it to the GPU, or read .value to compute with it in Lua.

A vector is not a scalar on CUDA. Scalars pass by value, but a float3, a matrix or a quaternion is bound with cuda.array and arrives as a pointer to its components; passing one by value gives a correct first component and garbage after it. On Slang every value goes into the parameter block.

Two traps in the type mapping. A quaternion is written (w, x, y, z) in USDA and arrives as (x, y, z, w). A token arrives as a null-terminated char array and an asset as the file's raw bytes, so both are bound with array, not a scalar constructor.

Depth, including every USD type SPG reads: docs/spg/ref/usd.rst and docs/spg/do/values.rst.

Resources

Backing is fixed when a resource is created. image asks for a texture, empty for a buffer, and the binder used later has to agree. A texture has one, two or three dimensions; anything else is buffer-backed.

On Slang the binder must match how the shader declares the parameter. The binders check rank, direction, backing and buffer kind in Lua before anything reaches the GPU, naming the port.

A uchar4 texture is normalised, alone among the 8-bit element types, so Slang writes float4 in 0..1 with hardware rounding while CUDA writes raw bytes. The same colour can land one least-significant bit apart on the two backends; where a test must match byte for byte, pick colours exact in 8 bits.

Texture in and texture out, then buffers and textures mixed in one bind list:

Source (CUDA): examples/python/spg-grayscale/GrayscaleKernel.cu.lua snippet grayscale-launch-template Source (Slang): examples/python/spg-grayscale/GrayscaleKernel.slang.lua snippet grayscale-slang-launch-template Source (mixed): examples/python/spg-composite-aov/RangeHistogramKernel.cu.lua snippet histogram-binding

Depth: docs/spg/do/textures_buffers.rst.

Launch Geometry

Both keys are optional and SPG derives them. Left out, the geometry comes from the first output's shape: one thread per element. On CUDA the derived block is 16 x 16 x 1, or 256 x 1 x 1 for a rank-1 resource.

On Slang, numthreads states the group size the shader was compiled with, so SPG can divide the output's shape into groups. It is for a .spv, which carries no reflection; nothing compares it against the byte code, so a value that differs mis-covers the output. A shader that declares [numthreads] supersedes it, logged at INFO level and so invisible by default.

State the geometry when the iteration domain is not the output. A kernel whose threads walk a buffer, or take one thread per column rather than per pixel, must say so. Round up when dividing, or the last partial group never launches and the far edge is never written. Both cases, one pass stating nothing and one stating the domain:

Source (derived): examples/python/spg-blur/BlurKernel.cu.lua snippet blur-horizontal-launch Source (stated): examples/python/spg-blur/BlurKernel.cu.lua snippet blur-vertical-launch

Depth: docs/spg/do/launch_geometry.rst.

Wiring the Graph

A graph is authored under a RenderProduct: a RenderVar names an AOV, orderedVars lists what the product produces, and a shader-to-shader connection chains two nodes without publishing what passes between them.

Source (publishing): examples/python/spg-grayscale/grayscale_scene.usda snippet render-graph Source (chaining): examples/python/spg-pipeline/pipeline_scene.usda snippet render-graph Source (no input): examples/python/spg-generate/CheckerKernel.cu.lua snippet checker-launch

Order comes from connections, not from orderedVars. SPG runs nodes in topological order over the edges the connections create, so an unpublished intermediate stays inside the graph and is unreadable from the host. That is the default; publishing is the deliberate act.

Publishing needs both parts: a sourceName, which registers the name, and the connection. A RenderVar with only one produces nothing. Under a name the renderer already produces it overwrites in place, which downstream consumers pick up unknowingly.

Fan-out and fan-in are ordinary connections, but a node output binds to one RenderVar at a time, and a node cannot read an AOV and republish under that same name in one product.

A node needs no input at all. With no resource port it generates its output from authored values, and must state its own size because nothing hands it a shape.

Depth: docs/spg/do/chaining.rst, split_join.rst, aovs.rst, products.rst, generate.rst.

Built-In Nodes

Named rather than sourced: no GPU file and no launch script, just info:implementationSource = "id" and info:id = "spg:<node-id>".

Source: examples/python/spg-builtin-nodes/stdlib_scene.usda snippet render-graph

The four that exist:

That is the complete standard library and the whole of the published surface. Other renderer components register nodes into the same registry under their own prefix, so an info:id outside this table may still resolve; treat anything else as internal and unsupported. These four take 2D textures only and refuse integer formats. A built-in is the whole node: the moment you need something it does not do, you are writing a node.

Depth: docs/spg/ref/builtin_catalogue.rst.

Cross-Frame State

Two mechanisms, differing in who owns the data.

A stateful output is the node's own resource, handed back to it next frame instead of a fresh one. Mark it with a trailing cuda.stateful / slang.stateful on the allocation. It is zero-initialised on first use and needs no RenderVar.

A previous-frame read is an AOV, so any node connected to it sees the same history. Suffix a RenderVar's sourceName with :-N, for N from 1 to 8. Nothing on the node says anything about time; that follows from what the scene connects it to.

Source (stateful, CUDA): examples/python/spg-stateful/TrailKernel.cu.lua snippet trail-alloc Source (stateful, Slang): examples/python/spg-stateful/TrailKernel.slang.lua snippet trail-slang-alloc Source (previous frame): examples/python/spg-previous-frame/motion_scene.usda snippet render-graph

An unrecognised suffix is not an error: it leaves the name unchanged and reads the live AOV under that literal name, which looks like the feature silently not working.

Depth: docs/spg/do/state.rst, previous_frame.rst.

Uploading Data

array takes a Lua table and a dtype and creates a device buffer, or uploads an asset value input's raw bytes; zeros, ones and full fill one without Lua data.

An asset file is resolved exactly as the GPU source is, so it may sit beside the scene, inside a .usdz package, or behind a URI the Omniverse client can reach.

Wrap the construction in static so it is built once rather than on every frame, since the launch script runs per frame under an instruction budget.

Source (CUDA): examples/python/spg-blur/BlurKernel.cu.lua snippet blur-weights Source (Slang): examples/python/spg-blur/BlurKernel.slang.lua snippet blur-slang-weights

static caches the Lua work, not the GPU upload: the bytes are copied again each frame. For a filter kernel that costs nothing measurable, so size a design around the Lua saving rather than the transfer.

Depth: docs/spg/do/upload_data.rst, caching.rst.

Runtime Changes

A value input is addressable on the Shader prim by its USD name. Write it, publish the edit by advancing the write floor, then step.

Source: examples/python/spg-blur/main.py snippet blur-write-radius

An unconnected input takes effect on the next step, with no reset. A connected input ignores a direct write: the connection is the source, so the write succeeds and changes nothing; write the source attribute instead and reset the renderer. Each node carries its own copy, so retuning two nodes means writing both.

Depth: docs/spg/do/runtime_changes.rst.

Sensor Composites

A lidar or radar publishes a composite: several named channels under one render var rather than the single image a camera AOV gives. The node's port is declared like any other, and nothing on the shader side says "composite".

Source (launch script): examples/python/spg-composite-aov/RangeHistogramKernel.cu.lua snippet histogram-launch Source (scene): examples/python/spg-composite-aov/lidar_scene.usda snippet lidar-render-graph

Test inputs["X"].isComposite first — it is the only way to tell the two shapes apart, and a node written for one will not work on the other. Then take channels from .tensors[name] and scalars from .params[name], with .channelNames saying what this render var carries. A render var with no channels authored is not a composite: it binds as a one-dimensional uchar buffer of raw bytes.

Bound the work by the count channel. Point channels are allocated for the worst case and only the first Counts[0] entries hold a return. The launch script sees descriptors, not data, so pass Counts to the GPU and apply the bound there. Coordinates is [3, Nmax] — all x, then all y, then all z — so the stride between runs is the capacity, not the count.

Do not branch on .status: it reports what the render output said when the launch script ran, and reads "empty" on every frame of the working example while the channels carry a full sweep.

Depth: docs/spg/do/composites.rst.

Ray Generation

Slang only. The entry point is [shader("raygeneration")] and the launch script returns slang.rayQuery or slang.traceRays in place of slang.dispatch. Either way the ray grid is the output's shape, one invocation per element, so there is no numthreads.

slang.rayQuery traverses and shades inline in the ray-generation shader and needs nothing else. slang.traceRays drives a pipeline with a shader binding table, so shading moves into one miss and one hit group in the same file, named by the extra launch fields miss, hit, payloadSize and attributeSize.

Source (inline): examples/python/spg-raygen/RaygenCornellBox.slang.lua snippet raygen-launch Source (pipeline): examples/python/spg-raygen/RaygenCornellBoxPipeline.slang.lua snippet pipeline-launch

A shader under a RenderProduct is given the scene acceleration structure and its camera with no authored input and no connection: bind it with slang.binding("scene", inputs["scene"]) and place rays with the sceneTransform value input rather than assuming world coordinates. Exactly one hit group is supported, payloadSize and attributeSize are byte counts you work out yourself, and a .spv cannot be used on a ray-tracing stage.

No geometry is exposed. A hit yields distance and the surface's instance and primitive index, and nothing else, so the example recovers geometric normals from two extra probe rays.

Depth: docs/spg/do/raygen.rst.

Pre-Compiled Sources

The extension on info:spg:sourceAsset decides what happens: CUDA takes .cu, .ptx, .cubin and .fatbin, Slang takes .slang, .slang-module and .spv. The launch script is still required and still found by appending .lua, and a CUDA entry point must be extern "C" so the symbol is findable in the artifact.

A .spv carries no reflection at all: the descriptor space, the binding slots, the parameter-block offsets and the thread group size all have to be stated in the launch script, and it cannot be used on a ray-tracing stage. A .slang-module keeps its reflection and binds like source.

Sources and asset value inputs may live beside the scene, inside a .usdz package, or behind a URI the Omniverse client can reach, and the launch script always follows its source; inside a package the .lua goes within the brackets.

Depth: docs/spg/do/precompiled.rst, descriptor_sets.rst, docs/spg/ref/usd.rst.

The Traps

These bite across every task, and each produces a plausible result rather than an error.

1. Shapes are height-first, while the image constructors take width first.
2. Argument order is unchecked. CUDA args is matched against the C signature by position and a mismatch is not diagnosed: the kernel reads whatever occupies that slot.
3. A value input is a wrapper, so arithmetic on it raises. Use .value.
4. The launch script has a fixed VM instruction budget, about a thousand, renewed each frame and not adjustable. Loops over attributes or large tables exceed it and fail the node with an execution-limit error. Build assert messages as plain literals, since a concatenated message costs its instructions whether or not it fires.
5. The sandbox scans the script's text before running it, as a plain substring match over the whole file including comments. A comment mentioning a forbidden construct is enough to reject the script, which surfaces as an output that is never written.
6. info() and print() are invisible at the default log level. Use warning() when you need to see something.
7. A rejected or never-scheduled node leaves zeros and exits 0. Never take a clean run as evidence.

Output Format

State which of the three files you changed and why. Show the entry-point name and confirm it matches in all three. If you added a port, say whether it is a resource port or a value input. Name the check that proves it worked, and what its failing value would look like.

Scripts

This skill has no scripts. Every spg-* example runs itself with uv run main.py, as its README.md states.

Limitations

• No geometry, materials or vertex data reach a node, including a ray-generation node.
• No node reaches the CPU mid-frame, and no launch script reads pixel or point data.
• A launch script cannot read the scene: no prim attributes, no camera transform, no product layout. A node gets what its ports carry and its authored value inputs.
• A shader output binds to one RenderVar at a time; a node cannot read an AOV and republish under that same name in one product.
• Previous-frame reads reach 8 frames back.
• A cross-product read can be one frame stale when the consuming product renders first.
• SPG Shader prims must not sit under a Material prim, and shaders under an instance prototype are ignored.
• Slang has no 64-bit integers, CUDA cannot ray-generate or split across files, and the built-in nodes take 2D textures only and refuse integer formats.

Troubleshooting

Start from what was observed. Full index: docs/spg/diagnose/symptoms.rst.

The renderer writes its log to standard output. Launch-script warning() lines appear there prefixed Lua:.

References

Docs are the depth behind every section here, under docs/spg/: overview.rst for what SPG is, first_node.rst for one node built from nothing, do/index.rst for one page per task, ref/index.rst for every Lua function and USD attribute, and diagnose/index.rst for the symptom index and the limitations.

Runnable examples, each self-checking: spg-grayscale (smallest node), spg-pipeline (chaining, value inputs), spg-builtin-nodes, spg-generate (no input), spg-blur (uploads, caching, launch geometry, runtime changes, fan-out), spg-stateful, spg-previous-frame, spg-composite-aov, spg-raygen.

Related skills: reading-render-output for getting an AOV back on the host, loading-usd and stepping-and-rendering for the surrounding loop, cuda-interop for sharing GPU memory. Editor support: type stubs in docs/spg/.luarc/, wired up as docs/spg/ref/lua_contract.rst describes.