TouchDesigner Python API Reference
Always research TD features on the wiki before writing code. Assumptions about TD's Python API are frequently wrong.
- Wiki home: https://docs.derivative.ca/Main_Page
- OP Class: https://docs.derivative.ca/OP_Class
- COMP Class: https://docs.derivative.ca/COMP_Class
- Par Class: https://docs.derivative.ca/Par_Class
- Extensions: https://docs.derivative.ca/Extensions
- Cook cycle: https://docs.derivative.ca/Cook
Parameter Access Patterns
# CORRECT -- always use .eval() for the current runtime value:
value = op('geo1').par.tx.eval()
# WRONG -- .val only works in constant mode:
value = op('geo1').par.tx.val
# Setting values:
op('geo1').par.tx = 5
op('geo1').par.tx.val = 5 # CAUTION: silently switches to CONSTANT mode
# Menu parameters accept string name or index:
op('geo1').par.xord = 'trs' # by name
op('geo1').par.xord = 5 # by index
# Type casting requires explicit .eval():
me.par.tx.eval().hex() # CORRECT
me.par.tx.hex() # WRONG
Creating Custom Parameters
All append* methods return a ParGroup (tuple-like), not a single Par -- always index with [0].
page = comp.appendCustomPage('Controls')
pg = page.appendFloat('Speed', label='Speed') # Returns ParGroup
p = pg[0] # Get the actual Par
p.default = 0.5
p.normMin = 0; p.normMax = 2 # Slider range
p.min = 0; p.clampMin = True # Hard clamp
p.help = "Playback speed multiplier." # Tooltip on hover
p.startSection = True # Draw separator line above
# Other methods:
page.appendInt('Count')
page.appendToggle('Active')
page.appendStr('Label')
page.appendMenu('Mode') # Creates EMPTY menu -- set .menuNames/.menuLabels separately
page.appendPulse('Reset')
page.appendRGB('Color') # Creates Color1r, Color1g, Color1b
page.appendXYZ('Pos') # Creates Pos1, Pos2, Pos3
page.appendOP('Target')
page.appendFile('Path')
# Post-creation properties:
p.help = "Tooltip text shown on hover" # ALWAYS set this
p.startSection = True # Draw separator line above this parameter
p.order = 11.5 # Insert between order 11 and 12
p.readOnly = True # Display-only parameter
# Cleanup:
comp.destroyCustomPars() # Remove ALL custom pars
par.Speed.destroy() # Remove single custom par
Naming rule: First letter MUST be uppercase, rest lowercase/numbers. No underscores.
- Get-or-create (never blind
append*), callbacks, styles, ranges, help text:/parameter-design - Docs: https://docs.derivative.ca/Custom_Parameters
op() vs opex()
node = op('/nonexistent/path') # Returns None -- silent failure
node = opex('/nonexistent/path') # Raises tdError -- clear message
all_noises = ops('noise*') # Returns a LIST (supports wildcards)
Use op() only when None is an acceptable result.
Operator Referencing Patterns
How you reference an operator matters. A wrong choice works today and breaks tomorrow -- when the component is renamed, instanced, or moved. Always pick the narrowest, most portable reference that correctly resolves from where the code runs. Absolute paths (op('/project1/...')) are always wrong -- in code, expressions, AND parameter values.
The ladder -- take the highest rung that resolves:
| Rung | Pattern | Use when | Prerequisite |
|---|---|---|---|
| 1 | self.ownerComp |
Anywhere inside an extension. Capture it once in __init__. |
none |
| 2 | parent.CompName |
Inside the COMP but outside the extension -- callback DATs, parameter expressions | the COMP sets par.parentshortcut |
| 3 | op('sibling'), op('./child') |
Target is in the same network or below you | none |
| 3b | iop.Name / ipar.Name |
A specific internal operator or parameter the COMP publishes for its own descendants | the COMP sets Internal OP / Internal OP Shortcut on its Common page |
| 4 | op.CompName |
A project-wide singleton no parent shortcut can reach | the COMP sets par.opshortcut |
| -- | op('/abs/path') |
never | -- |
The bare global parent() / parent(N) is banned inside an extension class body -- use self.ownerComp. parent(2) encodes the nesting depth and resolves to the wrong COMP the moment anything moves; a parent shortcut searches up by name and survives renesting. Three exemptions, all real: a COMP's own callback DAT (where parent() is the owner), generic tooling operating inside a COMP it did not author, and parameter expressions on replicated or cloned widgets. some_op.parent() is untouched -- that is documented OP-class API, not the global.
Set par.parentshortcut when you create a reusable COMP. The shortcut does not exist until something sets it, which is exactly why so much code falls back to parent() chains.
Relative Paths -- for operators near you
Use relative paths when the target operator is in the same network or a nearby one. These are the simplest and most portable references because they describe relationships, not locations.
op('sibling_name')-- another operator in the same network (same parent COMP).op('./child_name')-- an operator insideme(only valid from a COMP).op('../sibling_of_parent')-- an operator in the parent's network (go up one level, then find by name).
Relative paths break down when you need to reach across distant parts of the network. That's where shortcuts come in.
Parent Shortcuts (parent.CompName) -- for reaching your owner
A Parent Shortcut is set on a COMP's Common page via the parentshortcut parameter. Once configured, any operator that is a descendant of that COMP (child, grandchild, etc.) can reference it as parent.CompName. TD resolves this by walking up the parent chain from the caller until it finds a COMP whose Parent Shortcut matches.
This is the right choice when code running inside a component needs to reach the component itself -- typically to call extension methods or navigate relative to the component root.
parent.Embody.Update()-- call a promoted extension method.parent.Embody.ext.Embody.helperMethod()-- reach a non-promoted method.parent.Embody.op('subpath/op_name')-- navigate from the component root to find an internal operator.
Key properties:
- Reusable across instances: Multiple COMPs can use the same Parent Shortcut name. Each descendant resolves to its own nearest matching ancestor -- so the same code works identically across every instance of a component.
- Only resolves from inside: Code that is not a descendant of the COMP will not find it via
parent.CompName. This is a feature, not a limitation -- it keeps references scoped to where they belong. - Not the same as
parent():parent()always returns the immediate parent COMP.parent.CompNamesearches upward by name and can skip multiple levels.
Internal shortcuts (iop.Name / ipar.Name) -- stable internal addresses
Same family as parent.CompName: both search up the parent hierarchy by a shortcut set on the Common page, so both survive renesting. The difference is granularity -- parent.Comp hands you the component and you navigate down; iop.Name names one internal operator directly, ipar.Name a parameter-holding COMP. Use them when a component wants to publish a stable internal address to its own descendants without freezing the path between them (a widget reaching iop.Readout keeps working when the readout moves). Two caveats: they resolve only from INSIDE the component, and an operator-valued internal parameter needs op(ipar.Effect.Operatorpath) to yield the operator rather than the path string. Every shortcut is configuration on the COMP rather than in the code (a reader has to open the Common page to follow it), so reserve them for genuinely stable internal addresses, not as a default. (Function Store, issue #94.)
Global OP Shortcuts (op.CompName) -- for project-wide access
A Global OP Shortcut is set on a COMP's Common page via the opshortcut parameter. It registers the COMP so that op.CompName resolves to it from anywhere in the project.
This is the right choice for singleton services that many unrelated parts of the project need to reach -- logging, test runners, shared managers.
op.Embody.Log('message')-- call Embody's logging from anywhere.op.unit_tests.RunTests()-- kick off the test runner from any script.
Key properties:
- Globally unique: Only one COMP can hold a given Global OP Shortcut name at a time. Assigning a name already in use removes it from the previous holder.
- Use sparingly: If
parent.CompNameworks, prefer it. Global shortcuts create invisible coupling -- any code anywhere can depend on the name existing, making renames and refactors risky.
Always verify references resolve correctly from the calling context -- a reference that returns None silently (or finds the wrong operator) is a latent bug.
debug() vs print()
debug('value is', x) # "myScript line 42: value is 42" (with source info)
print('value is', x) # "value is 42" (no source info)
Module-Level Code Hazard
Never call op(), parent(), or access TD objects at module level. They execute during import, before the network is ready.
# WRONG:
my_op = op('base1') # May be None during init
# CORRECT -- defer to methods:
class MyExt:
def doSomething(self):
my_op = op('base1') # Resolved at call time
Import Shadowing
TD searches for DATs by name before sys.path. A DAT named json shadows Python's json module.
mod() for Module Access
# In a parameter expression (import not available):
mod.utils.myFunction()
# In a script (decreasing performance):
import utils # Fastest
m = mod.utils; m.func() # OK -- cache the reference
mod.utils.func() # Slowest -- re-resolves every call
# Access by path (relative to the current op -- never absolute):
mod('utils').myFunction()
# Direct module property:
op('myDat').module.myFunction()
extensionsReady Guard
# In a parameter expression:
parent().MyExtensionProperty if parent().extensionsReady else 0
onInitTD and TDXN Import Timing
Any initialization that sets up state inside a TDXN-strategy COMP will be destroyed when TDXN import runs. TDXN reconstruction (ext.Embody.reconstructTDXNComps) calls ImportNetwork with clear_first=True, which deletes all children and recreates them from the .tdxn file. If an extension's onInitTD creates operators, sets parameters, stores values, or builds internal state inside a TDXN COMP, that work is wiped out by the import.
This applies to:
- Project open:
ext.Embody.reconstructTDXNCompsruns at frame 60. Extensions inside TDXN COMPs initialize earlier (when the COMP shell is created), soonInitTDfires before the import overwrites everything. - Ctrl+S /
project.save(): The strip/restore cycle deletes children pre-save, then re-imports them post-save. Extensions reinitialize after the restore, but the import may still be completing.
Rules:
- Defer initialization that depends on network state. Use
run('self.mySetup()', delayFrames=5)inonInitTDso the setup executes after the TDXN import completes. The delay must be long enough for all import phases to finish. - Never assume
onInitTDruns once. Inside TDXN COMPs, extensions may reinitialize multiple times: on project open, after every save (strip/restore), and on manual TDXN reimport.onInitTDmust be idempotent. - Guard against missing children. During the strip phase of a save, the COMP's children are temporarily gone. If
onInitTDfires during this window,op('child')returnsNone. Always null-check operators before accessing them. - Store persistent state outside the TDXN boundary. If an extension needs state that survives reimport, use
store()on the COMP itself (storage is preserved through TDXN import) or on an ancestor outside the TDXN COMP.
Parameter, Storage, or Dependency?
Short form: user-facing state -> a custom parameter; hidden durable bookkeeping -> storage (lives on the COMP, survives an extension reinit, wiped by a tox reload / TDXN reconstruction); a derived value that must recook its readers -> tdu.Dependency (dies with the extension instance on every source save -- rebuild it in __init__). The lifetime table, what actually notifies dependents, and the storage mechanics (fetch without a default RAISES; fetch searches UP; unstore is glob-matched; write through store(), never comp.storage[k] = v; pickling at save; op references ARE storable) are canonical in /parameter-design (Parameter, Storage, or Dependency?) -- read that, do not re-derive it here.
Operator Storage
op('base1').store('count', 42)
val = op('base1').fetch('count', 0) # 0 is default -- omit it and a miss RAISES
op('base1').unstore('count') # glob-matched: never a computed key
op('base1').storeStartupValue('version', 1) # Restored on project load
tdu.Dependency for Reactive Values
dep = tdu.Dependency(0)
dep.val = 5 # CORRECT -- triggers recooks
dep = 5 # WRONG -- destroys the Dependency object
current = dep.peekVal # Read without creating dependency
dep.val = [1, 2, 3]
dep.val.append(4) # Does NOT trigger update
dep.modified() # Required
tdu Utility Functions
tdu.clamp(val, min, max)
tdu.remap(val, fromMin, fromMax, toMin, toMax)
tdu.rand(seed) # Deterministic [0.0, 1.0)
tdu.base('noise3') # 'noise'
tdu.digits('noise3') # 3
tdu.validName('my op!') # 'my_op_'
tdu.match('noise*', ['noise1', 'c1']) # ['noise1']
tdu.expand('A[1-3]') # ['A1', 'A2', 'A3']
tdu.tryExcept(expr, fallback)
DAT Cell and Text Behavior
All DAT cells are internally strings. Auto-cast to numbers in expression contexts.
n = op('table1')
n[1,2] + 1 # 4 (auto-cast)
n[1,2].val + 1 # TypeError: str + int
dat.text-- tab/newline delimited; strips multi-line cell content. Usedat.csvfor cells with newlinesdat.jsonObject-- parses as JSON directly (nojson.loads()needed)dat.module-- access as Python moduledat.write(content)-- appends (does not overwrite)- Docs: https://docs.derivative.ca/DAT_Class
CHOP Channel Access
ch = op('noise1')['chan1'] # By name -- NO wildcard
chs = op('noise1').chans('tx*') # Pattern matching
val = ch.eval() # Current value
ch[0], ch[10] # Sample by index
ch.evalFrame(30) # At specific frame
arr = op('noise1').numpyArray() # Shape: (numChans, numSamples)
TOP Pixel Access
Coordinate system: TD places (0, 0) at the bottom-left, with Y increasing upward for all texture operations. TOP.numpyArray() is the exception: it returns rows top-to-bottom (numpy convention).
| Context | Origin | Y direction |
|---|---|---|
TOP.sample(x, y) |
Bottom-left | Up |
GLSL gl_FragCoord |
Bottom-left | Up |
| UV coordinates (0-1) | Bottom-left | Up |
| Crop/Transform TOP params | Bottom-left | Up |
scriptTOP pixel writing |
Bottom-left | Up |
TOP.numpyArray() return |
Top-left | Down |
| PIL / OpenCV images | Top-left | Down |
| Panel/widget screen coords | Top-left | Down |
TOP.sample(x, y) downloads the entire texture from GPU -- extremely expensive. Never in loops.
# sample() uses TD texture coords: y=0 is BOTTOM of image
r, g, b, a = op('noise1').sample(x=0.5, y=0.5) # Center of texture
r, g, b, a = op('noise1').sample(x=0, y=0) # Bottom-left corner
# numpyArray() returns rows TOP-to-BOTTOM (opposite of TD texture coords)
arr = op('noise1').numpyArray() # [height, width, channels] -- NOT [width, height]
# arr[0] is the TOP of the image (highest TD Y)
# arr[-1] is the BOTTOM of the image (TD y=0)
# Flip to match TD bottom-up order:
arr_td = np.flipud(arr)
Color domain: numpyArray() is NOT sRGB file bytes. TOP.numpyArray() returns the TOP's raw pixel values -- linearized/linear-light floats for a float TOP -- NOT the sRGB-gamma-encoded 8-bit bytes that cv2/PIL read from a .png/.jpg. A direct pixel diff across the two domains is invalid: it shows a ~0.1-0.4 baseline difference that swamps any real per-pixel change. For any pixel-comparison or frame-exactness workflow (e.g. verifying a movie encode), compare same-domain only -- reader numpyArray() vs reader numpyArray() -- or convert one side (apply/remove the sRGB transfer) before comparing. This is why in-TD readback and out-of-process file decodes must not be diffed against each other directly.
POPs -- GPU-Accelerated Point Operators
POPs process 3D geometry on the GPU (analogous to SOPs but GPU-accelerated).
grid = parent.create(gridPOP, 'grid1')
n = pop_op.numPoints(delayed=True) # Non-blocking
pts = pop_op.points('P') # Downloads (blocks GPU)
bounds = pop_op.bounds(delayed=True) # Non-blocking
attrs = pop_op.pointAttributes # Set of attribute names
Common types: gridPOP, noisePOP, transformPOP, particlePOP, spherePOP, linePOP, mergePOP, nullPOP, selectPOP, mathPOP, cachePOP, glslPOP. For files: fileinPOP (File In POP -- meshes/geometry) vs pointfileinPOP (Point File In POP -- 3D point clouds: .ply/.pts/.xyz/.e57, Gaussian splats) are distinct operators.
run() -- Delayed Code Execution
# Resolve ops via fromOP (sets `me`) or global shortcuts -- never absolute paths
run("me.cook(force=True)", fromOP=op('base1'), delayFrames=1)
run("print('done')", delayMilliSeconds=500)
run("op.Embody.Update()", endFrame=True)
run(myFunction, arg1, arg2, delayFrames=5)
Cook Model Gotchas
cook(force=True)does NOT advance a feedback loop within a frame. A Feedback TOP captures its target on frame boundaries, so force-cooking the chain repeatedly inside one synchronous Python loop returns the same state each time (totalCooksmay not even increment). Evolution needs real frames to pass with the chain demanded -- drive it withrun(..., delayFrames=1)or an Execute DATonFrameStart, never aforloop.- A Movie File In reload lands only across a real frame advance -- and even then not same-pass downstream. Changing
par.file/ pulsingreloadpulsethencook(force=True)in the SAME frame can silently serve the PREVIOUS texture (no error, no warning, right resolution); a pull-based reader that nothing demands never cooks at all. Worse, when the reload DOES apply mid-pass, ops DOWNSTREAM in that same forced-cook pass can still consume the pre-reload texture -- even with the whole chain force-cooked in dependency order -- so the reader's ownnumpyArray()shows fresh content while the chain output lags by one frame. Verify content at the POINT OF CAPTURE (the writer's input TOP), not at the source, and let each reload settle across a real frame advance. See /movie-export (Async file readers serve stale content). - A CHOP Execute DAT is a per-frame demand source. Its watched CHOP cooks every frame whether or not anything else pulls it (measured 2025.33230: 991 cooks in ~1000 frames vs 1 for an identical unwatched CHOP) -- a hidden always-cook to account for in idle-census counts. The reverse trap: an Execute DAT's
framestarttoggle is OFF by default, so a frame driver that was never enabled demands nothing. - Animate cheaply: static source + cheap downstream. A heavy generator (high-octave fBm, large feedback sim) cannot re-render every frame at high resolution. Make it static (remove every time reference so it cooks once and caches) and put the motion in a cheap downstream op -- animate the sampling (drift/rotate/warp the read coordinates), not the source. Verify with
cookedThisFrame: the source readsFalse, the animated opTrue.
Background and Long-Running Work
Ironclad rule (a read is treated exactly like a write). From any thread but the main thread, NEVER touch a main-thread-owned TD object: op()/opex(), a Par/ParGroup (read OR write, including .eval()/.val on a live parameter), DAT/CHOP/SOP/TOP content, storage (fetch/store), tdu.Dependency (setting .val recooks on the main thread), or debug()/print() (they route to the Textport / a DAT). Never call run()/td.run() from a worker - there is NO sanctioned exception. The call may not raise on current builds (2025.3x) and the scheduled code even executes later on the main thread, but the call itself touches TD state from the wrong thread and silently corrupts it - the crash surfaces later, far from the call site (Derivative-confirmed 2026-08-17; exactly what froze TD in the field, and why "it works when I test it" proves nothing). A worker may use ONLY: pure Python (math, json, requests), tdu math/value utilities (tdu.clamp/remap/Vector/Matrix - they do not reference TD data), parameter VALUES evaluated on the main thread and passed in, queue.Queue, threading.Event/Lock, td.isMainThread() as a guard, and the Thread Manager's InfoQueue/Get/Set*Safe/SafeLogger. Resolve every op path and value on the main thread BEFORE spawning the worker; the worker returns plain data for a main-thread callback to apply.
Do NOT reach for threading first. Match the rung to the problem TYPE (these are routes, not a strict escalation); threading is the LAST resort:
- Prototype synchronously to prove the URL/auth/parse - one-shot only, short explicit timeout, never in a per-frame callback or on project open, never shipped.
- Fast, TD-only, no I/O -> run it inline. Any network/disk/subprocess call is NEVER this step (latency is unbounded).
- Fetch data -> a native TD I/O operator, NOT Python threading. HTTP one-shot or streaming -> Web Client DAT:
op('webclient1').request(url, 'GET', timeout=8000)returns a connection id immediately and never blocks the frame; theonResponsecallback fires on the MAIN thread, so write the result there. Parse with a JSON DAT (ordat.jsonObject) and bridge numbers to channels with DAT to CHOP (there is no Web Client CHOP).ws://-> WebSocket DAT; inbound/host -> Web Server DAT; control -> OSC; files -> File In / Folder DAT. - Long main-thread (TD-touching) work -> chunk with
run(delayFrames=N), each chunk small enough to fit one frame.run()controls WHEN, not HOW MUCH; it is not a thread and is main-thread-only (a single heavy parse deferred withrun()still blocks whatever frame it lands on). - Blocking pure-Python work (custom auth, file/disk, subprocess, heavy CPU) -> the Thread Manager. Prefer the Palette Thread Manager Client; advanced:
op.TDResources.ThreadManager+ aTDTaskwhosetargettouches ZERO TD objects, applying results only in its main-thread hooks. Never callEnqueueTask()from a worker (ThreadManager is itself a TD COMP). - Long-lived server/loop -> ThreadManager
standalone=True(Envoy's own MCP server, drained by itsRefreshHookon the main thread) or a top-levelthreading.Threadthat touches ZERO TD objects, never callsrun(), and hands results to aqueue.Queuedrained every frame by a main-thread callback (an Execute DATonFrameStartor a ThreadManagerRefreshHook). Arm the drain BEFORE starting the worker, and keep a lazy main-thread drain as backstop (any natural main-thread visit - a status poll, the next request - also applies pending results), so a pump that failed to arm delays delivery instead of losing it. A worker spawning arun()-calling sub-thread is the crash, not a rung.
Engine COMP / TouchEngine offloads heavy COOKING to a separate process (TOP/CHOP/DAT I/O only) - never use it for an I/O fetch. Stock asyncio blocks the frame loop; a worker-hosted loop still needs zero TD access and a queue handoff.
Triggers. Prefer a user-driven Pulse parameter (onPulse / Par.pulse()) for a one-shot/manual fetch, or a Timer CHOP for genuine intervals (fire one request per tick; a fresh Timer ships with cycle off and cyclelimit on at maxcycles 4, so a periodic timer needs cycle on and the limit cleared or it stops after four ticks). Never a sleep loop, a self-rescheduling run() poller, or an auto-fetch on project open unless asked; do not start a new request while one is still pending.
Gates. Do not pre-optimize: a synchronous fetch that does not measurably drop a frame may not need anything above Step 2 (measurement decides whether a callback needs chunking or a worker - it never makes shipped blocking I/O acceptable on the main thread). After wiring, verify with primary evidence: get_project_performance shows fps/frameTime held vs baseline and droppedFrames flat, AND the result actually arrived (read the DAT/CHOP back; branch on statusCode['code'] - a callback that never fires leaves TD running but empty).
Code pattern: Web Client DAT (no threading)
The TD-native way to hit an HTTP API. request() is async - it returns a connection id immediately and never blocks the frame; TD does the networking on its own thread and delivers the response to the Callbacks DAT onResponse, which runs on the MAIN thread (so TD access there is safe).
# Main-thread code (e.g. a Pulse parameter's onPulse, or a Timer CHOP tick):
conn_id = op('webclient1').request(
'https://air-quality-api.open-meteo.com/v1/air-quality?latitude=43.7&longitude=-79.4&hourly=pm2_5',
'GET', # method is REQUIRED (no default)
timeout=8000, # ms; async, so it never freezes the frame
)
# Fire only ONE request per Web Client DAT per frame; do not start another while one is pending.
# In the Web Client DAT's Callbacks DAT - runs on the MAIN thread:
def onResponse(webClientDAT, statusCode, headerDict, data):
# statusCode is a DICT: {'code': int, 'message': str}
if statusCode['code'] != 200:
op('status').text = 'error %s' % statusCode['code'] # surface failures, do not fail silently
return
op('raw_json').text = data.decode('utf-8') # data is BYTES - decode first
return
Then parse and shape with native ops instead of Python in the callback:
webclient1 -> raw_json (Text DAT) -> JSON DAT (Filter = JSONPath, Output Format = Table) -> DAT to CHOP -> null_chop / out1. That yields BOTH the table (DAT) and the channels (CHOP) - the canonical "CHOP and DAT" deliverable. There is no Web Client CHOP. For a quick parse you can also read op('raw_json').jsonObject. request() also accepts authType + basic/appKey/OAuth params - prefer them over hand-rolled auth headers. TD has no built-in retry: on failure, re-issue request() via run(..., delayFrames=N) with a capped attempt count, never a synchronous loop.
Pick the operator (triggers: see the ladder above -- Pulse par for one-shot, Timer CHOP onCycleStart for periodic):
| Need | Operator | Callback (main thread) |
|---|---|---|
| HTTP request/response or HTTP streaming | Web Client DAT | onResponse |
Persistent push/stream (ws://) |
WebSocket DAT | onReceiveText/onReceiveBinary |
| TD must RECEIVE requests / host an endpoint | Web Server DAT | onHTTPRequest (NOT a fetch tool) |
| Low-latency control between apps | OSC In/Out DAT | per-message |
| Local file / directory listing | File In DAT / Folder DAT | cooked DAT, no thread |
(For streaming, enable the Web Client DAT's Stream mode + Clamp Output as Rows so the DAT does not grow unbounded and inflate cook time.)
Blocking pure-Python work: the Thread Manager
When no operator expresses the work (custom auth/sessions, a blocking SDK, a big subprocess, heavy CPU on plain data), run it OFF the main thread. Prefer the Palette Thread Manager Client (Palette > ThreadManager > threadManagerClient) - a callback-oriented component with a generated callback DAT; Derivative recommends it over the raw COMP.
Advanced (raw API): the target runs on a worker and must touch ZERO TD objects; hand results back through a queue.Queue you create on the MAIN thread and drain in the RefreshHook (exactly how Envoy's MCP server works - see EnvoyExt.py):
import queue
results = queue.Queue() # created on the MAIN thread
def fetch(url, out): # WORKER: no op(), no params, no run(), no debug()/print()
import requests
r = requests.get(url, timeout=(2, 8)) # requests timeout is SECONDS; always set it (no default)
r.raise_for_status()
out.put(r.json()) # plain Python only
def on_refresh(*args): # MAIN thread, at least once per frame while the task runs
while not results.empty():
data = results.get_nowait()
op('table_out').text = repr(data) # safe: main thread
task = op.TDResources.ThreadManager.TDTask(target=fetch, args=('https://...', results), RefreshHook=on_refresh)
op.TDResources.ThreadManager.EnqueueTask(task) # or standalone=True for a long-lived server/loop
Key: the target runs on a worker and must touch ZERO TD objects (no op(), no parameter read OR write, no DAT/CHOP content, no storage, no tdu.Dependency, no debug()/print()); apply results to TD only on the MAIN thread (a RefreshHook/SuccessHook/ExceptHook, or a queue.Queue drained by an Execute DAT onFrameStart). standalone=True for long-lived tasks; the worker pool defaults to 4 (capped at os.cpu_count()). Never call EnqueueTask() from a worker (ThreadManager is a TD COMP). For worker logging use the Thread Manager's SafeLogger, not debug()/print().
- Docs: https://docs.derivative.ca/Web_Client_DAT , https://docs.derivative.ca/JSON_DAT , https://docs.derivative.ca/Thread_Manager
Large payloads
A large response is delivered to onResponse on the MAIN thread, so a heavy parse there still stalls the frame. For big/expensive parsing: onResponse validates status and copies the raw string/bytes only, then hands it to a Thread Manager worker (zero TD access) that parses and returns plain data for a main-thread drain to write. Do not "fix" it with run() - that defers the parse, it does not shrink it.
Heavy-Build Safety: Crash Causes and Safe-Default Caps
Load this section before any heavy build; the gating protocol and stop conditions live in rules/performance.md and always apply.
| Cause | Mechanism | Warning metric (threshold) | Mitigation |
|---|---|---|---|
| Resolution explosion (Resolution TOP, Optimize) | Pixel count and TOP memory scale with width*height | gpuCookTimeMs spikes or GPU headroom < 20% |
Clamp to <= 1920x1080, lower format, use Limit Resolution |
| Unbounded feedback loop (Feedback TOP) | Loop keeps accumulating data every frame | Feedback gpuCookTime or memory.gpuMemUsedMB rises each check |
Fixed resolution, decay < 1, Reset wired, bypass while wiring |
| Always-cooking operators compounding (Cook, Optimize) | Render, output, viewer, or export chains demand cooks every frame (time-dependent ops are only flagged -- undemanded they do not cook; see td-python.md Cook Model) | totalCooks climbs and cookedThisFrame stays true while idle |
Bypass during build, terminate in Null, disable viewers/outputs until measured |
| Expression-driven cook cascade (Cook) | Parameter references pull upstream nodes repeatedly | Null/In/Out cpuCookTime is large |
Cache stable values, remove cross-network expressions, inspect dependent path |
| GLSL infinite loop or GPU timeout (GLSL crash debugging) | GPU work never completes or OS resets the device | Frame time spike, UI hang, fatal Vulkan error | Constant-bounded loops only; reduce shader complexity |
| GLSL out-of-bounds array access (GLSL crash debugging) | Illegal sampler or uniform array index can crash TD | Info DAT error or crash on cook | Guard dynamic indexes with TD_NUM_*_INPUTS; validate uniform array sizes |
| Huge SOP geometry on CPU (Optimize) | CPU transforms or rebuilds many points/primitives | cpuCookTimeMs or childrenCPUCookTime jumps |
Reduce points, keep topology stable, transform at Geometry COMP object level |
| Instance/particle count explosion (Optimize) | Vertex count and buffers exceed CPU/GPU budget | gpuCookTimeMs, gpuMemoryBytes, or GPU headroom worsens |
Start small, ramp with metrics, prefer GPU instancing/POPs over Copy SOP |
| CHOP sample-count explosion (Time Slicing, Optimize) | Long buffers and audio-rate samples force large cooks | timing.cookRealTime false, timing.timeSliceMs large, CPU memory climb |
Enable Time Slicing, trim windows, reduce sample rate/buffer length |
| GPU memory exhaustion (Optimize) | TOPs, buffers, instances, and 32-bit float textures fill VRAM | GPU headroom < 20% | Stop allocation, reduce resolution/format/count, unload unused media |
| CPU memory exhaustion | Unbounded DAT/CHOP/SOP/Python data grows until process crash | memory.cpuMemUsedMB climbs with no new ops |
Bound buffers, clear caches, avoid accumulating Python lists/storage |
| Main-thread blocking Python (Python threading) | TD UI, timeline, and frame generation share the main thread | Frame time spike, UI unresponsive, MCP near 30s timeout | Chunk work, no blocking I/O or sleep, move long work off main thread safely |
Safe-Default Caps (apply when creating risky operators)
- TOP resolution: default new TOPs to bounded resolution (
<= 1920x1080). Never create 4K, 8K, or 16k unless the user explicitly asked. Before allocating, confirmw*h*channels*bytesagainstmemory.totalGpuMemMB. Prefer 8/16-bit fixed pixel formats over 32-bit float unless precision is required. - Feedback loops: ALWAYS bound them. Fix the resolution inside the loop, add a decay/multiply
< 1, wire a Reset, and keep the loop bypassed while wiring so it is not live during construction. Terminate the loop, and every TOP/CHOP chain, in a Null. - Bypass while wiring: bypass or disable cooking while wiring heavy chains. Do not leave Movie File In, Audio, Render, Timer, feedback, output, or viewer-driven ops live and cooking while building around them. Enable only after the chain is complete and measured.
allowCooking = Falseis COMP-only (tdErroron a TOP/CHOP/SOP/MAT, while= Trueis accepted anywhere, so a mixed-type gate passes its enable pass and aborts on disable); gate non-COMPs withbypass, or remove the demand. - Geometry and duplication: cap SOP point/primitive counts. For many duplicates, use GPU instancing or POPs, not Copy SOP or
comp.copy(). Transform at the Geometry COMP object level, not the SOP level. - Instances and particles: start modest and ramp up while watching
memory.gpuMemUsedMB. Never default to millions. CPU particle systems should start around 10k max; beyond that, go GPU/instancing. - CHOPs: keep sample rates and Trail/buffer windows small. Enable Time Slicing. Never create audio-scale sample-rate CHOPs without it. Use Audio File In CHOP, not Audio Play CHOP, for long files.
- GLSL: never write unbounded
fororwhileloops. Cap iterations with a constant. Bounds-check every dynamic array index withTD_NUM_*_INPUTSguards. Check the Info DAT for compile errors before relying on the op. - Python via
execute_python: keep calls short and non-blocking. No synchronous blocking I/O orsleepon the main thread. NoTOP.sample()in loops; usenumpyArray(). Avoidstore()in hot paths. Chunk large builds across frames.
Pre-Installed Packages
Commonly importable without installation: numpy, cv2 (OpenCV), requests, yaml (PyYAML), cryptography, attrs (only numpy and cv2 are documented as bundled; verify the rest in your build before relying on them). Auto-imported stdlib: math, re, sys, collections, enum, inspect, traceback, warnings.
requests blocks the frame - see the Threading ladder above. execute_python, parameter expressions, and operator/cook callbacks all run on TD's main thread, so a synchronous requests.get(...) (or urllib/socket, a large file read, subprocess.run, or a blocking DB call) freezes the whole UI/cook cycle for the round-trip - on a slow endpoint it can hang TD or exceed the 30s MCP timeout. requests has no default timeout; always pass timeout=(connect, read) in seconds. To fetch data, use the Web Client DAT (async, never blocks); if you must use requests, run it in a Thread Manager worker.
Explicit Type Conversion
TD parameters auto-cast in expression contexts but remain TD objects. Convert with int(), float(), str() for standard Python functions. Use repr() to reveal actual type.