Scene3D Interactions
Make a composed scene behave: motion, input, picking, and game loops. Exact
signatures live in references/script-api.md;
ready-made patterns in references/patterns.md.
Announce at start: "I'm using the scene3d-interactions skill to wire the
scene behavior."
The three control layers — pick deliberately
- Per-frame
script — continuous motion that lives entirely inside the
scene (spin, bob, follow-pointer). It must be a pure expression (an
object literal or IIFE) — expressions run every frame on the worklet
thread with time, dt, objects, camera, variables, keys, pointer in scope
and return a mutation map. A statement-form script instead runs ONCE
at mount (setup only) — the #1 authoring mistake is writing statements and
expecting per-frame execution.
eventScript — react to keys/pointer without app round-trips.
Enable via eventScriptEvents (keys/pointer/all), optionally
allowlist keys with eventScriptKeys, and use key-repeat props for
hold-to-move. Receives event/input plus the scene scope; returns the
same mutation-map shape.- App-level control — the scene talks to the rest of the app:
- Out: brick events (
ON_OBJECT_CLICK, ON_ANIMATION_END,
ON_FRAME, ON_LOAD/_ERROR) → BRICKS event handlers → Property Bank
state, navigation, sounds.
- In: brick actions (
BRICK_SCENE_3D_*) triggered by events, or the
scriptMember functions called from Data Calculations (the built-in
bricks-ctor skill documents the data-calc sandbox and how to call
brick members). Feed dynamic values into the per-frame script via the
brick's variables property — updates flow in on the next frame.
Rule of thumb: motion → layer 1; twitch input → layer 2; anything that
changes app state (score, selection, navigation) → layer 3. Keep game state
either in __variables (scene-internal) or the Property Bank (app-visible) —
never split one piece of state across both.
Mutation map essentials
// Per-frame expression: spin one object, remember elapsed time
{
spinner: { rotation: { x: 0, y: time * 0.8, z: 0 } },
__camera: { position: { x: 3, y: 3, z: 5 } }, // optional camera drive
__variables: { t: (variables.t || 0) + dt } // persists across frames
}
- Keys are object ids from the scene declaration, plus
__camera and
__variables.
keys is a map of currently-held keys; pointer is
{ x, y, down, type, touchCount }.- Wire
BRICK_SCENE_3D_ON_SCRIPT_ERROR to a visible dev-time indicator —
script failures are otherwise silent stillness.
- Native
frameRuntime (ui vs js) tunes which thread evaluates the
expression; leave default unless profiling says otherwise.
Picking
Set enableRaycast: true → clicks/taps emit ON_OBJECT_CLICK with the
object id (hover works on pointer devices only — never make hover the only
path on touch kiosks). For custom hit-testing (e.g. from a screen overlay),
the raycast(x, y) script member returns { objectId, point, distance } or
null.
No physics engine
There are no rigid bodies or collisions. Fake what the experience needs:
distance checks between object positions (sphere-vs-sphere), clamped
positions for walls, manual gravity (vy += g*dt) for arcs. Keep gameplay in
the "kiosk game" class — catchers, whack-a-mole, spinners, quizzes — and it
will feel right; don't attempt physics-heavy genres.
Performance budget
Per-frame scripts run every frame on the device: keep the expression small
(no allocation-heavy loops over hundreds of objects), prefer playAnimation
clips over scripted joint motion, throttle ON_FRAME events
(frameEventInterval ≥ 250 ms — they cross into app event handling), and
test on the weakest target device. screenshot is web/desktop only (native
returns an empty URI) — don't build features on it for device fleets.
Verify
Simulator first: walk every interaction (click each pickable object, hold
keys, idle for the attract timer) and watch for script errors. Then on
device: touch behavior (tap vs drag on orbit controls), frame rate during the
busiest moment, and input latency. The bricks-cli skill covers deploy and
on-device inspection.
When Not To Use
- Static scene composition (objects/lights/camera) —
scene3d-authoring.
- Asset creation/optimization —
blender-pipeline.
- Physics-dependent gameplay — out of scope; redesign kinematically or use a
different approach.
1---2name: scene3d-interactions3description: Use when making a Scene3D brick interactive — spin/inspect product viewers, click-to-explore demos, kiosk attract loops, or simple 3D mini-games. Covers the per-frame script (a pure expression evaluated every frame with time, dt, objects, camera, variables, keys, pointer in scope, returning a mutation map; statement-form scripts run once at mount), eventScript for key and pointer handling, raycast picking wired to BRICKS events and actions, and the imperative scriptMember API (addObject, removeObject, updateObject, setCamera, lookAt, playAnimation, stopAnimation, setBackground, setControls, screenshot, getSceneState, raycast). Includes movement and kinematics patterns — there is no physics engine, so no collisions or rigid bodies — plus performance budgets and connecting scene events to app state in the Property Bank. Triggers on "make it interactive", "clickable 3D objects", "rotate on touch", "simple 3D game", "attract loop". For static scene setup use scene3d-authoring.4license: MIT5---67# Scene3D Interactions89Make a composed scene behave: motion, input, picking, and game loops. Exact10signatures live in [references/script-api.md](references/script-api.md);11ready-made patterns in [references/patterns.md](references/patterns.md).1213**Announce at start:** "I'm using the scene3d-interactions skill to wire the14scene behavior."1516## The three control layers — pick deliberately17181. **Per-frame `script`** — continuous motion that lives entirely inside the19 scene (spin, bob, follow-pointer). It must be a **pure expression** (an20 object literal or IIFE) — expressions run every frame on the worklet21 thread with `time, dt, objects, camera, variables, keys, pointer` in scope22 and return a **mutation map**. A statement-form script instead runs ONCE23 at mount (setup only) — the #1 authoring mistake is writing statements and24 expecting per-frame execution.252. **`eventScript`** — react to keys/pointer without app round-trips.26 Enable via `eventScriptEvents` (`keys`/`pointer`/`all`), optionally27 allowlist keys with `eventScriptKeys`, and use key-repeat props for28 hold-to-move. Receives `event`/`input` plus the scene scope; returns the29 same mutation-map shape.303. **App-level control** — the scene talks to the rest of the app:31 - **Out:** brick events (`ON_OBJECT_CLICK`, `ON_ANIMATION_END`,32 `ON_FRAME`, `ON_LOAD/_ERROR`) → BRICKS event handlers → Property Bank33 state, navigation, sounds.34 - **In:** brick actions (`BRICK_SCENE_3D_*`) triggered by events, or the35 scriptMember functions called from Data Calculations (the built-in36 `bricks-ctor` skill documents the data-calc sandbox and how to call37 brick members). Feed dynamic values into the per-frame script via the38 brick's `variables` property — updates flow in on the next frame.3940Rule of thumb: motion → layer 1; twitch input → layer 2; anything that41changes app state (score, selection, navigation) → layer 3. Keep game state42either in `__variables` (scene-internal) or the Property Bank (app-visible) —43never split one piece of state across both.4445## Mutation map essentials4647```js48// Per-frame expression: spin one object, remember elapsed time49{50 spinner: { rotation: { x: 0, y: time * 0.8, z: 0 } },51 __camera: { position: { x: 3, y: 3, z: 5 } }, // optional camera drive52 __variables: { t: (variables.t || 0) + dt } // persists across frames53}54```5556- Keys are object **ids** from the scene declaration, plus `__camera` and57 `__variables`.58- `keys` is a map of currently-held keys; `pointer` is59 `{ x, y, down, type, touchCount }`.60- Wire `BRICK_SCENE_3D_ON_SCRIPT_ERROR` to a visible dev-time indicator —61 script failures are otherwise silent stillness.62- Native `frameRuntime` (`ui` vs `js`) tunes which thread evaluates the63 expression; leave default unless profiling says otherwise.6465## Picking6667Set `enableRaycast: true` → clicks/taps emit `ON_OBJECT_CLICK` with the68object id (hover works on pointer devices only — never make hover the only69path on touch kiosks). For custom hit-testing (e.g. from a screen overlay),70the `raycast(x, y)` script member returns `{ objectId, point, distance }` or71null.7273## No physics engine7475There are no rigid bodies or collisions. Fake what the experience needs:76distance checks between object positions (sphere-vs-sphere), clamped77positions for walls, manual gravity (`vy += g*dt`) for arcs. Keep gameplay in78the "kiosk game" class — catchers, whack-a-mole, spinners, quizzes — and it79will feel right; don't attempt physics-heavy genres.8081## Performance budget8283Per-frame scripts run every frame on the device: keep the expression small84(no allocation-heavy loops over hundreds of objects), prefer `playAnimation`85clips over scripted joint motion, throttle `ON_FRAME` events86(`frameEventInterval` ≥ 250 ms — they cross into app event handling), and87test on the weakest target device. `screenshot` is web/desktop only (native88returns an empty URI) — don't build features on it for device fleets.8990## Verify9192Simulator first: walk every interaction (click each pickable object, hold93keys, idle for the attract timer) and watch for script errors. Then on94device: touch behavior (tap vs drag on orbit controls), frame rate during the95busiest moment, and input latency. The `bricks-cli` skill covers deploy and96on-device inspection.9798## When Not To Use99100- Static scene composition (objects/lights/camera) — `scene3d-authoring`.101- Asset creation/optimization — `blender-pipeline`.102- Physics-dependent gameplay — out of scope; redesign kinematically or use a103 different approach.
mybigday