Developing Cross-Platform <three-live> & <xr-live> Plugins
Every <xr-live> feature is authored once and executes across Web (WebGL2 / WebXR), Android (Filament / Jetpack XR SceneCore), and iOS & visionOS (RealityKit). Follow this 5-step workflow whenever adding or modifying a plugin or userData.xr trait so the platforms never drift.
Step 1: Canonical Trait Schema (three-live/src/xr-trait-schema.ts)
three-live/src/xr-trait-schema.ts is the single source of truth for trait validation, clamping, and export serialization:
- Define the normalized TypeScript interface and a pure parser
parse<Feature>(raw: unknown) using clampNum(v, lo, hi, fallback), isFiniteVec3(v), shortString(v, max), and safeMediaUrl(v).
- Add the field to
NormalizedXRTraits and normalizeXRTraits(raw) so both web plugins and exporters see identical defaults and bounds.
- Export to glTF (
buildXRSceneCoreExtension): Map the normalized field into nodes[i].extensions.ANDROID_xr_scenecore.
- Export to USDZ (
flattenXRTraitsForUsd): Flatten nested objects into camelCase scalar/vec3 keys (<trait><Field>) so buildUsdXRLines() emits valid USDA customData = { dictionary xr = { ... } } lines without injection risk.
Step 2: Web Plugin (xr-live/src/plugins/<name>.ts)
- Implement
ThreeLivePlugin (three-live/src/plugin.ts):
name: Unique 'xr-live-<name>' identifier.
onElementConnected(element) / onElementDisconnected(element): Attach and clean up DOM event listeners (xr-select-start, xr-select-move, xr-select-end) via WeakMap state.
onConfigureXRSessionInit(element, sessionInit): Add required/optional WebXR features (e.g. 'plane-detection', 'mesh-detection', 'hand-tracking').
onXRFrame(viewer, session, frame, refSpace) & onUpdate(viewer, dt): Step per-frame simulation and dynamic meshes (createDynamicMesh, updateDynamicMesh, removeDynamicMesh, setWorldTRS from xr-live/src/plugins/common.ts).
- Register and export the plugin in
xr-live/src/plugins/index.ts.
- Bound all resources: Enforce explicit constants (
MAX_*) on any dynamic geometry, array, or network/event payload.
Step 3: Native Android Parity (mobile/android/)
Update the Android runtime so compiled .glb scenes and scripts execute the trait identically:
mobile/android/src/main/java/com/google/threelive/XRLiveSceneCoreHelper.kt: Parse the new extras.xr / ANDROID_xr_scenecore fields with the exact same clamping ranges as xr-trait-schema.ts.
- Modular Feature File
(
mobile/android/src/main/java/com/google/threelive/<FeatureName>.kt) &
XRSpatialMath.kt: Implement pure, allocation-free math and per-feature
state machines in a dedicated Kotlin file (e.g. SpatialMeasure.kt,
MagneticSnap.kt, GazeDwell.kt, ExplodedView.kt,
PersistentAnchors.kt, SpatialLocomotion.kt, SharedRoom.kt) rather than
bloating a single monolith.
mobile/android/src/main/java/com/google/threelive/XRSpatialFeatures.kt &
XRLiveRuntimeSession.kt: Wire the modular controller into
stepFrame(dtSeconds) and emit dynamic ThreeLiveMeshPrimitive geometry
(and optional sectionPlane uniforms) for ThreeLiveFilamentRenderer.kt.
Step 4: Native iOS & visionOS Parity (mobile/ios/)
Update the Swift package under mobile/ios/Sources/ThreeLiveCompiler/:
XRLiveRuntimeSession.swift & XRLiveRealityKit.swift: Parse the trait from GLB extras.xr and USDZ customData.xr.
- Modular Feature File (
<FeatureName>.swift),
XRSpatialRealityKitFeatures.swift, & XRLiveRealityKitSession.swift:
Mirror the math and dynamic entity updates in a dedicated Swift file (e.g.
SpatialMeasure.swift, MagneticSnap.swift, GazeDwell.swift,
ExplodedView.swift, PersistentAnchors.swift, SpatialLocomotion.swift,
SharedRoom.swift) orchestrated by XRSpatialRealityKitFeatures.swift.
- Parity Test Check: Android's
mobile/android/src/test/java/com/google/threelive/XRLiveSwiftSourceParityTest.kt verifies that the Swift sources implement all required symbols and trait keys even when building on Linux.
Step 5: Build, Bundle Budget & Test Verification
Rebuild Web bundles & embedded Rust engine asset:
node scripts/build.js
Verify that all 15 .min.js.gz bundles in scripts/build.js remain within
their enforced byte budgets:
three-live-viewer.min.js: < 30 KB gz
three-live.min.js: < 73 KB gz
three-live-studio.min.js: < 29 KB gz
three-live-cad.min.js: < 20 KB gz
three-live-procedural.min.js: < 28 KB gz
three-live-animation.min.js: < 11 KB gz
three-live-ai.min.js: < 20 KB gz
three-live-full.min.js: < 160 KB gz
xr-live-viewer.min.js: < 55 KB gz
xr-live.min.js: < 84 KB gz
xr-live-sensors.min.js: < 10 KB gz
xr-live-spatial.min.js: < 63 KB gz
xr-live-sim.min.js: < 30 KB gz
xr-live-webgpu.min.js: < 12 KB gz
xr-live-full.min.js: < 268 KB gz
Run Web, Rust, and Mobile tests:
node scripts/build.js --test
cd compiler && cargo test
Update llms.txt and llms-full.txt:
node scripts/build_llms_txt.js
1---2name: xr-live-plugin-dev3description: >-4 Adds or modifies <three-live> and <xr-live> plugins, userData.xr spatial5 traits, glTF ANDROID_xr_scenecore extensions, USDZ customData.xr metadata,6 and native Android (Filament / SceneCore) or iOS/visionOS (RealityKit)7 runtime features. Use when contributing new capabilities to the three-live8 monorepo or keeping Web, Rust, Android, and iOS/visionOS runtimes in parity.9metadata:10 source: https://skillmd.com/skills/google/xr-live-plugin-dev11---1213# Developing Cross-Platform `<three-live>` & `<xr-live>` Plugins1415Every `<xr-live>` feature is authored once and executes across **Web (`WebGL2` / `WebXR`)**, **Android (`Filament` / Jetpack XR `SceneCore`)**, and **iOS & visionOS (`RealityKit`)**. Follow this 5-step workflow whenever adding or modifying a plugin or `userData.xr` trait so the platforms never drift.1617---1819## Step 1: Canonical Trait Schema (`three-live/src/xr-trait-schema.ts`)2021`three-live/src/xr-trait-schema.ts` is the single source of truth for trait validation, clamping, and export serialization:22231. **Define the normalized TypeScript interface** and a pure parser `parse<Feature>(raw: unknown)` using `clampNum(v, lo, hi, fallback)`, `isFiniteVec3(v)`, `shortString(v, max)`, and `safeMediaUrl(v)`.242. **Add the field to `NormalizedXRTraits` and `normalizeXRTraits(raw)`** so both web plugins and exporters see identical defaults and bounds.253. **Export to glTF (`buildXRSceneCoreExtension`)**: Map the normalized field into `nodes[i].extensions.ANDROID_xr_scenecore`.264. **Export to USDZ (`flattenXRTraitsForUsd`)**: Flatten nested objects into camelCase scalar/vec3 keys (`<trait><Field>`) so `buildUsdXRLines()` emits valid USDA `customData = { dictionary xr = { ... } }` lines without injection risk.2728---2930## Step 2: Web Plugin (`xr-live/src/plugins/<name>.ts`)31321. Implement `ThreeLivePlugin` (`three-live/src/plugin.ts`):33 - `name`: Unique `'xr-live-<name>'` identifier.34 - `onElementConnected(element)` / `onElementDisconnected(element)`: Attach and clean up DOM event listeners (`xr-select-start`, `xr-select-move`, `xr-select-end`) via `WeakMap` state.35 - `onConfigureXRSessionInit(element, sessionInit)`: Add required/optional WebXR features (e.g. `'plane-detection'`, `'mesh-detection'`, `'hand-tracking'`).36 - `onXRFrame(viewer, session, frame, refSpace)` & `onUpdate(viewer, dt)`: Step per-frame simulation and dynamic meshes (`createDynamicMesh`, `updateDynamicMesh`, `removeDynamicMesh`, `setWorldTRS` from `xr-live/src/plugins/common.ts`).372. **Register and export** the plugin in `xr-live/src/plugins/index.ts`.383. **Bound all resources:** Enforce explicit constants (`MAX_*`) on any dynamic geometry, array, or network/event payload.3940---4142## Step 3: Native Android Parity (`mobile/android/`)4344Update the Android runtime so compiled `.glb` scenes and scripts execute the trait identically:45461. **`mobile/android/src/main/java/com/google/threelive/XRLiveSceneCoreHelper.kt`:** Parse the new `extras.xr` / `ANDROID_xr_scenecore` fields with the exact same clamping ranges as `xr-trait-schema.ts`.472. **Modular Feature File48 (`mobile/android/src/main/java/com/google/threelive/<FeatureName>.kt`) &49 `XRSpatialMath.kt`:** Implement pure, allocation-free math and per-feature50 state machines in a dedicated Kotlin file (e.g. `SpatialMeasure.kt`,51 `MagneticSnap.kt`, `GazeDwell.kt`, `ExplodedView.kt`,52 `PersistentAnchors.kt`, `SpatialLocomotion.kt`, `SharedRoom.kt`) rather than53 bloating a single monolith.543. **`mobile/android/src/main/java/com/google/threelive/XRSpatialFeatures.kt` &55 `XRLiveRuntimeSession.kt`:** Wire the modular controller into56 `stepFrame(dtSeconds)` and emit dynamic `ThreeLiveMeshPrimitive` geometry57 (and optional `sectionPlane` uniforms) for `ThreeLiveFilamentRenderer.kt`.5859---6061## Step 4: Native iOS & visionOS Parity (`mobile/ios/`)6263Update the Swift package under `mobile/ios/Sources/ThreeLiveCompiler/`:64651. **`XRLiveRuntimeSession.swift` & `XRLiveRealityKit.swift`:** Parse the trait from GLB `extras.xr` and USDZ `customData.xr`.662. **Modular Feature File (`<FeatureName>.swift`),67 `XRSpatialRealityKitFeatures.swift`, & `XRLiveRealityKitSession.swift`:**68 Mirror the math and dynamic entity updates in a dedicated Swift file (e.g.69 `SpatialMeasure.swift`, `MagneticSnap.swift`, `GazeDwell.swift`,70 `ExplodedView.swift`, `PersistentAnchors.swift`, `SpatialLocomotion.swift`,71 `SharedRoom.swift`) orchestrated by `XRSpatialRealityKitFeatures.swift`.723. **Parity Test Check:** Android's `mobile/android/src/test/java/com/google/threelive/XRLiveSwiftSourceParityTest.kt` verifies that the Swift sources implement all required symbols and trait keys even when building on Linux.7374---7576## Step 5: Build, Bundle Budget & Test Verification77781. **Rebuild Web bundles & embedded Rust engine asset:**7980 ```bash81 node scripts/build.js82 ```8384 Verify that all 15 `.min.js.gz` bundles in `scripts/build.js` remain within85 their enforced byte budgets:8687 - `three-live-viewer.min.js`: `< 30 KB gz`88 - `three-live.min.js`: `< 73 KB gz`89 - `three-live-studio.min.js`: `< 29 KB gz`90 - `three-live-cad.min.js`: `< 20 KB gz`91 - `three-live-procedural.min.js`: `< 28 KB gz`92 - `three-live-animation.min.js`: `< 11 KB gz`93 - `three-live-ai.min.js`: `< 20 KB gz`94 - `three-live-full.min.js`: `< 160 KB gz`95 - `xr-live-viewer.min.js`: `< 55 KB gz`96 - `xr-live.min.js`: `< 84 KB gz`97 - `xr-live-sensors.min.js`: `< 10 KB gz`98 - `xr-live-spatial.min.js`: `< 63 KB gz`99 - `xr-live-sim.min.js`: `< 30 KB gz`100 - `xr-live-webgpu.min.js`: `< 12 KB gz`101 - `xr-live-full.min.js`: `< 268 KB gz`1022. **Run Web, Rust, and Mobile tests:**103104 ```bash105 node scripts/build.js --test106 cd compiler && cargo test107 ```1083. **Update `llms.txt` and `llms-full.txt`:**109110 ```bash111 node scripts/build_llms_txt.js112 ```