Scenario Video to Motion
Overview
Video-to-motion members watch the people in a clip and return 3D animation: either your character
mesh, auto-rigged and moving, or the motion alone on a standard skeleton. They are the video23d
capability, and they differ more than their shared name suggests: which character input they take,
which skeleton comes back, and which file formats they write.
Siblings: scenario (connection, scope, upload, jobs_wait), scenario-3d (making or rigging the
mesh first), scenario-video (generating a reference performance when no footage exists), and
scenario-video-editing (trimming and resizing the clip). If a sibling skill named here is missing
from your available skills, load it with skill_get (Scenario MCP) or ask the user to install it
(https://github.com/scenario-labs/skills#install); unattended, proceed from tool schemas and flag
the gap.
Quick reference
| Step |
Call |
Watch for |
| Upload inputs |
upload_asset with kind: "video", then kind: "3d" for a mesh |
The mesh is optional on every member |
| Fit the clip |
asset_get, then trim or resize per scenario-video-editing |
Duration, size, and even-dimension limits per member |
| Pick a member |
recommend with capability: "video23d" and the user's own words |
Read next_step per the scenario skill |
| Read the inputs |
model_schema_get |
Each file field's kind, the output enums |
| Price, then run |
model_run with dry_run: true, then wait: false |
The video field carries cost_impact |
| Wait and inspect |
jobs_wait, then asset_display |
Facing, foot contact, and frame count |
Choose the output before the member
Three outcomes, and members cover different ones. Settle it with the user first (unattended: a
character whose existing rig the project depends on takes motion only; otherwise animate the mesh).
- Your character, animated. A member takes the mesh, auto-rigs it, and applies the motion. The
rig is the member's own: it does not retarget onto a skeleton the mesh already carries, so clips
built for an existing rig need bone remapping.
- Motion only, on a named skeleton. A member returns the animation on a Mixamo-style or
MetaHuman skeleton without uploading your character. Choose this when the character is already
rigged. The file can include a template mesh: inspect it, then omit that mesh when retargeting
onto the user's rig.
- Motion only, from a mesh member. Some mesh members also offer an animation-only flag, which
drops the mesh from the download.
recommend ranks the members on measured cost and latency and names which take a character. Ask it
with the user's words, including the outcome above, rather than with a bare capability.
Read the schema, field by field
Field names repeat across members with different meanings, so read each field's kind before
filling it:
- One member's
characterFile takes a 3D mesh; another member's characterFile takes a reference
image of the character, while its mesh field is characterModel. The field's kind (3d or
image), not its name, says which asset it takes.
- Output formats differ: one member writes engine-specific FBX variants (
fbx-blender,
fbx-unreal, fbx-roblox, Maya scene files), one a plain glb or fbx (its schema names fbx
for Unity), and one exposes no format field at all. Pick the engine target in the run:
asset_download's format conversion is not documented to keep the animation.
- An FBX export can be ASCII. Blender's legacy FBX importer rejects it; use its newer ASCII-capable
importer. Check the imported animation before calling the file Blender-ready.
- Controls worth setting when present:
moveInPlace for game locomotion driven by code, loop for
cycles, the frame rate field (named frameRate, fps, or targetFps), keyframe cleanup, and the
forward and up axes when the motion arrives turned or lying down.
- Some members track several people (
numPeople) or capture the face; both carry cost_impact.
Footage that captures well
- One person, full body in frame for the whole clip, feet visible, a static camera, no one crossing
in front. Loose clothing and props hide joints.
- Trim to the move that ships: price follows the clip, and members cap the input. At authoring time
one capped it at 30 seconds and about 30 MB, another required even pixel width and height. Read
max_duration and max_size when exposed by the schema. An absent cap is not a promise of
unlimited input. Fix an over-cap clip with the video editing tools before pricing.
- No footage? Generate the performance with a video model per
scenario-video: a single full-body
character on a plain background, a locked-off camera, and the action described beat by beat. The
generated clip is then the input like any other.
- Motion extraction targets human movement. A creature or a non-biped mesh needs checking frame by
frame, and often a DCC pass.
Worked example: a dance clip on a game character
The user has a phone clip of a dance and a character GLB, and wants an Unreal-ready loop.
upload_asset the clip (kind: "video") and the GLB (kind: "3d") per the scenario skill.
asset_get the clip: read the duration and dimensions. Trim to the eight seconds that loop with
the video cut tool, and resize to even dimensions if a candidate member requires it.
recommend with capability: "video23d" and a prompt such as "apply the dance in this video to
my uploaded character mesh, looping, exported for Unreal". Follow next_step per the scenario
skill: present ask_user options; on proceed, take its pick without asking.
model_schema_get on the pick. Put the GLB in the field whose kind is 3d. Set the Unreal
output target if the format enum has one, loop: true, and moveInPlace: true if the game moves
the character itself.
model_run with dry_run: true and quote the price. Then run with wait: false: these jobs
take minutes. Retire it with jobs_wait, re-calling with pending_job_ids on timeout.
asset_display the result, or open it in a DCC. Check the facing, that the feet meet the ground,
and that the loop point does not pop. If facing is wrong, check forward and up when the
schema exposes them before blaming the capture. A preview link alone is not a visual check.
asset_download with no format, and hand over the file with the skeleton it uses.
Common mistakes
- Expecting the motion on the character's existing rig: mesh members auto-rig from scratch. Use a
motion-only member and retarget when the rig must stay.
- Filling a field by its name:
characterFile is a mesh on one member and an image on another.
- Sending the full source video: the price grows with the clip, and over-cap clips fail.
- Converting the download to another 3D format instead of setting the export target in the run.
- Treating a sideways or upside-down result as a failed capture before trying the axis fields.
- Polling
job_get instead of jobs_wait, or re-running a job that only timed out.
1---2name: scenario-video-to-motion3description: "Use when turning a reference video into 3D character animation through the Scenario MCP: markerless motion capture from a phone clip or a generated performance, auto-rigging an uploaded FBX, GLB, or OBJ character and applying the captured motion, raw motion on a Mixamo or MetaHuman skeleton to retarget yourself, in-place locomotion, looping clips, facial capture, or FBX exports for Blender, Maya, Unreal, Unity, or Roblox. Keywords: mocap, video to motion, video23d, animation from video."4license: MIT5metadata:6 source: https://skillmd.com/skills/scenario-labs/scenario-video-to-motion7---89# Scenario Video to Motion1011## Overview1213Video-to-motion members watch the people in a clip and return 3D animation: either your character14mesh, auto-rigged and moving, or the motion alone on a standard skeleton. They are the `video23d`15capability, and they differ more than their shared name suggests: which character input they take,16which skeleton comes back, and which file formats they write.1718Siblings: `scenario` (connection, scope, upload, `jobs_wait`), `scenario-3d` (making or rigging the19mesh first), `scenario-video` (generating a reference performance when no footage exists), and20`scenario-video-editing` (trimming and resizing the clip). If a sibling skill named here is missing21from your available skills, load it with `skill_get` (Scenario MCP) or ask the user to install it22(https://github.com/scenario-labs/skills#install); unattended, proceed from tool schemas and flag23the gap.2425## Quick reference2627| Step | Call | Watch for |28| ---------------- | ------------------------------------------------------------------ | ---------------------------------------------------- |29| Upload inputs | `upload_asset` with `kind: "video"`, then `kind: "3d"` for a mesh | The mesh is optional on every member |30| Fit the clip | `asset_get`, then trim or resize per `scenario-video-editing` | Duration, size, and even-dimension limits per member |31| Pick a member | `recommend` with `capability: "video23d"` and the user's own words | Read `next_step` per the `scenario` skill |32| Read the inputs | `model_schema_get` | Each file field's `kind`, the output enums |33| Price, then run | `model_run` with `dry_run: true`, then `wait: false` | The `video` field carries `cost_impact` |34| Wait and inspect | `jobs_wait`, then `asset_display` | Facing, foot contact, and frame count |3536## Choose the output before the member3738Three outcomes, and members cover different ones. Settle it with the user first (unattended: a39character whose existing rig the project depends on takes motion only; otherwise animate the mesh).4041- **Your character, animated.** A member takes the mesh, auto-rigs it, and applies the motion. The42 rig is the member's own: it does not retarget onto a skeleton the mesh already carries, so clips43 built for an existing rig need bone remapping.44- **Motion only, on a named skeleton.** A member returns the animation on a Mixamo-style or45 MetaHuman skeleton without uploading your character. Choose this when the character is already46 rigged. The file can include a template mesh: inspect it, then omit that mesh when retargeting47 onto the user's rig.48- **Motion only, from a mesh member.** Some mesh members also offer an animation-only flag, which49 drops the mesh from the download.5051`recommend` ranks the members on measured cost and latency and names which take a character. Ask it52with the user's words, including the outcome above, rather than with a bare capability.5354## Read the schema, field by field5556Field names repeat across members with different meanings, so read each field's `kind` before57filling it:5859- One member's `characterFile` takes a 3D mesh; another member's `characterFile` takes a reference60 image of the character, while its mesh field is `characterModel`. The field's `kind` (`3d` or61 `image`), not its name, says which asset it takes.62- Output formats differ: one member writes engine-specific FBX variants (`fbx-blender`,63 `fbx-unreal`, `fbx-roblox`, Maya scene files), one a plain `glb` or `fbx` (its schema names `fbx`64 for Unity), and one exposes no format field at all. Pick the engine target in the run:65 `asset_download`'s `format` conversion is not documented to keep the animation.66- An FBX export can be ASCII. Blender's legacy FBX importer rejects it; use its newer ASCII-capable67 importer. Check the imported animation before calling the file Blender-ready.68- Controls worth setting when present: `moveInPlace` for game locomotion driven by code, `loop` for69 cycles, the frame rate field (named `frameRate`, `fps`, or `targetFps`), keyframe cleanup, and the70 `forward` and `up` axes when the motion arrives turned or lying down.71- Some members track several people (`numPeople`) or capture the face; both carry `cost_impact`.7273## Footage that captures well7475- One person, full body in frame for the whole clip, feet visible, a static camera, no one crossing76 in front. Loose clothing and props hide joints.77- Trim to the move that ships: price follows the clip, and members cap the input. At authoring time78 one capped it at 30 seconds and about 30 MB, another required even pixel width and height. Read79 `max_duration` and `max_size` when exposed by the schema. An absent cap is not a promise of80 unlimited input. Fix an over-cap clip with the video editing tools before pricing.81- No footage? Generate the performance with a video model per `scenario-video`: a single full-body82 character on a plain background, a locked-off camera, and the action described beat by beat. The83 generated clip is then the input like any other.84- Motion extraction targets human movement. A creature or a non-biped mesh needs checking frame by85 frame, and often a DCC pass.8687## Worked example: a dance clip on a game character8889The user has a phone clip of a dance and a character GLB, and wants an Unreal-ready loop.90911. `upload_asset` the clip (`kind: "video"`) and the GLB (`kind: "3d"`) per the `scenario` skill.922. `asset_get` the clip: read the duration and dimensions. Trim to the eight seconds that loop with93 the video cut tool, and resize to even dimensions if a candidate member requires it.943. `recommend` with `capability: "video23d"` and a prompt such as "apply the dance in this video to95 my uploaded character mesh, looping, exported for Unreal". Follow `next_step` per the `scenario`96 skill: present `ask_user` options; on `proceed`, take its pick without asking.974. `model_schema_get` on the pick. Put the GLB in the field whose `kind` is `3d`. Set the Unreal98 output target if the format enum has one, `loop: true`, and `moveInPlace: true` if the game moves99 the character itself.1005. `model_run` with `dry_run: true` and quote the price. Then run with `wait: false`: these jobs101 take minutes. Retire it with `jobs_wait`, re-calling with `pending_job_ids` on timeout.1026. `asset_display` the result, or open it in a DCC. Check the facing, that the feet meet the ground,103 and that the loop point does not pop. If facing is wrong, check `forward` and `up` when the104 schema exposes them before blaming the capture. A preview link alone is not a visual check.1057. `asset_download` with no `format`, and hand over the file with the skeleton it uses.106107## Common mistakes108109- Expecting the motion on the character's existing rig: mesh members auto-rig from scratch. Use a110 motion-only member and retarget when the rig must stay.111- Filling a field by its name: `characterFile` is a mesh on one member and an image on another.112- Sending the full source video: the price grows with the clip, and over-cap clips fail.113- Converting the download to another 3D format instead of setting the export target in the run.114- Treating a sideways or upside-down result as a failed capture before trying the axis fields.115- Polling `job_get` instead of `jobs_wait`, or re-running a job that only timed out.