Scenario Runway Video
Overview
Runway's family on Scenario splits by direction: Gen4.5 generates new clips from text or a
first-frame image; Aleph 2 rewrites existing footage, changing what the prompt names while motion,
audio, and structure carry over. Discover both with search; model_schema_get is the contract.
Connection and the core loop: the scenario skill; model-agnostic video work: the scenario-video
skill. If a sibling skill named here is missing from your available skills, load it with skill_get
(Scenario MCP) or ask the user to install it (https://github.com/scenario-labs/skills#install);
unattended, proceed from tool schemas and flag the gap.
Quick reference
Mode follows from the inputs (names and caps from the live schema):
| Mode |
Inputs |
Behavior |
| Text (Gen4.5) |
prompt |
aspectRatio honored; duration 2 to 10 seconds |
| First frame (Gen4.5) |
image + prompt |
opens on that frame; same duration; aspectRatio ignored |
| Edit (Aleph 2) |
video + prompt |
rewrites visuals; motion, audio, structure persist |
At authoring time Aleph capped sources at 30 seconds and 16 MB, separate limits: high-bitrate clips
hit the size cap first. Its median cost ran six times Gen4.5's, a fifteen-fold spread following the
source video: dry_run the exact payload first. Median latency neared six minutes, so jobs_wait
will time out mid-job; re-call it with pending_job_ids.
Gen4.5 prompts follow the mode
Text mode needs both the visuals and the motion described. A first frame fixes subject, composition,
and palette: spend the whole prompt on motion, camera, and atmosphere. Order beats explicitly
("kneels, then rises, camera cranes up") and give multi-beat prompts 8 to 10 seconds of duration.
Name camera moves plainly (locked camera, pan, dolly, tracking, orbit, crane), one or two per clip,
in present-tense prose.
aspectRatio takes pixel pairs ("1280:720", "960:960"), and text mode honored only the 16:9 and
9:16 pairs at authoring time; get other shapes (1:1, 4:3, 3:4, 21:9) from a first frame with that
geometry instead.
Aleph changes what you name
An Aleph prompt states the edit: remove, swap, restyle, relight. Everything unnamed persists, along
with motion and audio: Aleph cannot re-choreograph or add camera moves; when motion must change,
generate with Gen4.5. To steer the edit, pin up to 5 reference images: keyframes places each image
(uri) by seconds or at (a 0 to 1 fraction); promptImage by positionType plus
timestampSeconds or positionPercentage. The arrays are alternatives, never combined.
Worked example: swap a product in real footage
search with target="models", query="runway", public=true, e.g. model_runway-aleph-2 for
editing, model_runway-gen4-5 for generation (live hits at authoring time: re-discover each
session).
model_schema_get with the Aleph id before anything else.
- Trim or compress a source over the caps (see
scenario-video); upload_asset the clip and
product still (see the scenario skill).
model_run with dry_run=true and the exact
parameters={"prompt": "Replace the soda can with the bottle from the reference image. Keep everything else unchanged.", "video": "asset_clip", "keyframes": [{"uri": "asset_bottle", "at": 0}]};
re-estimate after any trim.
- Repeat with
wait=false, then jobs_wait the returned job id, re-calling with
pending_job_ids; a timeout is not a failure and never justifies a second model_run.
asset_display the output; check the swap holds in every shot.
Common mistakes
- Sending both
keyframes and promptImage: exclusive alternatives; pick one.
- Restating the first frame in an image-mode prompt: prompt motion and camera instead.
- Passing
aspectRatio as "16:9" or alongside image: pixel pairs only, and a first frame
overrides it.
- Asking Aleph for new motion or camera moves: the source's motion persists by design.
1---2name: scenario-runway3description: "Use when generating or editing video with Runway models on Scenario via MCP: text-to-video or image-to-video from a first frame with Gen4.5 (cinematic motion, sequenced actions, camera choreography), or video-to-video editing of real footage with Aleph 2 (remove objects, swap products, restyle or relight shots, keyframe-pinned reference images, motion and audio preserved). Keywords: Runway ML, Gen4.5, Gen-4.5, Aleph, T2V, I2V, V2V, first frame."4license: MIT5metadata:6 source: https://skillmd.com/skills/scenario-labs/scenario-runway7---89# Scenario Runway Video1011## Overview1213Runway's family on Scenario splits by direction: Gen4.5 generates new clips from text or a14first-frame image; Aleph 2 rewrites existing footage, changing what the prompt names while motion,15audio, and structure carry over. Discover both with `search`; `model_schema_get` is the contract.1617Connection and the core loop: the `scenario` skill; model-agnostic video work: the `scenario-video`18skill. If a sibling skill named here is missing from your available skills, load it with `skill_get`19(Scenario MCP) or ask the user to install it (https://github.com/scenario-labs/skills#install);20unattended, proceed from tool schemas and flag the gap.2122## Quick reference2324Mode follows from the inputs (names and caps from the live schema):2526| Mode | Inputs | Behavior |27| -------------------- | ------------------ | ----------------------------------------------------------- |28| Text (Gen4.5) | `prompt` | `aspectRatio` honored; `duration` 2 to 10 seconds |29| First frame (Gen4.5) | `image` + `prompt` | opens on that frame; same `duration`; `aspectRatio` ignored |30| Edit (Aleph 2) | `video` + `prompt` | rewrites visuals; motion, audio, structure persist |3132At authoring time Aleph capped sources at 30 seconds and 16 MB, separate limits: high-bitrate clips33hit the size cap first. Its median cost ran six times Gen4.5's, a fifteen-fold spread following the34source video: `dry_run` the exact payload first. Median latency neared six minutes, so `jobs_wait`35will time out mid-job; re-call it with `pending_job_ids`.3637## Gen4.5 prompts follow the mode3839Text mode needs both the visuals and the motion described. A first frame fixes subject, composition,40and palette: spend the whole prompt on motion, camera, and atmosphere. Order beats explicitly41("kneels, then rises, camera cranes up") and give multi-beat prompts 8 to 10 seconds of `duration`.42Name camera moves plainly (locked camera, pan, dolly, tracking, orbit, crane), one or two per clip,43in present-tense prose.4445`aspectRatio` takes pixel pairs (`"1280:720"`, `"960:960"`), and text mode honored only the 16:9 and469:16 pairs at authoring time; get other shapes (1:1, 4:3, 3:4, 21:9) from a first frame with that47geometry instead.4849## Aleph changes what you name5051An Aleph prompt states the edit: remove, swap, restyle, relight. Everything unnamed persists, along52with motion and audio: Aleph cannot re-choreograph or add camera moves; when motion must change,53generate with Gen4.5. To steer the edit, pin up to 5 reference images: `keyframes` places each image54(`uri`) by `seconds` or `at` (a 0 to 1 fraction); `promptImage` by `positionType` plus55`timestampSeconds` or `positionPercentage`. The arrays are alternatives, never combined.5657## Worked example: swap a product in real footage58591. `search` with `target="models"`, `query="runway"`, `public=true`, e.g. `model_runway-aleph-2` for60 editing, `model_runway-gen4-5` for generation (live hits at authoring time: re-discover each61 session).622. `model_schema_get` with the Aleph id before anything else.633. Trim or compress a source over the caps (see `scenario-video`); `upload_asset` the clip and64 product still (see the `scenario` skill).654. `model_run` with `dry_run=true` and the exact66 `parameters={"prompt": "Replace the soda can with the bottle from the reference image. Keep everything else unchanged.", "video": "asset_clip", "keyframes": [{"uri": "asset_bottle", "at": 0}]}`;67 re-estimate after any trim.685. Repeat with `wait=false`, then `jobs_wait` the returned job id, re-calling with69 `pending_job_ids`; a timeout is not a failure and never justifies a second `model_run`.706. `asset_display` the output; check the swap holds in every shot.7172## Common mistakes7374- Sending both `keyframes` and `promptImage`: exclusive alternatives; pick one.75- Restating the first frame in an image-mode prompt: prompt motion and camera instead.76- Passing `aspectRatio` as `"16:9"` or alongside `image`: pixel pairs only, and a first frame77 overrides it.78- Asking Aleph for new motion or camera moves: the source's motion persists by design.