grok-imagine-video — video generation, editing, and extension via xAI
Calls xAI's video API (grok-imagine-video). Five modes, all in scripts/video.py:
| Mode |
When to use |
CLI |
| text-to-video |
Pure prompt → video |
--prompt "..." |
| image-to-video |
Animate a still image (image becomes the first frame) |
--prompt "..." --image PATH_OR_URL |
| reference-to-video |
Make a video featuring people / clothing / objects from reference images (no first-frame lock) |
--prompt "..." --reference-image URL [URL ...] |
| edit-video |
Modify an existing video while preserving the rest |
--prompt "..." --edit-video URL |
| extend-video |
Continue a video from its last frame |
--prompt "..." --extend-video URL |
For full parameters, status flow, and gotchas, see references/api-reference.md. For Python recipes (xai-sdk, manual polling, async batches), see references/examples.md.
Prerequisites
XAI_API_KEY exported in the environment.- Python 3.9+ with stdlib only — no SDK required (the script uses
urllib). If the user prefers the official SDK, pip install xai-sdk and use the snippets in references/examples.md.
Quickstart
# Text-to-video
python scripts/video.py \
--prompt "Glowing crystal-powered rocket launching from red Mars dunes, ancient alien ruins lighting up" \
--duration 10 --aspect-ratio 16:9 --resolution 720p \
--output rocket.mp4
# Image-to-video (local file → animated)
python scripts/video.py \
--prompt "Camera pushes in slowly. The leaves rustle in a gentle breeze." \
--image ./still.jpg \
--output animated.mp4
# Reference-to-video (virtual try-on / character consistency)
python scripts/video.py \
--prompt "The model from <IMAGE_1> walks the runway wearing the shirt from <IMAGE_2>, slow motion, dramatic lighting" \
--reference-image https://.../model.jpg https://.../shirt.jpg \
--duration 10 --aspect-ratio 16:9 --resolution 720p \
--output runway.mp4
# Edit an existing video (preserves duration / aspect / resolution)
python scripts/video.py \
--prompt "Give the woman a silver necklace" \
--edit-video https://.../source.mp4 \
--output edited.mp4
# Extend an existing video (duration = length of NEW portion only)
python scripts/video.py \
--prompt "The cat notices a butterfly and leaps off the windowsill" \
--extend-video https://.../cat.mp4 --duration 6 \
--output extended.mp4
The script polls until status=done, downloads the resulting MP4 to --output, and prints both the local path and the source URL on stderr.
Decision rules
When the user wants a video, route by which inputs they have:
- No inputs, just a description → text-to-video.
- One image, "make this move" → image-to-video. The image becomes the first frame; aspect ratio defaults to the image's unless overridden.
- One or more images, "use these people/clothes/objects" → reference-to-video. Refer to images in the prompt as
<IMAGE_1>, <IMAGE_2>, etc.
- An existing video, "change X about it" → edit-video. Duration / aspect / resolution are inherited and cannot be overridden (capped at 720p).
- An existing video, "show what happens next" → extend-video.
--duration is the length of the extension only; output total = original + extension.
Mutually exclusive — don't combine --image with --reference-image, and don't pass --edit-video with --extend-video. The script enforces this.
Parameters cheat sheet
| Flag |
Values |
Default |
Notes |
--prompt |
str |
required |
Use <IMAGE_1>, <IMAGE_2> markers when referencing images |
--duration |
int 1–15 (seconds) |
model default |
For extend, this is the added length only. Edit mode ignores. |
--aspect-ratio |
1:1, 16:9, 9:16, 4:3, 3:4, 3:2, 2:3 |
16:9 |
Edit mode ignores; image-to-video defaults to image's ratio |
--resolution |
480p, 720p |
480p |
Edit mode ignores (caps at 720p) |
--image |
path or https:// URL or data: URI |
— |
Image-to-video. Local files auto-encoded as base64 data URIs. |
--reference-image |
one or more URLs / paths |
— |
Reference-to-video. Local files auto-encoded. |
--edit-video |
URL |
— |
Edit-video mode. Must be an HTTPS URL (xAI-hosted is fine). |
--extend-video |
URL |
— |
Extend-video mode. Source must be 2–15 s. |
--poll-interval |
seconds |
5 |
How often to check status |
--poll-timeout |
seconds |
900 (15 min) |
Bail out if not done by then |
--output |
path |
required |
Where to save the downloaded MP4 |
--keep-url |
flag |
off |
Skip the download, only print the xAI-hosted URL |
Things to know
- Asynchronous. Generation typically takes a couple minutes; long / 720p / edit jobs take longer. The script handles polling — don't introduce your own loop on top.
- URLs are ephemeral. xAI returns a temporary hosted MP4 URL. The script downloads it immediately. Pass
--keep-url only if you'll fetch within minutes.
- Edit mode is constrained. Editing inherits the source video's duration, aspect, and resolution; passing those flags is silently ignored by the API. Editing input is also capped at 8.7 seconds.
- Extend mode bookkeeping.
--duration is the length of the new portion. A 10s source + --duration 5 → 15s output.
- Mode collisions error out 400. Don't combine
--image and --reference-image, and don't combine --edit-video and --extend-video.
- Moderation. Output may be filtered. The response carries a
respect_moderation flag; if the model declines the prompt, rewrite rather than retry verbatim.
- Status enum.
pending → keep polling, done → download, expired → request lifecycle ended (re-submit), failed → permanent failure (rewrite prompt or check inputs).
When to escalate
- Need still images, not video → use the sibling
grok-image or gpt-image-2 skills.
- Need precise editing of specific regions / masks → not supported here; use
gpt-image-2 for image masking and consider re-animating.
- Need >15 s of fully novel content → chain extends (each adds up to 10s) or generate multiple clips and stitch them externally.
1---2name: grok-imagine-video3description: Generate, edit, and extend videos with xAI's grok-imagine-video model. Use when the user wants to create a video from text, animate a still image, build a reference-driven video (virtual try-on, character consistency), edit an existing video, or extend one. Asynchronous — the script handles polling.4---56# grok-imagine-video — video generation, editing, and extension via xAI78Calls xAI's video API (`grok-imagine-video`). Five modes, all in `scripts/video.py`:910| Mode | When to use | CLI |11|------|-------------|-----|12| **text-to-video** | Pure prompt → video | `--prompt "..."` |13| **image-to-video** | Animate a still image (image becomes the first frame) | `--prompt "..." --image PATH_OR_URL` |14| **reference-to-video** | Make a video featuring people / clothing / objects from reference images (no first-frame lock) | `--prompt "..." --reference-image URL [URL ...]` |15| **edit-video** | Modify an existing video while preserving the rest | `--prompt "..." --edit-video URL` |16| **extend-video** | Continue a video from its last frame | `--prompt "..." --extend-video URL` |1718For full parameters, status flow, and gotchas, see `references/api-reference.md`. For Python recipes (xai-sdk, manual polling, async batches), see `references/examples.md`.1920## Prerequisites21221. `XAI_API_KEY` exported in the environment.232. Python 3.9+ with stdlib only — no SDK required (the script uses `urllib`). If the user prefers the official SDK, `pip install xai-sdk` and use the snippets in `references/examples.md`.2425## Quickstart2627```bash28# Text-to-video29python scripts/video.py \30 --prompt "Glowing crystal-powered rocket launching from red Mars dunes, ancient alien ruins lighting up" \31 --duration 10 --aspect-ratio 16:9 --resolution 720p \32 --output rocket.mp43334# Image-to-video (local file → animated)35python scripts/video.py \36 --prompt "Camera pushes in slowly. The leaves rustle in a gentle breeze." \37 --image ./still.jpg \38 --output animated.mp43940# Reference-to-video (virtual try-on / character consistency)41python scripts/video.py \42 --prompt "The model from <IMAGE_1> walks the runway wearing the shirt from <IMAGE_2>, slow motion, dramatic lighting" \43 --reference-image https://.../model.jpg https://.../shirt.jpg \44 --duration 10 --aspect-ratio 16:9 --resolution 720p \45 --output runway.mp44647# Edit an existing video (preserves duration / aspect / resolution)48python scripts/video.py \49 --prompt "Give the woman a silver necklace" \50 --edit-video https://.../source.mp4 \51 --output edited.mp45253# Extend an existing video (duration = length of NEW portion only)54python scripts/video.py \55 --prompt "The cat notices a butterfly and leaps off the windowsill" \56 --extend-video https://.../cat.mp4 --duration 6 \57 --output extended.mp458```5960The script polls until `status=done`, downloads the resulting MP4 to `--output`, and prints both the local path and the source URL on stderr.6162## Decision rules6364When the user wants a video, route by which inputs they have:65661. **No inputs, just a description** → text-to-video.672. **One image, "make this move"** → image-to-video. The image becomes the first frame; aspect ratio defaults to the image's unless overridden.683. **One or more images, "use these people/clothes/objects"** → reference-to-video. Refer to images in the prompt as `<IMAGE_1>`, `<IMAGE_2>`, etc.694. **An existing video, "change X about it"** → edit-video. Duration / aspect / resolution are inherited and **cannot be overridden** (capped at 720p).705. **An existing video, "show what happens next"** → extend-video. `--duration` is the length of the **extension only**; output total = original + extension.7172Mutually exclusive — don't combine `--image` with `--reference-image`, and don't pass `--edit-video` with `--extend-video`. The script enforces this.7374## Parameters cheat sheet7576| Flag | Values | Default | Notes |77|------|--------|---------|-------|78| `--prompt` | str | required | Use `<IMAGE_1>`, `<IMAGE_2>` markers when referencing images |79| `--duration` | int 1–15 (seconds) | model default | For extend, this is the **added** length only. Edit mode ignores. |80| `--aspect-ratio` | `1:1`, `16:9`, `9:16`, `4:3`, `3:4`, `3:2`, `2:3` | `16:9` | Edit mode ignores; image-to-video defaults to image's ratio |81| `--resolution` | `480p`, `720p` | `480p` | Edit mode ignores (caps at 720p) |82| `--image` | path or `https://` URL or `data:` URI | — | Image-to-video. Local files auto-encoded as base64 data URIs. |83| `--reference-image` | one or more URLs / paths | — | Reference-to-video. Local files auto-encoded. |84| `--edit-video` | URL | — | Edit-video mode. Must be an HTTPS URL (xAI-hosted is fine). |85| `--extend-video` | URL | — | Extend-video mode. Source must be 2–15 s. |86| `--poll-interval` | seconds | 5 | How often to check status |87| `--poll-timeout` | seconds | 900 (15 min) | Bail out if not done by then |88| `--output` | path | required | Where to save the downloaded MP4 |89| `--keep-url` | flag | off | Skip the download, only print the xAI-hosted URL |9091## Things to know9293- **Asynchronous.** Generation typically takes a couple minutes; long / 720p / edit jobs take longer. The script handles polling — don't introduce your own loop on top.94- **URLs are ephemeral.** xAI returns a temporary hosted MP4 URL. The script downloads it immediately. Pass `--keep-url` only if you'll fetch within minutes.95- **Edit mode is constrained.** Editing inherits the source video's duration, aspect, and resolution; passing those flags is silently ignored by the API. Editing input is also capped at **8.7 seconds**.96- **Extend mode bookkeeping.** `--duration` is the length of the **new portion**. A 10s source + `--duration 5` → 15s output.97- **Mode collisions error out 400.** Don't combine `--image` and `--reference-image`, and don't combine `--edit-video` and `--extend-video`.98- **Moderation.** Output may be filtered. The response carries a `respect_moderation` flag; if the model declines the prompt, rewrite rather than retry verbatim.99- **Status enum.** `pending` → keep polling, `done` → download, `expired` → request lifecycle ended (re-submit), `failed` → permanent failure (rewrite prompt or check inputs).100101## When to escalate102103- Need still images, not video → use the sibling `grok-image` or `gpt-image-2` skills.104- Need precise editing of specific regions / masks → not supported here; use `gpt-image-2` for image masking and consider re-animating.105- Need >15 s of fully novel content → chain extends (each adds up to 10s) or generate multiple clips and stitch them externally.
satasuk03