Gen AI Video

Video generation with gen-ai (CLI & MCP)

Video is the most expensive modality — a single Sora 2 Pro call can exceed $5. This skill covers how to produce video reliably and cost-effectively via the Picsart gen-ai CLI or the @picsart/gen-ai-mcp MCP server.

When to use

Activate whenever the user asks to:

• Generate a video, clip, reel, trailer, demo, teaser, or ad spot
• Animate a still image (image-to-video / I2V)
• Edit or re-style an existing video (video-to-video / V2V)
• Extend a short clip to a longer one
• Build a talking-head avatar / explainer
• Attach an AI-generated soundtrack to a clip
• Pick between Sora, Kling, Veo, Runway, Luma, Hailuo, Pika, LTX, Wan, or Seedance

CLI vs MCP — both work

• CLI when installed — best for scripting, CI pipelines, and long-running extend chains.
• MCP server (@picsart/gen-ai-mcp) when the user is in Claude Code, Codex, Cursor, Windsurf, or ChatGPT and the CLI isn't installed — the agent calls gen-ai_generate and gen-ai_extend natively.

Model selection cheat sheet

Use family aliases. gen-ai models --mode video shows what's current.

Future-proofing: family aliases (kling-v3-pro, sora-2) survive model upgrades. When the user names a specific version, honor it. New T2V and I2V models land frequently — always check list-models before using a version pin.

Quick decision tree

Hero / narrative / premium?            → sora-2-pro  or  veo-3.1        (check pricing first!)
On-brand product/campaign reel?        → runway-gen45-t2v  or  kling-v3-pro
Animate a still image?                 → kling-motion-control-v3  (pass -i hero)
On-brand animated from a reference?    → runway-gen4-ref           (pass -i ref.png)
Talking-head avatar / explainer?       → kling-avatar              (pass -i portrait)
Short draft / iteration?               → kling-v3-standard  or  luma-ray2-t2v
Restyle / reformat an existing clip?   → runway-aleph              (pass --video input.mp4)
Add music/foley to an existing clip?   → kling-v2a                 (pass --video clip.mp4)
Need longer than one call?             → gen-ai extend  --video clip.mp4 --times N --ar 16:9
Need 4K?                               → bytedance-upscaler post-gen

Prompting best practices

Shot, motion, mood — in that order

Video models read motion first, mood second, subject third. Lead with the camera.

1. Camera / motion. "Slow dolly in," "hand-held push-in," "subject-tracking pan right," "locked-off tripod, no motion."
2. Shot type. "Wide establishing," "medium close-up," "over-the-shoulder," "macro product detail."
3. Subject + action. "Barista pours a latte; steam rises."
4. Mood / style. "Cinematic, warm tungsten, golden hour, shallow depth of field."
5. Duration cue. State the duration in the prompt AND pass --duration. Models that don't respect --duration still honor "a 5-second clip" in the prompt.

Rules of thumb

• Short phrases beat run-ons. 2–3 short phrases separated by commas outperform one long sentence.
• Lock the subject with a reference image whenever possible. runway-gen4-ref and kling-motion-control-v3 beat any T2V for brand consistency.
• Avoid contradictions. "Static subject, fast-moving camera" confuses every model. Pick one.
• Negative prompts: --negative-prompt "warped faces, extra limbs, watermark, text, morphing" — still useful even on video.
• Aspect + duration determine cost. 16:9 / 5s is the cheapest safe default for social reels.

Family-specific tips

Duration planning

Most models cap at 5–10 seconds per call. Build longer pieces with gen-ai extend. Pass --ar when extending local files so the workflow does not depend on local media probes:

gen-ai extend --video clip.mp4 --times 3 --ar 16:9   # chain 3 extensions (~7s each)

Each extend is billed separately — factor that into estimates.

Common recipes

T2V — 5s, 16:9 (cheapest safe default)

gen-ai generate -m kling-v3-standard \
  -p "slow dolly in, ocean waves at dusk, golden light, cinematic" \
  --duration 5 --ar 16:9 --json --no-input | jq -r '.url' | xargs curl -L -o reel.mp4

T2V — premium hero reel

gen-ai pricing sora-2-pro --duration 8
gen-ai generate -m sora-2-pro \
  -p "first, establishing wide of a ceramic studio; then, slow push-in to a single espresso cup; finally, steam curls into frame" \
  --duration 8 --ar 16:9 --json --no-input | jq -r '.url' | xargs curl -L -o hero-reel.mp4

I2V — animate a hero image

gen-ai generate -m kling-motion-control-v3 -i hero.webp \
  -p "slow parallax push-in, subtle camera drift, steam rises" \
  --duration 5 --json --no-input | jq -r '.url' | xargs curl -L -o hero-reel.mp4

Reference-based brand motion

gen-ai generate -m runway-gen4-ref -i brand-ref.png \
  -p "camera dolly right, subject confident, warm studio light" \
  --duration 5 --ar 16:9 --json --no-input | jq -r '.url' | xargs curl -L -o brand-reel.mp4

Talking-head avatar from a portrait

gen-ai generate -m kling-avatar -i portrait.png \
  -p "Welcome to Picsart. Let's build something together." \
  --duration 8 --json --no-input | jq -r '.url' | xargs curl -L -o explainer.mp4

Extend a 5s clip to ~28s

gen-ai extend --video hero-reel.mp4 --times 3 --ar 9:16   # 3 × 7s tails ≈ 26s + original 5s

Video-to-video edit (reformat / restyle)

gen-ai generate -m runway-aleph --video input.mp4 \
  -p "reframe 16:9 to 9:16, preserve subject centered, slight blur on edges" \
  --json --no-input | jq -r '.url' | xargs curl -L -o social.mp4

Attach an AI soundtrack

# Option A: no local mixer needed; V2A writes matched audio into the video
gen-ai video-audio -m kling-v2a --video reel.mp4 \
  -p "confident warm synth bed with subtle foley, uplifting, 30s" \
  --json --no-input | jq -r '.url' | xargs curl -L -o reel-with-sound.mp4

# Option B: optional local mixer if you already have ffmpeg and separate stems
gen-ai generate -m minimax-music -p "confident warm synth, 120bpm, uplifting, 30s" \
  --json --no-input | jq -r '.url' | xargs curl -L -o music.mp3
ffmpeg -i reel.mp4 -i music.mp3 -c:v copy -map 0:v -map 1:a reel-with-music.mp4

Upscale post-gen

gen-ai generate -m bytedance-upscaler --video reel.mp4 --scale 2 --json --no-input \
  | jq -r '.url' | xargs curl -L -o reel-4k.mp4

End-to-end — photo → 30s reel with music

# 1. Animate the hero
gen-ai generate -m kling-motion-control-v3 -i hero.webp --json --no-input \
  | jq -r '.url' | xargs curl -L -o v.mp4

# 2. Extend twice (total ~21s)
gen-ai extend --video v.mp4 --times 2 --ar 9:16 --download ./extended

# 3. Add AI-matched music/foley without local media tools
# Replace <extended-video>.mp4 with the downloaded file name from ./extended.
gen-ai video-audio -m kling-v2a --video ./extended/<extended-video>.mp4 \
  -p "confident warm synth bed, 120bpm, uplifting, subtle product foley" \
  --json --no-input | jq -r '.url' | xargs curl -L -o final.mp4

MCP / natural-language patterns

When the CLI isn't installed, the MCP server gives the agent the same capabilities. Good prompts:

• "Animate hero.webp into a 10-second reel with Kling motion control. Add a MiniMax soundtrack. Save as reel.mp4."
• "Generate a cinematic 5s hero reel for a new ceramic cup product. Use Runway Gen 4.5."
• "Turn this portrait into a 8-second explainer avatar reading: 'Welcome to Picsart.' Use Kling Avatar."
• "Take reel.mp4 and reformat 16:9 → 9:16 preserving the subject. Use Runway Aleph."
• "Extend clip.mp4 by ~20 seconds with the same vibe."
• "Estimate first — then render a Sora 2 Pro hero shot 8 seconds 16:9. Show me the cost before spending."

The agent must:

1. Always check gen-ai pricing first for any T2V, I2V, or extend operation. Video is expensive.
2. Prefer kling-v3-standard, luma-ray2-t2v, or veo-3.1-fast for drafts; only upgrade to sora-2-pro or veo-3.1 for the final.
3. Use I2V (reference-locked) whenever the user has a hero or brand asset — never T2V alone if a reference is available.
4. Pause and confirm for any single call likely over $5 or batch likely over $20.

Cost control

Always check pricing before spending on video:

gen-ai pricing sora-2-pro --duration 8
gen-ai pricing kling-v3-pro --duration 5
gen-ai pricing veo-3 --duration 7

Rule-of-thumb per-clip cost:

• Kling Standard / Luma / Hailuo — ~$0.30–$0.80
• Kling Pro / Veo Fast / Runway Gen 4.5 — ~$1–$2
• Sora 2 / Veo Flagship — ~$3–$5
• Sora 2 Pro / 10s+ extends — $5–$15+

Draft → final pattern: iterate on prompts with kling-v3-standard (cheap). Once the shot is right, regenerate the final with the flagship. Cuts cost 5–10×.

Output hygiene

• Container: .mp4 is the safe default. Some tools emit .webm — use --format mp4 to force.
• Duration: social reels 5–15s; hero films 15–30s; demos 30–60s. Over 30s usually means chaining.
• Aspect: 16:9 for YouTube / LP; 9:16 for IG Story / TikTok / Shorts; 1:1 for feed posts.
• Poster frame: use the original hero image, an approved still, or a generated thumbnail for thumbnail / OG. Optional local extraction if ffmpeg is already available: ffmpeg -ss 00:00:02 -i reel.mp4 -frames:v 1 poster.webp.

Troubleshooting

For agents

• Always check pricing before spending on video. A single Sora 2 Pro call can exceed $5.
• Prefer Kling Standard / Luma / Veo Fast for drafts; upgrade only the final cut.
• If the user has a reference image, use I2V (runway-gen4-ref, kling-motion-control-v3) — never T2V alone.
• Default duration = 5s at kling-v3-standard unless user specifies otherwise.
• Default aspect = 16:9 unless the user names a social platform.
• Never chain more than 3 extends without warning the user — cost compounds.
• If the CLI isn't installed and MCP is available, call MCP tools directly.
• Log every video generation in ~/.gen-ai/history/ — expensive artifacts should never be lost.