Video Generation

Video Generation Skill

Domain: AI Video Generation
Version: 2.0.0
Last Updated: 2026-04-15
Author: the AI assistant (your AI assistant)
Source: Patterns from AlexVideos CLI toolkit
Related: image-handling (images), text-to-speech (audio)

Overview

Generate AI videos using 17 cloud models on Replicate. Supports text-to-video, image-to-video, video editing, and talking-head workflows.

Model Catalog (17 Models)

Parameter Support Matrix

Notes:

• Fixed values (e.g., 5/9) mean only those exact values accepted
• Ranges (e.g., 3–15) accept any integer in range
• Sora models use "portrait"/"landscape" internally (mapped from 9:16/16:9)

Model Selection Guide

Talking-Head Video Workflow

Proven pipeline for generating a person speaking to camera with synchronized speech:

# Step 1 — Age-progress reference photo (if needed)
node scripts/generate-image.js "A 26-year-old professional in studio" --model nanapro --image ref.png

# Step 2 — Generate video with Kling v2.6 (best talking-head i2v)
node scripts/generate-video.js "Person looking directly into camera, speaking confidently" --model kling26 --image aged.jpg --duration 10

# Step 3 — Generate speech
node scripts/generate-voice.js "[script]" --model mm28turbo

# Step 4 — Merge video + speech
node scripts/generate-edit-video.js --model avmerge --video clip.mp4 --audio speech.mp3

Model notes:

• kling26 — Best overall motion quality for talking-head i2v
• grok — Best lip sync, but no native audio (requires avmerge)
• veo3fast / sora — May flag real-person reference photos

Prompt pattern:

"A sharp [age]-year-old [description] looking directly into camera, speaking to the audience about [topic]. Calm, confident, broadcast-quality delivery. Professional studio lighting."

Video Editing (10 Models)

Generation Workflows

Text-to-Video

import Replicate from "replicate";

const replicate = new Replicate();

const output = await replicate.run("google/veo-3.1-fast", {
  input: {
    prompt: "A person walking through a futuristic city at sunset, cinematic lighting",
    duration: 5, // seconds
  },
});

console.log("Video URL:", output);

Image-to-Video (Animation)

import { readFileSync } from "fs";

const output = await replicate.run("kwaivgi/kling-v3-video", {
  input: {
    prompt: "Gentle wind blowing through hair, subtle movement",
    image: readFileSync("source-image.png"), // or URL
    duration: 5,
  },
});

Talking Head with Lip-Sync

const output = await replicate.run("xai/grok-imagine-video", {
  input: {
    prompt: "Person speaking to camera with natural expressions",
    audio: readFileSync("narration.mp3"), // Voice audio
    duration: 10,
  },
});

Quality Guidelines

Prompt Engineering

Duration Selection

Resolution

• Most models default to 720p or 1080p
• Kling v3 supports explicit 1080p
• Higher resolution = longer generation time

Audio Handling

Models with Built-in Audio

Adding Audio Post-Generation

For models without audio, combine with TTS:

// 1. Generate video (no audio)
const video = await replicate.run("kwaivgi/kling-v3-video", {
  input: { prompt: "...", duration: 5 },
});

// 2. Generate narration via TTS
const audio = await replicate.run("minimax/speech-2.8-turbo", {
  input: { text: "Narration text", voice: "Wise_Woman" },
});

// 3. Combine with ffmpeg
// ffmpeg -i video.mp4 -i audio.mp3 -c:v copy -c:a aac output.mp4

Cost Estimation

Output Handling

Download Video

import { writeFileSync } from "fs";

const response = await fetch(output);
const buffer = await response.arrayBuffer();
writeFileSync("output.mp4", Buffer.from(buffer));

Verify Quality

# Check video properties with ffprobe
ffprobe -v quiet -print_format json -show_format -show_streams output.mp4

Error Handling

Integration with Studio Agents

• Image Studio: Generates source images for image-to-video
• Audio Studio: Creates narration for post-hoc audio
• Visual Memory: Stores reference images/videos