LLM Video Generator

LLM Video Generator

Generate videos via ZhipuAI CogVideoX-3. Each API call produces ~5s of video. For longer videos, chain multiple calls using last-frame continuation, then concatenate.

Scripts

All scripts use /opt/anaconda3/bin/python3. Resolve <skill-dir> to this skill's directory.

Workflow

Step 1: Assess Request & Clarify

Clear request → proceed to Step 2. A request is clear when:

• Video content/scene is described with enough detail
• Style or visual tone is specified or implied
• Duration is stated (default: 5s if not specified)

Vague request → propose a plan first:

基于你的需求，我拟定了以下视频方案：

📹 **视频内容**: [detailed scene description with key moments]
🎨 **视频风格**: [e.g., 写实/动画/电影感/温馨...]
⏱️ **视频时长**: [Xs, note: will be generated in 5s segments]
🔊 **背景音乐**: 有/无
📐 **分辨率**: 1920x1080
🎞️ **帧率**: 30fps

你觉得这个方案可以吗？需要调整哪些部分？

Iterate with the user until confirmed.

Step 2: Estimate Time & Notify User

Before starting generation, calculate and report the estimated time:

Time estimation formula:

• Base: 1 minute per second of video (e.g., 20s video ≈ 20 minutes)
• High-definition (4K or 60fps): add +30% (e.g., 20s 4K video ≈ 26 minutes)
• Additional overhead: ~2 minutes for frame extraction, concatenation, and compression
• Segments: ceil(target_duration / 5)

MUST send this message to the user before starting generation:

⏳ **视频生成预估**

📊 分段计划：{N} 段（每段约5秒）
⏱️ 预计总耗时：约 {estimated_minutes} 分钟
📐 分辨率：{resolution}

视频生成是一个耗时过程，请耐心等待。我会在每段完成后实时汇报进度。

Example for a 30s 1080P video:

• 6 segments, base time = 30 minutes, +2 min overhead → ~32 minutes
• Message: "预计总耗时：约 32 分钟"

Example for a 20s 4K video:

• 4 segments, base time = 20 * 1.3 = 26 min, +2 min → ~28 minutes

Step 3: Plan Generation Segments

Each API call produces ~5 seconds. Calculate segments: ceil(target_duration / 5)

For multi-segment videos, plan how the content evolves across segments. Write a prompt for each segment describing what happens in that 5-second window, maintaining visual continuity.

Step 4: Execute Generation with Progress Reports

CRITICAL: After each segment completes, IMMEDIATELY send a progress message to the user before starting the next segment. Do not wait until all segments are done.

Progress message format (send via message tool or inline reply after each segment):

✅ 进度：{completed}/{total} 段完成（第{N}段已生成）
📝 内容：{brief segment description}
⏱️ 本段耗时：{minutes}分钟
📊 预计剩余：约 {remaining_minutes} 分钟

Generation process:

Segment 1 — Text-to-Video:

/opt/anaconda3/bin/python3 <skill-dir>/scripts/video_gen.py text2video \
  --prompt "<segment_1_prompt>" \
  --quality quality --audio true --size 1920x1080 --fps 30 \
  --output-dir <output-dir> --max-wait 900

→ Send progress message to user

Segments 2+ — Image-to-Video (last-frame continuation):

For each subsequent segment:

1. Extract last frame from the previous segment's video:

/opt/anaconda3/bin/python3 <skill-dir>/scripts/extract_last_frame.py \
  <previous_video.mp4> --output <output-dir>/frame_segN.png

2. Generate next segment using the last frame as input:

/opt/anaconda3/bin/python3 <skill-dir>/scripts/video_gen.py image2video \
  --prompt "<segment_N_prompt>" \
  --image-url <output-dir>/frame_segN.png \
  --quality quality --audio true --size 1920x1080 --fps 30 \
  --output-dir <output-dir> --max-wait 900

3. → Send progress message to user

Repeat for all segments.

Alternative — Frames-to-Video mode:

If you have both a starting and ending image for a segment:

/opt/anaconda3/bin/python3 <skill-dir>/scripts/video_gen.py frames2video \
  --prompt "<description>" \
  --first-frame <first.png> --last-frame <last.png> \
  --quality quality --audio true --size 1920x1080 --fps 30 \
  --output-dir <output-dir>

Step 5: Concatenate Segments

After all segments are generated, combine them:

/opt/anaconda3/bin/python3 <skill-dir>/scripts/concat_videos.py \
  --inputs <seg1.mp4> <seg2.mp4> ... \
  --output <output-dir>/final_video.mp4

If the final file exceeds 25MB (Feishu upload limit), compress with ffmpeg:

ffmpeg -i <input> -c:v libx264 -crf 32 -c:a aac -b:a 96k -vf "scale=1280:720" -y <output>

Step 6: Deliver

• Share the final video file with the user
• For Feishu delivery: use feishu-send-file skill to send the .mp4 file
• Final report:

🎬 **视频生成完成！**

⏱️ 总时长：{duration}秒
📦 文件大小：{size}MB
📊 共 {N} 段，总耗时 {total_minutes} 分钟

Prompt Tips

• Use English prompts for best quality (translate Chinese descriptions)
• Be specific: scene, camera angle, lighting, motion, atmosphere
• Include style keywords: cinematic, realistic, cartoon, watercolor, etc.
• For continuation segments, describe the action progression, not the full scene from scratch
• Keep each segment prompt concise (1-3 sentences)

Parameters Reference

Error Handling

• Missing ZHIPU_API_KEY: Ask user to set environment variable
• Missing zai-sdk: pip install zai-sdk (under anaconda)
• Missing ffmpeg: Required for frame extraction and concatenation
• Task timeout: Increase --max-wait or retry; check task status manually via API
• Task failed: Simplify the prompt and retry
• File too large for Feishu: Compress with ffmpeg (reduce resolution or increase CRF)