# Compose Cinematic Mv

> compose-cinematic-mv — music + intention → beat-synced cinematic MV

- Skill: `yijiaduan/compose-cinematic-mv` (Agent Skill, multi-file: 5 files)
- Install (CLI): `npx skillmds@latest add yijiaduan/compose-cinematic-mv`
- Raw SKILL.md: https://api.skillmd.com/api/skills/yijiaduan/compose-cinematic-mv/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: YijiaDuan (https://skillmd.com/u/yijiaduan)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/yijiaduan/compose-cinematic-mv

---


# compose-cinematic-mv — music + intention → beat-synced cinematic MV

Turn a **text intention** (a mood / imagery / scene idea) plus **music** into a short vertical cinematic MV: design an original scene → first-frames with **Seedream 4.0** (text-to-image) → motion with **Kling** (image-to-video) → cut the shots **on the music's beats** with ffmpeg. The aesthetic is modeled on Xiaohongshu creator *Rainshae* (lone figure in a vast landscape, cold cinematic mood, 3:4) — see `reference/style-patterns.md`.

> **The feeling comes from cuts riding the music, with short information-dense shots — not from pretty stills alone.** So this skill is **music-first** and **beat-synced**.

## Preconditions

- macOS with `ffmpeg`/`ffprobe`, `curl`, `openssl`, `bc`.
- Credentials as environment variables: `ARK_API_KEY` (Volcengine Ark / Seedream), `KLING_AK`, `KLING_SK` (Kling open platform). Keep them in a file **outside this repo** — e.g. `~/.config/secrets.env`, `chmod 600` — and source it before running: `set -a; . ~/.config/secrets.env; set +a`. **Never paste keys into the chat.**
- A working internet path to `ark.cn-beijing.volces.com` and `api-beijing.klingai.com` — **see Network gotchas below; a local proxy will break these unless bypassed.**

## The workflow (music-first)

### 1. Design an original scene from the intention
From the user's intention, invent a NEW scene in the Rainshae idiom (title formula `{年份}，你{第二人称处境}`, lone figure / vast nature / strong mood / 3:4). **Do not reproduce the reference creator's specific scenes** (no "英国白崖", "长安", etc.). Read `reference/style-patterns.md` for the pattern, then create fresh.

### 2. Find matching music FIRST, then analyze its beats
Pick a background track that fits the scene's mood **before** planning shots. Reference the example videos for what fits: slow (~60–80 BPM), solo piano / ambient / strings, melancholic-serene ("深夜疗愈"), with phrase rise-and-fall (not a static drone).
- Source royalty-free / public-domain audio (e.g. archive.org public-domain piano — Satie *Gymnopédie/Gnossienne*, Chopin nocturnes, Debussy; or licensed ambient). archive.org direct files work; **Pixabay blocks scraping (403)**.
- **Avoid noisy recordings.** archive.org "Great 78 Project" transfers (identifiers starting `78_` or containing `gbia`) have heavy surface hiss — denoise can't save them. Prefer modern/clean uploads. **Verify the noise floor before committing:** the quietest 0.5s RMS should be very low for a clean digital recording (≲ −60 dB); a noisy 78 sits around −40 dB. Check with:
  `ffmpeg -v error -i x.mp3 -af "aformat=channel_layouts=mono,asetnsamples=22050,astats=metadata=1:reset=1,ametadata=print:key=lavfi.astats.Overall.RMS_level:file=nf.txt" -f null - ; grep RMS_level= nf.txt | sed -E 's/.*=//' | sort -n | head -1`
- Download into the working dir, take a ~20–30s excerpt, normalize loudness, fade:
  `ffmpeg -ss <start> -t 26 -i src.mp3 -af "loudnorm=I=-16:TP=-1.5,afade=t=in:st=0:d=1.2,afade=t=out:st=23.5:d=2.5" -ar 44100 -ac 2 excerpt.mp3`
- Detect downbeats: `scripts/find_beats.sh excerpt.mp3` → space-separated beat times. These are your cut points.

### 3. Plan shots ON the beats (short!)
- **Short shots, ~2–3s each** (5s reads as boring — each shot carries little info). Set the timeline cut points to a subset of the detected downbeats.
- One **distinct** generated video per shot — **no reusing/re-cutting one clip into multiple shots** (that was a cost-saving exception in the prototype, not the method).
- Mix shot types across the scene (wide establishing → medium figure → detail/atmosphere → closing long shot), camera moves slow.

### 4. Generate first-frames — Seedream, in parallel
For each shot write a model-ready English image prompt (subject, setting, era, framing, light, palette, mood, lens, film grain) ending in the right aspect. Generate concurrently (Seedream allows it):
```bash
set -a; . ~/.config/secrets.env; set +a
scripts/gen_image.sh "<prompt s1>" shot1.jpeg &
scripts/gen_image.sh "<prompt s2>" shot2.jpeg &
... ; wait
```
View each frame (Read tool) and regenerate any that miss before spending video credits.

### 5. Generate per-shot videos — Kling, parallel, short, fast poll
Submit up to **5 concurrent** tasks; generate **short** single clips (`duration` 5, matching the ~2–3s shots — cheaper and fits fast cutting). Poll every **~6–8s**.
```bash
scripts/kling.sh submit shot1.jpeg "<video prompt s1>" kling-v1-6 5   # -> task_id
# ...submit all, collect task_ids, then poll each until task_status=succeed
scripts/kling.sh poll <task_id>                                       # -> url when done
curl --noproxy '*' -o clip1.mp4 "<url>"                               # CDN needs --noproxy
```
Parse with grep (avoid python, see gotchas): status `grep -oE '"task_status":"[a-z]+"'`; url `grep -oE '"url":"[^"]+"'`.

### 6. Assemble beat-synced
Build `plan.tsv` (one line per shot, in order): `clip<TAB>inpoint<TAB>duration`, where the cumulative durations equal your chosen downbeat cut points. Then:
```bash
scripts/assemble.sh plan.tsv excerpt.mp3 out.mp4
```
Verify: probe duration, extract a couple frames at cut boundaries (Read) to confirm each cut changes the image.

### 7. QA the film — mandatory, before showing the user

You do the QA, not the user. Run every item below before showing a cut — all three failure classes have really happened:

- **物理/逻辑 bug**：对每个 clip 抽 2–3 帧（开头/中间/结尾）用 Read 看。重点抓"运动载具窗外景色静止"这类矛盾（火车在开、窗外雪原不动 = 重做该 shot）、人脸/手部崩坏、水面倒影错误。
- **风格一致性**：把所有 shot 的首帧并排看一遍，气质突兀的（如整组荒漠废墟里突然插两张壁画特写）换掉重生成，保持同一意象世界。
- **配乐**：交付前完整听一遍 excerpt。有底噪/嘶声（RMS 检查漏网的）就换曲子，不要指望降噪。
- **剪辑**：完整看一遍成片（或按节拍点抽帧），确认每个 cut 都换了画面、节奏踩在强拍上。

发现问题：只重做有问题的 shot（Kling 按条计费），别整片重跑。修完再交付，不要交付已知有 bug 的片子。

### 8. Deliver

Hand over `out.mp4` together with the shot list and the track you used (title + source + license), so the user knows what they can publish.

**Local overlay:** if a file named `SKILL.local.md` exists next to this one, read it — that is where an individual user keeps their own private delivery conventions (where finished films get uploaded, which site to update, which host to deploy to). It is git-ignored and never part of this repo. Absent that file, just deliver the video.

## Things that will bite you (all learned the hard way)

- **Local proxy breaks the APIs.** If the user runs ClashX/Clash (e.g. `HTTPS_PROXY=127.0.0.1:7890`), TLS to `volces.com`/`klingai.com` gets reset. Fix on their side: rule mode + DIRECT rules for `volces.com` and `klingai.com`. For **downloads**, always `curl --noproxy '*'` — result CDNs (`kechuangai.com` for Kling, Ark image CDN) are NOT covered by those rules and the proxy MITMs them (self-signed-cert / truncated downloads).
- **Prefer bash + curl + openssl over python.** In this sandbox python intermittently throws `PermissionError: Operation not permitted` on stdlib import (`_path_importer_cache`). All scripts here are python-free. (If you do use python for SSL, python.org's interpreter lacks CA certs → `ssl.create_default_context(cafile="/etc/ssl/cert.pem")`.)
- **Beat detection via ffmpeg, not librosa.** numpy/librosa hit the same PermissionError; `find_beats.sh` uses an ffmpeg RMS envelope.
- **Kling caps concurrency at 5.** Submitting a 6th in-flight task returns `1303 parallel task over resource pack limit`. Submit in batches of ≤5: poll+download the first batch, then submit the next.
- **Account setup is the usual blocker, not code.** Ark: the Seedream model must be **activated** in the Ark console (`ModelNotOpen`). Kling: the **API resource pack** must be purchased (`1102 Account balance not enough`) — distinct from klingai.com consumer credits.
- **Quoting:** prompts passed to the scripts must contain **no double-quotes** (JSON is built inline). Keep prompts to commas/letters.
- **Older files can become unreadable.** Files downloaded/created in earlier sessions may carry `com.apple.quarantine`/`provenance` and the sandbox denies access (`Operation not permitted`) — keep a run's working files together and don't depend on re-reading prior-session media.
- **Model ids:** Seedream `doubao-seedream-4-0-250828`; Kling `kling-v1-6` (std). Endpoints: `ark.cn-beijing.volces.com/api/v3/images/generations`, `api-beijing.klingai.com/v1/videos/image2video`.

## Files

- `SKILL.md` — this playbook.
- `scripts/find_beats.sh` — audio → downbeat times (ffmpeg RMS envelope).
- `scripts/gen_image.sh` — Seedream 4.0 text-to-image (curl); run in parallel.
- `scripts/kling.sh` — Kling image-to-video submit/poll (openssl JWT).
- `scripts/assemble.sh` — beat-synced assembler: (clip, in-point, duration) shots + music.
- `reference/style-patterns.md` — the Rainshae style规律 (title/visual/pacing/music) for creating original scenes.