Tiktok Video

TikTok / Douyin Short Video Generator

You (the agent) do the creative work — script, scene breakdown, search keywords, style choices, and quality review. The scripts in scripts/ do the mechanical work — TTS with word timestamps, asset search & download, caption rendering, ffmpeg composition, loudness normalization.

Everything runs with core ffmpeg only (no libass/drawtext needed) plus a Python venv. Works with zero API keys; PEXELS_API_KEY / PIXABAY_API_KEY env vars unlock real stock video clips (better) when present.

Workflow

0. One-time setup (skip if .venv/ and assets/fonts/ exist)

bash scripts/setup.sh

1. VERIFY FACTS FIRST, then write the script + storyboard

Your internal knowledge may be outdated or wrong — never put an unverified claim in a video. Before writing any scene text, use your web search tool to check every concrete claim: numbers, statistics, records, prices, dates, rankings, and every "first / biggest / only / fastest" statement — and anything time-sensitive (news, products, versions, "今年/最新") regardless of how confident you feel. Rules:

• A claim goes into the script only as a source states it — adjust your wording to the source, not the other way around.
• Can't verify it? Replace it with a claim you can, or cut it. A wrong number in a published video is worse than a weaker hook.
• Record the URLs you relied on in the storyboard's "sources": [...] — they are echoed in review/report.txt so the user can audit the video.
• If you genuinely have no web access, stick to long-established textbook facts, avoid specific figures, and tell the user the video is unverified.

Read references/writing-guide.md for the script formulas (hook types, pacing, scene structure). Then create projects/<slug>/storyboard.json:

{
  "title": "深海里最诡异的5种生物",
  "lang": "zh",
  "aspect": "9:16",
  "voice": "zh-CN-YunjianNeural",
  "rate": "+10%",
  "caption_style": "karaoke",
  "bgm": {"mood": "mystery", "gain_db": -16},
  "hook": {"text": "深海禁区", "seconds": 2.3},
  "scenes": [
    {
      "text": "你知道吗?在阳光永远照不到的深海,藏着比科幻电影更诡异的生物。",
      "keywords": ["deep sea NOAA ocean exploration", "submarine dark ocean"],
      "providers": ["openverse", "wikimedia"],
      "badge": null,
      "effect": "kb_in"
    },
    {
      "text": "第一名,鮟鱇鱼……",
      "keywords": ["humpback anglerfish", "anglerfish museum"],
      "badge": "第1名"
    }
  ]
}

Field reference:

Rules of thumb (publish bar): total 45–75s, 6–9 scenes, one idea per scene, 2–3 shots per scene. Scene text 20–45 Chinese chars / 14–30 English words. Anything under ~40s or with single-shot scenes throughout will feel thin — don't ship it.

2. Run the pipeline

.venv/bin/python scripts/pipeline.py projects/<slug>

Or stage by stage: tts.py → assets.py → bgm.py → compose.py → check.py.

3. REVIEW — this is what makes the difference (mandatory)

Free-image search is imperfect. Always view these two files with your image tool and judge them like a human editor:

1. projects/<slug>/media/assets_sheet.jpg — after the assets stage. Shots are labeled 01a 01b 02a …. For every shot that does not clearly match the narration, refetch it:

   .venv/bin/python scripts/assets.py projects/<slug> --scene 3 --shot 2 --keywords "better english nouns"
   

   (Refetching automatically blacklists the rejected asset. Repeat until all scenes match. You may also download an image yourself with your own tools and point scene media at it.)

   The pipeline also text-audits every shot's source title against its query and prints [!] OFF-TOPIC flags. If you visually confirmed a flagged shot is actually right (e.g. an untitled but perfect photo), set "approved": true on that shot in media/manifest.json to clear it — your eyes outrank the text audit. Never blind-approve.

2. projects/<slug>/review/contact_sheet.jpg — after compose+check. Verify: captions readable & synced, no wrong/ugly frames, hook visible. Fix issues (edit storyboard → rerun compose, it's fast) until it looks like a video a 营销号 editor would actually publish.

review/report.txt contains duration/loudness checks and a ready-to-paste attribution block (CC-BY sources + music credit) — always give it to the user together with final.mp4.

Asset search tips (biggest quality lever)

• Keywords must be English, 1–4 words, concrete visible nouns ("humpback anglerfish", "scuba diver silhouette"), never abstract concepts ("mystery", "success").
• wikimedia is best for animals/science/history/places; openverse (Flickr etc.) for lifestyle/scenery/mood; nasa for space; keys unlock pexels_video/pixabay_video real footage — prefer those when available.
• Add a context word to disambiguate: "NOAA", "museum", "aquarium", "macro".
• If a scene keeps failing, change the visual concept, not just the words (e.g. for "5% explored" show a diver silhouette, not "statistics").
• wikimedia_video searches support negative terms — Commons weather/space queries drown in satellite loops, so write e.g. "lightning storm -CIRA -satellite -JPSS". Non-English words unlock extra footage ("Blitz Gewitter", "tormenta rayos"). Prefer named subjects ("Fagradalsfjall lava drone") over generic nouns, and always eyeball video thumbs in the sheet: watch out for burned-in captions, watermarks, tutorials and CGI promos.

Troubleshooting

• edge-tts network errors: it retries 4×; rerun tts.py if it still fails.
• BGM download fails: rerun bgm.py (retries + cache), or --mood none, or drop an mp3 into assets/bgm/ and set bgm.file.
• BGM style doesn't fit: refine the query (bgm.py <dir> --query "..." then recompose with --skip-tts --skip-assets). For a 真正的当下热门 sound, deliver with mood: none (or low gain) and tell the user to add the trending track from TikTok/抖音's in-app licensed music library at upload time — that is both legal and algorithm-favored. Cut timing will still be clean because scene pacing is unchanged.
• check.py prints a "cuts land on the beat" line — if beat sync was skipped (low grid confidence, e.g. beatless ambient music), pick a steadier track (--query "trap beat" etc.) and recompose.
• A provider erroring/empty is fine — others cover it; check per-scene logs.
• Recompose after any storyboard edit is cheap (--skip-tts --skip-assets via pipeline.py, or run compose.py directly). Changing scene text requires rerunning tts.py.
• Emoji in captions render on macOS (Apple Color Emoji); on Linux they are dropped unless NotoColorEmoji is installed. Never rely on emoji for meaning.

Deliver

Give the user: final.mp4, the attribution block from review/report.txt, and (if asked) review/cover.jpg as the cover image.