Avatar Talking Video
Turn text → a lip-synced talking-head video of an avatar, in two stages:
- Voice (TTS). Delegates to the
voice-clone
skill to speak the text in the avatar's cloned voice. It reuses the avatar's
already-trained voice, or trains one from --source if the avatar has none.
- Video (lip-sync). Sends a camera-angle image (
image, from the
avatar-camera-angles skill) plus the
generated mp3 (audio) to prunaai/p-video-avatar on Replicate. When
audio is supplied it drives the speech directly (the model's own voice
settings are ignored), so lip-sync follows the cloned voice exactly.
Output lands in <avatar>/generated-videos/ with a manifest.json recording
the text, voice, image, audio, and video params.
Requirements
pip3 install -r requirements.txt (Replicate client). The voice-clone skill
must be installed (its TTS does stage 1); install its requirements too.- A Replicate API token, shared with the other Replicate skills (voice-clone,
gpt-image-2, avatar-video-reel, …) and discovered automatically. To set/refresh:
python3 scripts/setup_key.py YOUR_REPLICATE_API_TOKEN.
Inputs you need
- An avatar folder (e.g.
lolo/) — the one containing a videos/ dir. It is
auto-inferred from --image/--audio, or pass --avatar-dir.
- A camera-angle image — produce it first with
avatar-camera-angles
(a front-ish, clean portrait works best). Pass it with --image. If omitted,
<avatar>/frames/frame_0001.png is used.
- The text to say (positional or
--text-file). The avatar must have a
trained voice, or pass --source so stage 1 trains one first.
Usage
# Text -> cloned-voice audio -> talking-head video, from a camera angle
python3 scripts/generate_video.py "Hola, soy Lolo y te cuento algo." \
--image lolo/angles/skill_test/lolo_push_in.png
# 1080p + happy delivery + a custom visual prompt
python3 scripts/generate_video.py "Big news today!" \
--image lolo/angles/skill_test/lolo_push_in.png \
--resolution 1080p --emotion happy \
--video-prompt "The person is talking and smiling warmly."
# Reuse an existing mp3 (skip TTS) — just lip-sync it to the angle
python3 scripts/generate_video.py \
--audio lolo/generated-audios/001_hola.mp3 \
--image lolo/angles/skill_test/lolo_push_in.png
# Avatar has no voice yet: train it first from a clean voice clip
python3 scripts/generate_video.py "Hello there" \
--image lolo/angles/skill_test/lolo_push_in.png \
--source lolo/videos/2026-05-16_12-25-46_voice/voice_concat.mp3
Key options
| Option |
Default |
Description |
text / --text-file |
— |
What the avatar says (drives stage-1 TTS). |
--audio PATH |
— |
Use this mp3/wav directly and skip TTS. |
--image PATH |
<avatar>/frames/frame_0001.png |
Camera-angle / portrait image (image input). |
--avatar-dir |
auto |
Avatar folder (else inferred from --image/--audio). |
--resolution |
720p |
720p or 1080p (1080p ≈ 2× the cost). |
--video-prompt |
The person is talking. |
What the person is doing while speaking. |
--emotion |
auto |
TTS delivery (forwarded to voice-clone). |
--language-boost |
detect |
TTS language (auto-detected; forwarded). |
--voice-id / --name / --source |
— |
Pick / train the voice (forwarded to voice-clone). |
--negative-prompt |
— |
What to avoid (e.g. subtitles, text, watermark). |
--seed |
— |
Reproducible generation. |
--disable-prompt-upsampling |
off |
Use --video-prompt verbatim. |
Output (in <avatar>/generated-videos/)
| File |
What it is |
<NNN>_<slug>.mp4 |
The talking-head video (auto-numbered) |
manifest.json |
items[] mapping each video → text, voice_id, image, audio, resolution, video_prompt, and the rest of the params |
Stage 1 also leaves the generated audio in <avatar>/generated-audios/ (the
voice-clone skill's own output). Report the video path, the voice_id used, and
the resolution when done.
Notes
audio overrides voice settings. Because we always pass audio, the
model's voice / voice_script / voice_language inputs are intentionally
unused — lip-sync follows the cloned-voice mp3.- Cost is per second of output: 720p ≈ $0.025/s, 1080p ≈ $0.045/s.
- Image quality matters. A clean, front-ish angle preserves identity best;
heavy angles or occlusion hurt lip-sync and likeness.
- Per-shot reels: generate several angles with
avatar-camera-angles, run
this skill once per angle with the same text/voice, then stitch the clips.
Related skills
1---2name: avatar-talking-video3description: Generate a talking-head avatar video from a line of text. Stage 1 synthesizes the text in the avatar's cloned voice via the voice-clone skill (reusing the avatar's trained voice, or training one from a source recording if it has none). Stage 2 feeds a camera-angle image (from avatar-camera-angles) plus the generated mp3 to prunaai/p-video-avatar on Replicate at 720p (or 1080p) and saves the lip-synced MP4 under <avatar>/generated-videos/ with a manifest.json. Use when the user wants a talking avatar / talking-head video, a lip-synced clip of an avatar saying some text, to animate a camera-angle still into a speaking video, or mentions "video del avatar hablando", "avatar que hable", "talking head", "lip sync", or p-video-avatar.4---56# Avatar Talking Video78Turn **text → a lip-synced talking-head video** of an avatar, in two stages:9101. **Voice (TTS).** Delegates to the [`voice-clone`](../voice-clone/SKILL.md)11 skill to speak the text in the avatar's cloned voice. It reuses the avatar's12 already-trained voice, or trains one from `--source` if the avatar has none.132. **Video (lip-sync).** Sends a **camera-angle image** (`image`, from the14 [`avatar-camera-angles`](../avatar-camera-angles/SKILL.md) skill) plus the15 generated **mp3** (`audio`) to **`prunaai/p-video-avatar`** on Replicate. When16 `audio` is supplied it drives the speech directly (the model's own voice17 settings are ignored), so lip-sync follows the cloned voice exactly.1819Output lands in `<avatar>/generated-videos/` with a `manifest.json` recording20the text, voice, image, audio, and video params.2122## Requirements2324- `pip3 install -r requirements.txt` (Replicate client). The `voice-clone` skill25 must be installed (its TTS does stage 1); install its requirements too.26- A Replicate API token, **shared** with the other Replicate skills (voice-clone,27 gpt-image-2, avatar-video-reel, …) and discovered automatically. To set/refresh:28 `python3 scripts/setup_key.py YOUR_REPLICATE_API_TOKEN`.2930## Inputs you need3132- **An avatar folder** (e.g. `lolo/`) — the one containing a `videos/` dir. It is33 auto-inferred from `--image`/`--audio`, or pass `--avatar-dir`.34- **A camera-angle image** — produce it first with `avatar-camera-angles`35 (a front-ish, clean portrait works best). Pass it with `--image`. If omitted,36 `<avatar>/frames/frame_0001.png` is used.37- **The text to say** (positional or `--text-file`). The avatar must have a38 trained voice, or pass `--source` so stage 1 trains one first.3940## Usage4142```bash43# Text -> cloned-voice audio -> talking-head video, from a camera angle44python3 scripts/generate_video.py "Hola, soy Lolo y te cuento algo." \45 --image lolo/angles/skill_test/lolo_push_in.png4647# 1080p + happy delivery + a custom visual prompt48python3 scripts/generate_video.py "Big news today!" \49 --image lolo/angles/skill_test/lolo_push_in.png \50 --resolution 1080p --emotion happy \51 --video-prompt "The person is talking and smiling warmly."5253# Reuse an existing mp3 (skip TTS) — just lip-sync it to the angle54python3 scripts/generate_video.py \55 --audio lolo/generated-audios/001_hola.mp3 \56 --image lolo/angles/skill_test/lolo_push_in.png5758# Avatar has no voice yet: train it first from a clean voice clip59python3 scripts/generate_video.py "Hello there" \60 --image lolo/angles/skill_test/lolo_push_in.png \61 --source lolo/videos/2026-05-16_12-25-46_voice/voice_concat.mp362```6364## Key options6566| Option | Default | Description |67|---|---|---|68| `text` / `--text-file` | — | What the avatar says (drives stage-1 TTS). |69| `--audio PATH` | — | Use this mp3/wav directly and **skip** TTS. |70| `--image PATH` | `<avatar>/frames/frame_0001.png` | Camera-angle / portrait image (`image` input). |71| `--avatar-dir` | auto | Avatar folder (else inferred from `--image`/`--audio`). |72| `--resolution` | `720p` | `720p` or `1080p` (1080p ≈ 2× the cost). |73| `--video-prompt` | `The person is talking.` | What the person is doing while speaking. |74| `--emotion` | `auto` | TTS delivery (forwarded to voice-clone). |75| `--language-boost` | `detect` | TTS language (auto-detected; forwarded). |76| `--voice-id` / `--name` / `--source` | — | Pick / train the voice (forwarded to voice-clone). |77| `--negative-prompt` | — | What to avoid (e.g. `subtitles, text, watermark`). |78| `--seed` | — | Reproducible generation. |79| `--disable-prompt-upsampling` | off | Use `--video-prompt` verbatim. |8081## Output (in `<avatar>/generated-videos/`)8283| File | What it is |84|------|------------|85| `<NNN>_<slug>.mp4` | The talking-head video (auto-numbered) |86| `manifest.json` | `items[]` mapping each video → `text`, `voice_id`, `image`, `audio`, `resolution`, `video_prompt`, and the rest of the params |8788Stage 1 also leaves the generated audio in `<avatar>/generated-audios/` (the89voice-clone skill's own output). Report the video path, the `voice_id` used, and90the resolution when done.9192## Notes9394- **`audio` overrides voice settings.** Because we always pass `audio`, the95 model's `voice` / `voice_script` / `voice_language` inputs are intentionally96 unused — lip-sync follows the cloned-voice mp3.97- **Cost** is per second of output: 720p ≈ $0.025/s, 1080p ≈ $0.045/s.98- **Image quality matters.** A clean, front-ish angle preserves identity best;99 heavy angles or occlusion hurt lip-sync and likeness.100- **Per-shot reels:** generate several angles with `avatar-camera-angles`, run101 this skill once per angle with the same text/voice, then stitch the clips.102103## Related skills104105- [`voice-clone`](../voice-clone/SKILL.md) — stage-1 TTS / voice training.106- [`avatar-camera-angles`](../avatar-camera-angles/SKILL.md) — produce the `image` input.107- [`avatar-frames`](../avatar-frames/SKILL.md) — extract a clean reference frame.
puntorigen