Media Tts AI

Media TTS AI

Context: $ARGUMENTS

Quick start

• Default TTS (no cloning), best quality-for-size: scripts/tts.py speak --model kokoro --text "Hello" --voice af_bella --out out.wav → Step 3
• Voice clone from a 5-10s sample: scripts/tts.py clone --model openvoice --reference ref.wav --text "..." --out out.wav → Step 4
• Audiobook from a text/markdown file: scripts/tts.py audiobook --text-file book.md --model kokoro --voice af_bella --chapter-split --out-dir chapters/ → Step 5
• Realtime embedded (lowest latency, local): scripts/tts.py speak --model piper --text "Hi" --voice en_US-amy-medium --out out.wav
• Expressive + non-verbal ([laughs], [sighs]): scripts/tts.py speak --model bark --text "Nice. [laughs]" --out out.wav

When to use

• Generate narration, voiceover, or podcast intros from text
• Clone a specific voice from a short reference clip (OpenVoice / CosyVoice / Chatterbox)
• Build audiobooks with chapter splits from markdown / plain text
• Replace Apple's say or eSpeak with neural-quality TTS
• Batch-dub SRT lines (time-aligned per cue) for localization
• Do NOT use for: lip-sync to a face video — that's media-lipsync. Transcription (opposite direction) is media-whisper.

Step 1 — Pick a model

All nine models below are open-source + commercial-safe. Pick by task:

Full comparison matrix (quality / speed / GPU req / clone-support / emotion-support) in references/models.md. License proofs in references/LICENSES.md.

DO NOT use: XTTS-v2 (Coqui CPML — non-commercial), F5-TTS (CC-BY-NC weights), MusicGen (CC-BY-NC weights), Suno-proprietary Bark v2 derivatives. This skill explicitly refuses those.

Step 2 — Install

# Default path (Kokoro ONNX — CPU-friendly, no torch required)
pip install kokoro-onnx onnxruntime soundfile

# Or the PyTorch variant (bigger, supports more voices)
pip install kokoro soundfile

# Piper (ONNX, <50 MB for a voice)
pip install piper-tts

# Voice cloning (pick one)
pip install openvoice-cli                     # OpenVoice v2 CLI wrapper
pip install "funasr[llm]" modelscope          # CosyVoice 2 via ModelScope
pip install chatterbox-tts                    # Chatterbox
pip install suno-bark                         # Bark (Apache 2.0 weights only)
pip install orpheus-speech                    # Orpheus
pip install parler-tts                        # Parler-TTS

# StyleTTS2 (builds from source)
pip install styletts2

Or let the script bootstrap for you:

uv run ${CLAUDE_SKILL_DIR}/scripts/tts.py install kokoro

Check what's available:

uv run ${CLAUDE_SKILL_DIR}/scripts/tts.py check

Step 3 — Synthesize speech (no cloning)

# Kokoro (default)
uv run ${CLAUDE_SKILL_DIR}/scripts/tts.py speak \
    --model kokoro \
    --text "Hello world, this is a test." \
    --voice af_bella \
    --lang en-us \
    --out out.wav

# Piper (fastest)
uv run ${CLAUDE_SKILL_DIR}/scripts/tts.py speak \
    --model piper \
    --text "Hello" \
    --voice en_US-amy-medium \
    --out out.wav

# Bark (expressive)
uv run ${CLAUDE_SKILL_DIR}/scripts/tts.py speak \
    --model bark \
    --text "Hmm, interesting. [laughs] That's wild." \
    --voice v2/en_speaker_6 \
    --out out.wav

List available voices per model:

uv run ${CLAUDE_SKILL_DIR}/scripts/tts.py list-voices --model kokoro
uv run ${CLAUDE_SKILL_DIR}/scripts/tts.py list-voices --model piper

See references/voices.md for the ~50 Kokoro voice names and language codes.

Step 4 — Voice cloning

A clean 5-10 s reference clip of the target voice works best. Mono, 16-24 kHz, minimal background noise.

# OpenVoice v2
uv run ${CLAUDE_SKILL_DIR}/scripts/tts.py clone \
    --model openvoice \
    --reference speaker_ref.wav \
    --text "Text to say in the cloned voice." \
    --lang en \
    --out out.wav

# CosyVoice 2 (SOTA 2025)
uv run ${CLAUDE_SKILL_DIR}/scripts/tts.py clone \
    --model cosyvoice \
    --reference ref.wav \
    --text "Hello in the cloned voice." \
    --out out.wav

# Chatterbox with emotion intensity
uv run ${CLAUDE_SKILL_DIR}/scripts/tts.py clone \
    --model chatterbox \
    --reference ref.wav \
    --text "I'm actually pretty excited about this!" \
    --emotion 0.8 \
    --out out.wav

Ethical note: only clone voices you have explicit consent to clone. Logging the reference path is recommended for provenance.

Step 5 — Batch / audiobook / SRT dub

# Audiobook: split on `# heading` into chapter files
uv run ${CLAUDE_SKILL_DIR}/scripts/tts.py audiobook \
    --text-file book.md \
    --model kokoro \
    --voice af_bella \
    --chapter-split \
    --out-dir chapters/

# Batch: each SRT cue → aligned WAV inside out-dir/
uv run ${CLAUDE_SKILL_DIR}/scripts/tts.py batch \
    --script script.srt \
    --model kokoro \
    --voice af_bella \
    --out-dir dub/

# Or a plain text file, one line = one utterance
uv run ${CLAUDE_SKILL_DIR}/scripts/tts.py batch \
    --script lines.txt \
    --model piper \
    --voice en_US-amy-medium \
    --out-dir lines/

Step 6 — Post-process and mux

• Loudness normalization: hand off to media-ffmpeg-normalize (EBU R128, -16 LUFS for podcasts, -23 LUFS for broadcast).
• Format conversion: ffmpeg -i out.wav -c:a libopus -b:a 64k out.opus
• Replace audio track in a video: ffmpeg -i video.mp4 -i out.wav -map 0:v -map 1:a -c:v copy -shortest dubbed.mp4
• Lip-sync to an existing face video: hand off to media-lipsync (LivePortrait / LatentSync).

Gotchas

• License boundaries matter. XTTS-v2, F5-TTS, Suno's paid Bark v2, and MusicGen weights are NOT commercial-safe. This skill intentionally excludes them. See references/LICENSES.md.
• Kokoro ONNX vs PyTorch: kokoro-onnx runs CPU-only without torch; the PyTorch kokoro package adds more voices but requires torch + CUDA for speed. The script auto-picks ONNX first.
• Piper voice files are per-voice — download from huggingface.co/rhasspy/piper-voices. Each voice is two files: .onnx + .onnx.json. Path to .onnx is the voice argument.
• Bark hard-cuts at ~13s per generation. Longer text needs sentence-level chunking; the audiobook and batch subcommands do this automatically.
• OpenVoice v2 needs a base TTS (it does tone-color transfer, not end-to-end synthesis). The wrapper handles this by using a MeloTTS base model under the hood.
• CosyVoice 2 models are 1-3GB. First run downloads; subsequent runs are cached in ~/.cache/modelscope/.
• Chatterbox emotion intensity range is 0.0 to 1.0 (higher = more expressive). Default 0.5 feels natural; 0.8+ can sound cartoonish.
• Bark prompts: non-verbal cues are bracketed: [laughs], [sighs], [music], [gasps], [clears throat]. Music notes like ♪ la la la ♪ trigger sung output (unreliable).
• Sample rate: Kokoro outputs 24 kHz, Piper 16-22 kHz (voice-dependent), Bark 24 kHz, CosyVoice 22.05 kHz. If mixing voices in a DAW, resample all to 48 kHz first.
• No interactive prompts anywhere. All models run non-interactive via the script.
• Hallucinated words on long sentences: split text at sentence boundaries; the batch/audiobook paths do this with NLTK-free regex (.!? splits).
• GPU vs CPU: Kokoro / Piper / StyleTTS2 run realtime on CPU. Bark / CosyVoice / Orpheus / Parler-TTS need a GPU for near-realtime; on CPU they run at 0.1-0.3x realtime.
• Apple Silicon MPS: most of these libraries have partial MPS support. If you hit errors, force CPU: --device cpu.
• Licensing of generated audio: you own the output under the model's license — but if you cloned a voice, make sure you have rights to that speaker's voice. Written consent for any voice clone used commercially.
• Language coverage: Kokoro ships en-us / en-gb / fr / ja / ko / zh / hi / it / pt-br. Other models vary — see references/models.md.

Examples

Example 1: English podcast intro from a single line

uv run ${CLAUDE_SKILL_DIR}/scripts/tts.py speak \
    --model kokoro \
    --text "Welcome to the show. Today we're talking about open-source AI." \
    --voice af_bella \
    --out intro.wav
ffmpeg -i intro.wav -c:a libopus -b:a 64k intro.opus

Example 2: Clone a speaker for an accessibility narration

# Record a 10s clean reference clip of the speaker
ffmpeg -f avfoundation -i ":0" -t 10 -ac 1 -ar 24000 ref.wav

# Clone with CosyVoice 2
uv run ${CLAUDE_SKILL_DIR}/scripts/tts.py clone \
    --model cosyvoice \
    --reference ref.wav \
    --text "This is the accessibility narration in the cloned voice." \
    --out narration.wav

Example 3: Audiobook from a Markdown book, split on # Chapter

uv run ${CLAUDE_SKILL_DIR}/scripts/tts.py audiobook \
    --text-file manuscript.md \
    --model kokoro \
    --voice af_bella \
    --chapter-split \
    --out-dir book/
# -> book/01-introduction.wav, book/02-chapter-one.wav, ...

Example 4: Dub an SRT into aligned WAV cues

uv run ${CLAUDE_SKILL_DIR}/scripts/tts.py batch \
    --script subs.srt \
    --model kokoro \
    --voice am_michael \
    --out-dir dub/
# -> dub/0001-00-00-00-00-000.wav, dub/0002-...wav, with timing in dub/manifest.json

Troubleshooting

Error: ModuleNotFoundError: No module named 'kokoro_onnx'

Cause: Default path isn't installed. Solution: pip install kokoro-onnx onnxruntime soundfile or run scripts/tts.py install kokoro.

Error: Voice not found: af_bella

Cause: Kokoro voice pack wasn't downloaded or the name is misspelled. Solution: scripts/tts.py list-voices --model kokoro to see the exact names; kokoro-onnx fetches voices on first use from HuggingFace (network required once).

Output sounds robotic / metallic

Cause: Using a smaller Piper voice (-low / -x_low) or CPU-quantized Kokoro. Solution: switch to a -medium or -high Piper voice; for Kokoro use the full PyTorch package instead of ONNX-int8.

Voice clone sounds nothing like the reference

Cause: reference clip is noisy, too short, or has music under it. Solution: pre-process with media-denoise-ai enhance ref.wav clean_ref.wav, then re-clone. Also try CosyVoice 2 which is SOTA 2025 for zero-shot cloning.

Bark cuts off mid-sentence

Cause: Bark's 13s hard limit per generation. Solution: use audiobook or batch subcommand — splits on sentence boundaries and concatenates.

CUDA out of memory on Bark / CosyVoice / Orpheus

Cause: VRAM pressure. Solution: --device cpu (slower), or free other GPU processes; Bark has a SUNO_USE_SMALL_MODELS=1 env var for an 8GB-VRAM mode.

Output file is silent

Cause: text was empty after preprocessing (e.g. only emoji), or model errored silently. Solution: rerun with --verbose to see the exact command and the model's stderr; confirm soundfile.info(out.wav) shows a nonzero duration.

Reference docs

• references/models.md — full comparison: license, clone support, emotion control, speed, GPU need, languages.
• references/voices.md — ~50 Kokoro voice names, Piper naming convention, Bark prompts, language codes.
• references/LICENSES.md — strict table confirming every model shipped here is commercial-safe. XTTS-v2 / F5-TTS / MusicGen are explicitly excluded.