Mlx Tts

MLX TTS (Qwen3-TTS)

High-quality local text-to-speech synthesis for Apple Silicon Macs using Qwen3-TTS via MLX framework. Optimized for Apple Neural Engine, runs entirely on-device.

Overview

Qwen3-TTS is a 1.7B parameter TTS model optimized for Apple Silicon (M1/M2/M3/M4) using the MLX framework. Features:

• Fast inference: Leverages Apple Neural Engine
• Prompt-based voice design: Describe the voice you want, no reference audio needed
• ASR support: Speech-to-text included
• Low memory: 8-bit quantized, runs on 16GB Mac
• Local only: No cloud, no API keys, fully private

When to Use

• Have an Apple Silicon Mac (M1/M2/M3/M4)
• Need high-quality TTS locally
• Want to design voices with text prompts (e.g., "a warm female voice, slightly soft")
• Need ASR (speech-to-text) capability
• Want fast inference with Apple Neural Engine

When NOT to Use

• Non-Mac system (Intel Mac or Windows/Linux) → Use other TTS solutions
• Need voice cloning from reference audio → Use other tools
• Less than 16GB RAM → May work but slower

Environment Setup

One-Command Install

brew install ffmpeg uv && uv tool install --force "mlx-audio" --prerelease=allow

Verify Installation

uv tool list | grep mlx
# Should show: mlx-audio v0.4.2

Model Download

Models auto-download on first run (2GB total) to `/.cache/huggingface/hub/`:

• mlx-community/Qwen3-TTS-12Hz-1.7B-VoiceDesign-8bit
• mlx-community/Qwen3-ASR-0.6B-bf16

For China users (accelerated download):

Option 1 - HuggingFace mirror:

export HF_ENDPOINT=https://hf-mirror.com

Option 2 - ModelScope (faster):

pip install modelscope

# Download TTS model
modelscope download \
  --model mlx-community/Qwen3-TTS-12Hz-1.7B-VoiceDesign-8bit \
  --local_dir ~/.cache/huggingface/hub/Qwen3-TTS

# Download ASR model
modelscope download \
  --model mlx-community/Qwen3-ASR-0.6B-bf16 \
  --local_dir ~/.cache/huggingface/hub/Qwen3-ASR

Usage

Basic TTS

mlx_audio.tts.generate \
  --text "你好，这是本地 TTS 测试" \
  --output-path ./output.wav

With Voice Design (Prompt-Based)

mlx_audio.tts.generate \
  --text "我是明日香" \
  --instruct "a confident teenage girl, flirtatious, seductive edge" \
  --output-path ./asuka.wav

Voice Prompt Examples

Advanced Voice Design Tips

Delivery Instructions (Qwen3-TTS supports these):

• "speak slowly and clearly"
• "whisper gently"
• "speak with excitement"
• "calm and soothing tone"
• "fast-paced, energetic delivery"

Combine multiple descriptors:

--instruct "a warm female voice, slightly soft, speak slowly with gentle pauses"

Long Text TTS (Auto-Chunking)

For texts longer than ~500 characters, use auto-chunking to avoid memory issues:

Option 1: Manual script

# Split text into sentences and generate separately
python3 << 'EOF'
import subprocess
import re

text = """Your long text here... Multiple sentences."""

# Split by sentence boundaries
sentences = re.split(r'(?<=[。！？.!?])\s+', text)
chunks = []
current_chunk = ""

for sent in sentences:
    if len(current_chunk) + len(sent) < 300:  # Max 300 chars per chunk
        current_chunk += sent
    else:
        if current_chunk:
            chunks.append(current_chunk)
        current_chunk = sent
if current_chunk:
    chunks.append(current_chunk)

# Generate each chunk
for i, chunk in enumerate(chunks):
    subprocess.run([
        "mlx_audio.tts.generate",
        "--text", chunk,
        "--instruct", "a warm, gentle female voice",
        "--output-path", f"./chunk_{i:03d}.wav"
    ])
print(f"Generated {len(chunks)} chunks. Use ffmpeg to concatenate.")
EOF

Option 2: Concatenate with ffmpeg

# After generating chunks, merge them
ffmpeg -i "concat:$(echo chunk_*.wav | tr ' ' '|')" -acodec copy final_output.wav

# Or with crossfade (smooth transition)
ffmpeg -f concat -safe 0 -i <(for f in chunk_*.wav; do echo "file '$PWD/$f'"; done) -c copy output.wav

ASR (Speech to Text)

mlx_audio.stt.generate \
  --audio ./input.wav \
  --output-path ./transcript.txt \
  --language zh

Long audio (auto-chunking):

mlx_audio.stt.generate \
  --audio /path/to/long_audio.m4a \
  --output-path ./transcript.txt \
  --format txt \
  --language zh \
  --chunk-duration 30

Batch Processing

Batch TTS from file list:

# Create text file with lines to synthesize
cat > texts.txt << 'EOF'
第一行要转换的文字
第二行要转换的文字
第三行要转换的文字
EOF

# Process each line
while IFS= read -r line; do
  safe_name=$(echo "$line" | tr -cd '[:alnum:]\n' | cut -c1-20)
  mlx_audio.tts.generate \
    --text "$line" \
    --instruct "a professional news anchor" \
    --output-path "./output/${safe_name}.wav"
done < texts.txt

Batch ASR:

# Batch convert all audio files in directory
for f in *.wav; do
  mlx_audio.stt.generate \
    --audio "$f" \
    --output-path "${f%.wav}.txt" \
    --language zh
done

Audio Post-Processing (with ffmpeg)

While mlx-audio doesn't have built-in effects like Voicebox, you can use ffmpeg:

Pitch Shift (音色调整):

# Raise pitch by 2 semitones (更尖声)
ffmpeg -i input.wav -af "asetrate=48000*1.12,aresample=48000" output_high.wav

# Lower pitch by 2 semitones (更低沉)
ffmpeg -i input.wav -af "asetrate=48000*0.89,aresample=48000" output_low.wav

Add Reverb (混响):

ffmpeg -i input.wav -af "aecho=0.8:0.9:1000:0.3" output_reverb.wav

Speed Control (语速):

# Speed up 1.2x (更快)
ffmpeg -i input.wav -af "atempo=1.2" output_fast.wav

# Slow down 0.8x (更慢)
ffmpeg -i input.wav -af "atempo=0.8" output_slow.wav

Volume Normalize (音量标准化):

ffmpeg -i input.wav -af "loudnorm" output_normalized.wav

Convert Format (格式转换):

# WAV to MP3
ffmpeg -i input.wav -b:a 192k output.mp3

# WAV to AAC (for Apple devices)
ffmpeg -i input.wav -c:a aac -b:a 192k output.m4a

# WAV to FLAC (lossless compression)
ffmpeg -i input.wav output.flac

Create Helper Script

Create ~/bin/tts.sh for quick TTS:

#!/bin/bash
TEXT="${1:-"Hello, Human!"}"
INSTRUCT="${2:-"a confident teenage girl with a flirtatious, seductive edge"}"
OUTPUT_DIR=./voice_output

mkdir -p "$OUTPUT_DIR"

mlx_audio.tts.generate \
  --text "$TEXT" \
  --instruct "$INSTRUCT" \
  --output-path "$OUTPUT_DIR/output.wav" \
  --audio-format wav

echo "Generated: $OUTPUT_DIR/output.wav"

Make executable and use:

chmod +x ~/bin/tts.sh

# Usage
tts.sh "要转换的文字"
tts.sh "要转换的文字" "a warm, gentle female voice"

Advanced Helper Script

Create ~/bin/tts-advanced.sh with more features:

#!/bin/bash
# TTS with auto-chunking for long texts

TEXT="${1:-"Hello"}"
VOICE="${2:-"a warm, gentle female voice"}"
OUTPUT="${3:-"./output.wav"}"
MAX_CHARS=300

# Check if text needs chunking
if [ ${#TEXT} -le $MAX_CHARS ]; then
  # Short text - direct generation
  mlx_audio.tts.generate \
    --text "$TEXT" \
    --instruct "$VOICE" \
    --output-path "$OUTPUT"
  echo "Generated: $OUTPUT"
else
  # Long text - need chunking
  echo "Text too long (${#TEXT} chars), auto-chunking..."
  
  # Create temp directory
  TMPDIR=$(mktemp -d)
  
  # Split by sentences (simplified)
  echo "$TEXT" | fold -w $MAX_CHARS -s | split -l 1 - "$TMPDIR/chunk_"
  
  # Generate each chunk
  i=0
  for chunk in "$TMPDIR"/chunk_*; do
    CHUNK_TEXT=$(cat "$chunk")
    [ -z "$CHUNK_TEXT" ] && continue
    
    mlx_audio.tts.generate \
      --text "$CHUNK_TEXT" \
      --instruct "$VOICE" \
      --output-path "$TMPDIR/part_$(printf "%03d" $i).wav"
    
    i=$((i+1))
  done
  
  # Concatenate with ffmpeg
  ffmpeg -f concat -safe 0 -i \
    <(for f in "$TMPDIR"/part_*.wav; do echo "file '$f'"; done) \
    -c copy "$OUTPUT"
  
  # Cleanup
  rm -rf "$TMPDIR"
  echo "Generated: $OUTPUT"
fi

Voicebox vs MLX-TTS Comparison

When to use Voicebox:

• Need GUI and visual timeline
• Need voice cloning from audio
• Need built-in effects
• Multi-platform support

When to use MLX-TTS:

• Apple Silicon Mac only
• Prefer CLI and scripting
• Quick setup (brew install)
• Lightweight solution

Quick Reference

Output Formats

TTS Output

• Default: WAV (48kHz, 16-bit)
• Options: --audio-format wav|mp3|flac

Note: mlx-audio outputs WAV by default. Convert to other formats with ffmpeg:

# High-quality MP3
ffmpeg -i input.wav -b:a 256k output.mp3

# AAC for Apple devices
ffmpeg -i input.wav -c:a aac -b:a 256k output.m4a

# FLAC for lossless
ffmpeg -i input.wav output.flac

# OGG Vorbis
ffmpeg -i input.wav -c:a libvorbis -q:a 6 output.ogg

ASR Output Formats

• --format txt: Plain text (default)
• --format json: JSON with timestamps
• --format srt: Subtitle format

Advanced Use Cases

Podcast/Audio Book Production

# Create consistent narrator voice
VOICE="a warm, articulate narrator, clear and engaging"

# Generate chapters
for chapter in {1..10}; do
  mlx_audio.tts.generate \
    --text "Chapter $chapter" \
    --instruct "a professional announcer, clear and authoritative" \
    --output-path "./podcast/chapter_${chapter}_title.wav"
  
  mlx_audio.tts.generate \
    --text "$(cat chapter_${chapter}.txt)" \
    --instruct "$VOICE" \
    --output-path "./podcast/chapter_${chapter}_content.wav"
done

# Merge with ffmpeg
ffmpeg -f concat -safe 0 -i <(for f in ./podcast/*.wav; do echo "file '$PWD/$f'"; done) -c copy audiobook.wav

Multi-Voice Dialogue

# Character A (confident young woman)
mlx_audio.tts.generate \
  --text "I'll take care of this." \
  --instruct "a confident teenage girl, slightly energetic" \
  --output-path ./voice_a.wav

# Character B (elderly wise man)
mlx_audio.tts.generate \
  --text "Be careful, my child." \
  --instruct "an elderly male voice, wise and gentle, slower pace" \
  --output-path ./voice_b.wav

# Character C (robot/AI)
mlx_audio.tts.generate \
  --text "Processing complete." \
  --instruct "a synthetic robotic voice, monotone, slightly metallic" \
  --output-path ./voice_c.wav

# Add robotic effect with ffmpeg
ffmpeg -i voice_c.wav -af "asetrate=48000*0.95,aresample=48000,aecho=0.6:0.4:300:0.5" voice_c_robot.wav

Voice Templates Library

Create reusable voice templates:

# Save templates in a file
cat > ~/voice_templates.txt << 'EOF'
narration=a warm, articulate narrator, clear and engaging, moderate pace
news=a professional news anchor, clear and authoritative, precise diction
friendly=a friendly, approachable voice, warm and inviting
teacher=a patient educator, clear and encouraging, moderate pace
storyteller=a mysterious storyteller, dramatic and engaging
calm=a soothing, meditative voice, very slow and gentle
excited=a highly energetic voice, fast-paced and enthusiastic
dramatic=a theatrical voice, expressive and emotional
EOF

# Usage
VOICE=$(grep "^narration=" ~/voice_templates.txt | cut -d'=' -f2)
mlx_audio.tts.generate \
  --text "Your text here" \
  --instruct "$VOICE" \
  --output-path output.wav

Troubleshooting

Common Issues

Performance Tips

1. First run is slow: Model downloads ~2GB on first use
2. Keep sentences together: Don't split mid-sentence
3. Simple prompts work better: Avoid overly complex instructions
4. Use --verbose: See detailed output for debugging

Debug Mode

# Verbose output
mlx_audio.tts.generate --verbose --text "Hello" --output test.wav

# Check model cache
ls -la ~/.cache/huggingface/hub/ | grep mlx

# Check disk space
df -h ~/.cache/huggingface/

# Test with minimal text
mlx_audio.tts.generate --text "Test" --output /tmp/test.wav

References

• MLX-Audio GitHub
• Qwen3-TTS Model
• Qwen3-ASR Model
• ModelScope
• HuggingFace Mirror