Audio Reply

Audio Reply Skill

Generate spoken audio responses using MLX Audio TTS (chatterbox-turbo model).

Trigger Phrases

• "read it to me [URL]" - Fetch content from URL and read it aloud
• "talk to me [topic/question]" - Generate a conversational response as audio
• "speak", "say it", "voice reply" - Convert your response to audio

How to Use

Mode 1: Read URL Content

User: read it to me https://example.com/article

1. Fetch the URL content using WebFetch
2. Extract readable text (strip HTML, focus on main content)
3. Generate audio using TTS
4. Play the audio and delete the file afterward

Mode 2: Conversational Audio Response

User: talk to me about the weather today

1. Generate a natural, conversational response
2. Keep it concise (TTS works best with shorter segments)
3. Convert to audio, play it, then delete the file

Implementation

TTS Command

uv run mlx_audio.tts.generate \
  --model mlx-community/chatterbox-turbo-fp16 \
  --text "Your text here" \
  --play \
  --file_prefix /tmp/audio_reply

Key Parameters

• --model mlx-community/chatterbox-turbo-fp16 - Fast, natural voice
• --play - Auto-play the generated audio
• --file_prefix - Save to temp location for cleanup
• --exaggeration 0.3 - Optional: add expressiveness (0.0-1.0)
• --speed 1.0 - Adjust speech rate if needed

Text Preparation Guidelines

For "read it to me" mode:

1. Fetch URL with WebFetch tool
2. Extract main content, strip navigation/ads/boilerplate
3. Summarize if very long (>500 words) - keep key points
4. Add natural pauses with periods and commas

For "talk to me" mode:

1. Write conversationally, as if speaking
2. Use contractions (I'm, you're, it's)
3. Add filler words sparingly for naturalness ([chuckle], um, anyway)
4. Keep responses under 200 words for best quality
5. Avoid technical jargon unless explaining it

Audio Generation & Cleanup (IMPORTANT)

Always delete the audio file after playing - it's already in the chat history.

# Generate with unique filename and play
OUTPUT_FILE="/tmp/audio_reply_$(date +%s)"
uv run mlx_audio.tts.generate \
  --model mlx-community/chatterbox-turbo-fp16 \
  --text "Your response text" \
  --play \
  --file_prefix "$OUTPUT_FILE"

# ALWAYS clean up after playing
rm -f "${OUTPUT_FILE}"*.wav 2>/dev/null

Error Handling

If TTS fails:

1. Check if model is downloaded (first run downloads ~500MB)
2. Ensure uv is installed and in PATH
3. Fall back to text response with apology

Example Workflows

Example 1: Read URL

User: read it to me https://blog.example.com/new-feature

Assistant actions:
1. WebFetch the URL
2. Extract article content
3. Generate TTS:
   uv run mlx_audio.tts.generate \
     --model mlx-community/chatterbox-turbo-fp16 \
     --text "Here's what I found... [article summary]" \
     --play --file_prefix /tmp/audio_reply_1706123456
4. Delete: rm -f /tmp/audio_reply_1706123456*.wav
5. Confirm: "Done reading the article to you."

Example 2: Talk to Me

User: talk to me about what you can help with

Assistant actions:
1. Generate conversational response text
2. Generate TTS:
   uv run mlx_audio.tts.generate \
     --model mlx-community/chatterbox-turbo-fp16 \
     --text "Hey! So I can help you with all kinds of things..." \
     --play --file_prefix /tmp/audio_reply_1706123789
3. Delete: rm -f /tmp/audio_reply_1706123789*.wav
4. (No text output needed - audio IS the response)

Notes

• First run may take longer as the model downloads (~500MB)
• Audio quality is best for English; other languages may vary
• For long content, consider chunking into multiple audio segments
• The --play flag uses system audio - ensure volume is up