Text-to-Speech (TTS)

Text-to-Speech (TTS)

Overview

Text-to-Speech models convert written text into natural-sounding audio. They support multiple voices, languages, and audio formats, enabling applications from voice assistants to audiobook generation.

Common Use Cases

• Voice Interfaces: Virtual assistants, smart speakers, IVR systems
• Content Accessibility: Audiobooks, screen readers, narration
• Media Production: Voice-overs, podcast intro/outros, video narration
• Notifications: Voice alerts, announcements, voice-enabled apps

Interface

Creating a Text-to-Speech Model

from esperanto.factory import AIFactory

# Basic usage
speaker = AIFactory.create_text_to_speech(
    provider="openai",
    model_name="tts-1"
)

# With configuration
speaker = AIFactory.create_text_to_speech(
    provider="elevenlabs",
    model_name="eleven_multilingual_v2",
    config={
        "timeout": 300.0,
        "voice": "default"
    }
)

Core Methods

generate_speech(text, voice=None, **kwargs)

Synchronous speech generation returning audio bytes.

# Generate audio bytes
audio_bytes = speaker.generate_speech(
    text="Hello, this is a test of text to speech.",
    voice="alloy"  # Provider-specific voice
)

# Save to file
with open("output.mp3", "wb") as f:
    f.write(audio_bytes)

agenerate_speech(text, voice=None, **kwargs)

Asynchronous speech generation (identical interface to generate_speech).

audio_bytes = await speaker.agenerate_speech(
    text="Async speech generation",
    voice="shimmer"
)

Parameters

Config Parameters

Method Parameters

Available Voices

Voices vary significantly by provider. Common patterns:

OpenAI Voices

• alloy: Neutral, balanced
• echo: Male, clear
• fable: Expressive, storytelling
• onyx: Deep, authoritative
• nova: Energetic, friendly
• shimmer: Warm, conversational

ElevenLabs Voices

• Custom voices via voice ID
• Pre-made voices library
• Voice cloning available

Google/Vertex Voices

• Language and gender variants
• Neural voices: en-US-Neural2-A, en-US-Neural2-C, etc.
• Standard voices: en-US-Standard-A, etc.

→ See individual Provider Setup Guides for complete voice lists.

Response Structure

All TTS providers return audio bytes directly:

speaker = AIFactory.create_text_to_speech("openai", "tts-1")

# Returns bytes object containing audio data
audio_bytes = speaker.generate_speech("Hello world", voice="alloy")

# Save directly to file
with open("speech.mp3", "wb") as f:
    f.write(audio_bytes)

# Or process in memory
# ... audio processing libraries can use audio_bytes directly

Audio Formats

Supported Formats (Provider-Dependent)

# Specify format in config
speaker = AIFactory.create_text_to_speech(
    "openai", "tts-1",
    config={"response_format": "opus"}
)

audio = speaker.generate_speech("Test audio", voice="alloy")

Provider Selection

→ See Provider Comparison for detailed comparison and selection guide.

Quick Provider Guide

• OpenAI: High quality, natural voices, good pricing
• ElevenLabs: Best voice quality, voice cloning, emotional control
• Google: Wide language support, neural voices, good quality
• Azure: Enterprise compliance, custom neural voices
• OpenAI-Compatible: Local deployment options

Examples

Basic Speech Generation

from esperanto.factory import AIFactory

speaker = AIFactory.create_text_to_speech("openai", "tts-1")

text = "Welcome to Esperanto text to speech."
audio = speaker.generate_speech(text, voice="nova")

with open("welcome.mp3", "wb") as f:
    f.write(audio)

print("Audio saved to welcome.mp3")

Multiple Voices Comparison

speaker = AIFactory.create_text_to_speech("openai", "tts-1")

text = "This is how I sound."
voices = ["alloy", "echo", "fable", "onyx", "nova", "shimmer"]

for voice in voices:
    audio = speaker.generate_speech(text, voice=voice)
    with open(f"sample_{voice}.mp3", "wb") as f:
        f.write(audio)

print(f"Generated {len(voices)} audio samples")

Adjust Speech Speed

speaker = AIFactory.create_text_to_speech(
    "openai", "tts-1",
    config={"speed": 1.5}  # 1.5x faster
)

text = "This speech will be generated at 1.5 times normal speed."
audio = speaker.generate_speech(text, voice="alloy")

with open("fast_speech.mp3", "wb") as f:
    f.write(audio)

Different Audio Formats

from esperanto.factory import AIFactory

text = "Testing different audio formats."

# High quality for editing
speaker_flac = AIFactory.create_text_to_speech(
    "openai", "tts-1-hd",
    config={"response_format": "flac"}
)
audio_flac = speaker_flac.generate_speech(text, voice="alloy")

# Compressed for streaming
speaker_opus = AIFactory.create_text_to_speech(
    "openai", "tts-1",
    config={"response_format": "opus"}
)
audio_opus = speaker_opus.generate_speech(text, voice="alloy")

# Save both
with open("high_quality.flac", "wb") as f:
    f.write(audio_flac)

with open("compressed.opus", "wb") as f:
    f.write(audio_opus)

Async Batch Generation

speaker = AIFactory.create_text_to_speech("openai", "tts-1")

texts = [
    "Chapter 1: Introduction",
    "Chapter 2: Getting Started",
    "Chapter 3: Advanced Topics"
]

async def generate_all():
    tasks = [
        speaker.agenerate_speech(text, voice="fable")
        for text in texts
    ]
    return await asyncio.gather(*tasks)

audio_files = await generate_all()

# Save each chapter
for i, audio in enumerate(audio_files, 1):
    with open(f"chapter_{i}.mp3", "wb") as f:
        f.write(audio)

Long Text Processing

speaker = AIFactory.create_text_to_speech(
    "openai", "tts-1",
    config={"timeout": 600.0}  # 10 minutes for long texts
)

# Split long text into chunks (most providers have length limits)
def split_text(text, max_length=4000):
    """Split text into chunks at sentence boundaries."""
    sentences = text.split(". ")
    chunks = []
    current_chunk = ""

    for sentence in sentences:
        if len(current_chunk) + len(sentence) < max_length:
            current_chunk += sentence + ". "
        else:
            chunks.append(current_chunk.strip())
            current_chunk = sentence + ". "

    if current_chunk:
        chunks.append(current_chunk.strip())

    return chunks

long_text = "..."  # Your long text here
chunks = split_text(long_text)

audio_chunks = []
for i, chunk in enumerate(chunks):
    print(f"Processing chunk {i+1}/{len(chunks)}")
    audio = speaker.generate_speech(chunk, voice="alloy")
    audio_chunks.append(audio)

# Combine audio chunks (requires audio processing library)
# Example with pydub:
# from pydub import AudioSegment
# combined = AudioSegment.empty()
# for audio_bytes in audio_chunks:
#     segment = AudioSegment.from_mp3(io.BytesIO(audio_bytes))
#     combined += segment
# combined.export("complete.mp3", format="mp3")

Stream and Play Audio

import io
from esperanto.factory import AIFactory

speaker = AIFactory.create_text_to_speech("openai", "tts-1")

text = "This audio will be played directly."
audio_bytes = speaker.generate_speech(text, voice="nova")

# Play using pygame (example)
# import pygame
# pygame.mixer.init()
# sound = pygame.mixer.Sound(io.BytesIO(audio_bytes))
# sound.play()

# Or use another audio library
# import sounddevice as sd
# import soundfile as sf
# audio, samplerate = sf.read(io.BytesIO(audio_bytes))
# sd.play(audio, samplerate)

Error Handling

from esperanto.factory import AIFactory

speaker = AIFactory.create_text_to_speech("openai", "tts-1")

try:
    audio = speaker.generate_speech(
        "Test text",
        voice="alloy"
    )
    with open("output.mp3", "wb") as f:
        f.write(audio)
except ValueError as e:
    print(f"Invalid parameter: {e}")
except Exception as e:
    print(f"Generation error: {e}")

Best Practices

Text Length Limits

Most providers limit input text length (typically 4096 characters for OpenAI). For longer content:

1. Split at sentence boundaries: Maintain natural speech flow
2. Process in chunks: Generate multiple audio files
3. Combine audio: Use audio processing library to merge

Voice Selection

• Consistency: Use same voice for related content
• Audience: Match voice characteristics to target audience
• Context: Formal voices for professional content, friendly for casual
• Test voices: Generate samples to find best fit

Audio Quality vs File Size

# Production/archival: Use HD models and lossless formats
speaker = AIFactory.create_text_to_speech(
    "openai", "tts-1-hd",
    config={"response_format": "flac"}
)

# Streaming/mobile: Use standard models and compressed formats
speaker = AIFactory.create_text_to_speech(
    "openai", "tts-1",
    config={"response_format": "opus"}
)

Performance Considerations

• Timeout: Allow 1-2 seconds per sentence minimum
• Async processing: Use agenerate_speech() for multiple texts
• Caching: Cache generated audio for repeated text
• Batch processing: Process multiple texts in parallel

Advanced Topics

• Timeout Configuration: docs/advanced/timeout-configuration.md
• Resource Management: docs/advanced/connection-resource-management.md

See Also

• Provider Setup Guides
• Speech-to-Text
• Language Models