Speech-to-Text (STT)

Speech-to-Text (STT)

Overview

Speech-to-Text models transcribe audio files into text. They support various audio formats, languages, and can include features like timestamp generation, speaker identification, and translation.

Common Use Cases

• Transcription: Meeting notes, podcast transcripts, interview documentation
• Voice Interfaces: Voice commands, dictation, voice search
• Content Accessibility: Captions for videos, subtitles for media
• Audio Analysis: Call center analytics, compliance monitoring

Interface

Creating a Speech-to-Text Model

from esperanto.factory import AIFactory

# Basic usage
transcriber = AIFactory.create_speech_to_text(
    provider="openai",
    model_name="whisper-1"
)

# With configuration
transcriber = AIFactory.create_speech_to_text(
    provider="groq",
    model_name="whisper-large-v3",
    config={
        "timeout": 300.0,  # Longer timeout for large files
        "language": "en",
        "response_format": "json"
    }
)

Core Methods

transcribe(audio_file, **kwargs)

Synchronous audio transcription from file path.

# From file path
transcript = transcriber.transcribe("/path/to/audio.mp3")
print(transcript)

# From file object
with open("/path/to/audio.mp3", "rb") as f:
    transcript = transcriber.transcribe(f)
print(transcript)

atranscribe(audio_file, **kwargs)

Asynchronous transcription (identical interface to transcribe).

transcript = await transcriber.atranscribe("/path/to/audio.mp3")
print(transcript)

Parameters

Config Parameters

Method Parameters

Supported Audio Formats

Common supported formats (varies by provider):

• MP3: Most widely supported
• WAV: Uncompressed audio
• M4A: Apple audio format
• FLAC: Lossless compression
• OGG: Open format
• WebM: Web multimedia

Response Structure

Text Format (Default)

Most providers return plain text:

transcriber = AIFactory.create_speech_to_text("openai", "whisper-1")
transcript = transcriber.transcribe("audio.mp3")
print(transcript)
# Output: "This is the transcribed text from the audio file."

JSON Format

Request structured output:

transcriber = AIFactory.create_speech_to_text(
    "openai", "whisper-1",
    config={"response_format": "json"}
)

result = transcriber.transcribe("audio.mp3")
# Returns: {"text": "Transcribed content..."}

Verbose JSON Format

Get detailed information including timestamps:

transcriber = AIFactory.create_speech_to_text(
    "openai", "whisper-1",
    config={"response_format": "verbose_json"}
)

result = transcriber.transcribe("audio.mp3")
# Returns:
# {
#   "text": "Full transcription...",
#   "language": "en",
#   "duration": 125.5,
#   "segments": [
#     {"start": 0.0, "end": 3.2, "text": "First segment"},
#     ...
#   ]
# }

Subtitle Formats

Generate subtitle files directly:

# SRT format (SubRip)
transcriber = AIFactory.create_speech_to_text(
    "openai", "whisper-1",
    config={"response_format": "srt"}
)

srt_content = transcriber.transcribe("audio.mp3")
# Returns SRT format string

# VTT format (WebVTT)
transcriber = AIFactory.create_speech_to_text(
    "openai", "whisper-1",
    config={"response_format": "vtt"}
)

vtt_content = transcriber.transcribe("audio.mp3")
# Returns VTT format string

Provider Selection

→ See Provider Comparison for detailed comparison and selection guide.

Quick Provider Guide

• OpenAI: Industry standard Whisper model, excellent accuracy
• Groq: Fastest transcription, Whisper-based, low latency
• Google: Gemini audio transcription, supports multiple formats, simple API key auth
• Azure: Enterprise compliance, private deployment
• ElevenLabs: Specialized for voice, good multilingual support
• OpenAI-Compatible: Local Whisper deployment (faster-whisper, etc.)

Examples

Basic Transcription

from esperanto.factory import AIFactory

transcriber = AIFactory.create_speech_to_text("openai", "whisper-1")

# Transcribe audio file
transcript = transcriber.transcribe("meeting_recording.mp3")
print(transcript)

With Language Specification

transcriber = AIFactory.create_speech_to_text(
    "openai", "whisper-1",
    config={"language": "es"}  # Spanish
)

transcript = transcriber.transcribe("spanish_audio.mp3")
print(transcript)

Generate Subtitles

transcriber = AIFactory.create_speech_to_text(
    "openai", "whisper-1",
    config={"response_format": "srt"}
)

# Get SRT subtitle content
srt_content = transcriber.transcribe("video_audio.mp3")

# Save to file
with open("subtitles.srt", "w") as f:
    f.write(srt_content)

With Context Prompt

transcriber = AIFactory.create_speech_to_text("groq", "whisper-large-v3")

# Provide context for better accuracy
prompt = "This is a technical discussion about machine learning and neural networks."

transcript = transcriber.transcribe(
    "tech_talk.mp3",
    prompt=prompt
)
print(transcript)

Google (Gemini) Transcription

# Basic Gemini transcription
transcriber = AIFactory.create_speech_to_text("google", "gemini-2.5-flash")

transcript = transcriber.transcribe("audio.mp3")
print(transcript.text)

# With language hint for better accuracy
transcript = transcriber.transcribe(
    "portuguese_audio.mp3",
    language="pt"
)
print(transcript.text)

# With custom prompt for specialized content
transcript = transcriber.transcribe(
    "medical_consultation.mp3",
    prompt="This is a medical consultation. Focus on medical terminology and patient symptoms."
)
print(transcript.text)

Note: Esperanto's Google STT provider uses Gemini API's audio transcription capabilities, not Cloud Speech-to-Text API v2 (Chirp 3). This provides simpler authentication (API key only) and consistent integration with other Google GenAI features. Supported formats: MP3, WAV, AIFF, AAC, OGG, FLAC.

Async Batch Processing

transcriber = AIFactory.create_speech_to_text(
    "openai", "whisper-1",
    config={"timeout": 600.0}  # 10 minutes for large files
)

audio_files = [
    "episode_01.mp3",
    "episode_02.mp3",
    "episode_03.mp3"
]

async def transcribe_all():
    tasks = [
        transcriber.atranscribe(file)
        for file in audio_files
    ]
    return await asyncio.gather(*tasks)

transcripts = await transcribe_all()
for i, transcript in enumerate(transcripts, 1):
    print(f"\n=== Episode {i} ===")
    print(transcript[:200], "...")

Detailed Transcription with Timestamps

transcriber = AIFactory.create_speech_to_text(
    "openai", "whisper-1",
    config={"response_format": "verbose_json"}
)

result = transcriber.transcribe("interview.mp3")

print(f"Language detected: {result['language']}")
print(f"Duration: {result['duration']:.2f} seconds")
print("\nSegments:")

for segment in result['segments']:
    start = segment['start']
    end = segment['end']
    text = segment['text']
    print(f"[{start:.2f}s - {end:.2f}s] {text}")

Error Handling

from esperanto.factory import AIFactory

transcriber = AIFactory.create_speech_to_text("openai", "whisper-1")

try:
    transcript = transcriber.transcribe("audio.mp3")
    print(transcript)
except FileNotFoundError:
    print("Audio file not found")
except Exception as e:
    print(f"Transcription error: {e}")

Best Practices

File Size Limits

Most providers have file size limits (typically 25MB). For larger files:

# Option 1: Split audio file before transcription
# Use ffmpeg or similar tool to split into chunks

# Option 2: Use streaming if supported by provider

# Option 3: Compress audio file
# ffmpeg -i input.wav -ar 16000 -ac 1 output.mp3

Audio Quality Recommendations

• Sample rate: 16kHz minimum for speech
• Format: MP3 or M4A for good compression
• Mono vs Stereo: Mono sufficient for speech, saves bandwidth
• Bitrate: 64-128 kbps adequate for voice

Accuracy Tips

1. Clear audio: Reduce background noise before transcription
2. Specify language: Better accuracy than auto-detection
3. Use prompts: Provide context for technical terms or names
4. Choose right model: Larger models (e.g., whisper-large) for difficult audio

Performance Considerations

• Timeout: Allow 1-2 minutes per minute of audio minimum
• Async processing: Use atranscribe() for multiple files
• Local deployment: OpenAI-Compatible with faster-whisper for best speed

Advanced Topics

• Timeout Configuration: docs/advanced/timeout-configuration.md
• Resource Management: docs/advanced/connection-resource-management.md

See Also

• Provider Setup Guides
• Text-to-Speech
• Language Models