Deepgram Stt

Deepgram Speech-to-Text Skill

Industry-leading speech-to-text via Deepgram Nova-3. Sub-300ms streaming latency, 36 languages, smart formatting.

Capabilities

Authentication

All requests require the Authorization header with a Deepgram API key:

export DEEPGRAM_API_KEY="your-api-key"

Header format: Authorization: Token $DEEPGRAM_API_KEY

Pre-recorded Transcription

Endpoint: POST https://api.deepgram.com/v1/listen

Best for transcribing audio files, recordings, and batch processing.

From a URL

curl -X POST "https://api.deepgram.com/v1/listen?model=nova-3&smart_format=true" \
  -H "Authorization: Token $DEEPGRAM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://dpgr.am/bueller.wav"}'

From a Local File

curl -X POST "https://api.deepgram.com/v1/listen?model=nova-3&smart_format=true" \
  -H "Authorization: Token $DEEPGRAM_API_KEY" \
  -H "Content-Type: audio/wav" \
  --data-binary @/path/to/audio.wav

With Speaker Diarization

curl -X POST "https://api.deepgram.com/v1/listen?model=nova-3&smart_format=true&diarize=true" \
  -H "Authorization: Token $DEEPGRAM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://dpgr.am/bueller.wav"}'

With Punctuation and Paragraphs

curl -X POST "https://api.deepgram.com/v1/listen?model=nova-3&punctuate=true&paragraphs=true&utterances=true" \
  -H "Authorization: Token $DEEPGRAM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://dpgr.am/bueller.wav"}'

Query Parameters Reference

Model Selection

Formatting Options

Intelligence Features

Audio Configuration

Streaming-Specific Parameters

Streaming (WebSocket) Transcription

Endpoint: wss://api.deepgram.com/v1/listen

Best for real-time applications: voice assistants, live captioning, telephony.

Python Example (Real-time Microphone)

import asyncio
import websockets
import json
import os

async def stream_microphone():
    """Stream audio to Deepgram via WebSocket for real-time transcription."""
    url = "wss://api.deepgram.com/v1/listen?model=nova-3&smart_format=true&encoding=linear16&sample_rate=16000&endpointing=300"
    
    headers = {
        "Authorization": f"Token {os.environ['DEEPGRAM_API_KEY']}"
    }
    
    async with websockets.connect(url, extra_headers=headers) as ws:
        # Send audio chunks (from microphone, file, or Twilio stream)
        # Each chunk should be raw audio bytes
        
        async def send_audio():
            # Example: send audio from a file in chunks
            with open("audio.raw", "rb") as f:
                while chunk := f.read(4096):
                    await ws.send(chunk)
                    await asyncio.sleep(0.1)  # Simulate real-time
            # Signal end of audio
            await ws.send(json.dumps({"type": "CloseStream"}))
        
        async def receive_transcripts():
            async for msg in ws:
                result = json.loads(msg)
                if result.get("type") == "Results":
                    transcript = result["channel"]["alternatives"][0]["transcript"]
                    is_final = result["is_final"]
                    if transcript:
                        prefix = "FINAL" if is_final else "INTERIM"
                        print(f"[{prefix}] {transcript}")
        
        await asyncio.gather(send_audio(), receive_transcripts())

asyncio.run(stream_microphone())

Pipecat Integration Pattern

For voice AI pipelines using Pipecat with Deepgram STT:

from pipecat.services.deepgram import DeepgramSTTService
from pipecat.audio.vad.silero import SileroVADAnalyzer

# Create STT service for Pipecat pipeline
stt = DeepgramSTTService(
    api_key=os.environ["DEEPGRAM_API_KEY"],
    params=DeepgramSTTService.InputParams(
        model="nova-3",
        language="en",
        smart_format=True,
        endpointing=300,       # 300ms silence = end of utterance
        utterance_end_ms=1000, # Max 1s to finalize utterance
        vad_events=True,       # Voice activity detection
        interim_results=True,  # Stream partial results
    )
)

# Use in a Pipecat pipeline:
# Twilio Audio → VAD → Deepgram STT → LLM → TTS → Twilio Audio
pipeline = Pipeline([
    transport.input(),
    SileroVADAnalyzer(),
    stt,
    llm,
    tts,
    transport.output(),
])

Twilio Media Streams Integration

For receiving Twilio phone call audio and transcribing in real-time:

import asyncio
import websockets
import json
import base64
import os

async def handle_twilio_stream(twilio_ws):
    """Bridge Twilio Media Stream to Deepgram for real-time transcription."""
    deepgram_url = "wss://api.deepgram.com/v1/listen?model=nova-3&encoding=mulaw&sample_rate=8000&channels=1&smart_format=true&endpointing=300"
    
    headers = {"Authorization": f"Token {os.environ['DEEPGRAM_API_KEY']}"}
    
    async with websockets.connect(deepgram_url, extra_headers=headers) as dg_ws:
        async def forward_twilio_to_deepgram():
            async for message in twilio_ws:
                data = json.loads(message)
                if data["event"] == "media":
                    # Decode Twilio's base64 mulaw audio and forward to Deepgram
                    audio = base64.b64decode(data["media"]["payload"])
                    await dg_ws.send(audio)
                elif data["event"] == "stop":
                    await dg_ws.send(json.dumps({"type": "CloseStream"}))
                    break
        
        async def receive_deepgram_results():
            async for msg in dg_ws:
                result = json.loads(msg)
                if result.get("type") == "Results":
                    transcript = result["channel"]["alternatives"][0]["transcript"]
                    if transcript and result["is_final"]:
                        print(f"Caller said: {transcript}")
                        # Send to LLM for response generation
        
        await asyncio.gather(forward_twilio_to_deepgram(), receive_deepgram_results())

Response Format

Pre-recorded Response

{
  "metadata": {
    "transaction_key": "...",
    "request_id": "...",
    "sha256": "...",
    "created": "2024-01-01T00:00:00.000Z",
    "duration": 5.0,
    "channels": 1,
    "models": ["nova-3"],
    "model_info": {"nova-3": {"name": "nova-3", "version": "2025-01-01", "arch": "nova-3"}}
  },
  "results": {
    "channels": [{
      "alternatives": [{
        "transcript": "Hello, how are you today?",
        "confidence": 0.99,
        "words": [
          {"word": "hello", "start": 0.0, "end": 0.5, "confidence": 0.99, "speaker": 0},
          {"word": "how", "start": 0.6, "end": 0.8, "confidence": 0.98, "speaker": 0}
        ]
      }]
    }]
  }
}

Streaming Response

{
  "type": "Results",
  "channel_index": [0, 1],
  "duration": 1.5,
  "start": 0.0,
  "is_final": true,
  "speech_final": true,
  "channel": {
    "alternatives": [{
      "transcript": "Hello how are you",
      "confidence": 0.98,
      "words": [
        {"word": "hello", "start": 0.0, "end": 0.4, "confidence": 0.99}
      ]
    }]
  }
}

Supported Languages (36)

Error Handling

Cost Optimization

• Nova-3: ~$0.0043/min (pay-as-you-go) — best accuracy-to-cost ratio
• Nova-2: ~$0.0036/min — slightly cheaper, still excellent
• Streaming vs Pre-recorded: Same price per minute
• Multichannel: Billed per channel, use only when needed
• Smart format: Free, always enable for cleaner output
• Keywords: Use keywords param to boost domain-specific terms (free)

Quick Test

# Test pre-recorded transcription (uses Deepgram's sample audio)
curl -s -X POST "https://api.deepgram.com/v1/listen?model=nova-3&smart_format=true" \
  -H "Authorization: Token $DEEPGRAM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://dpgr.am/bueller.wav"}' | jq -r '.results.channels[0].alternatives[0].transcript'

# Expected output: A transcript of the Bueller movie audio clip
# If you see text output, your API key is working!

# Test with diarization (speaker identification)
curl -s -X POST "https://api.deepgram.com/v1/listen?model=nova-3&smart_format=true&diarize=true" \
  -H "Authorization: Token $DEEPGRAM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://dpgr.am/bueller.wav"}' | jq '.results.channels[0].alternatives[0].words[:5]'