# Domain Audio

> Domain: Audio Processing

- Skill: `zhanghandong/domain-audio` (Agent Skill)
- Install (CLI): `npx skillmds@latest add zhanghandong/domain-audio`
- Raw SKILL.md: https://api.skillmd.com/api/skills/zhanghandong/domain-audio/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: ZhangHanDong (https://skillmd.com/u/zhanghandong)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/zhanghandong/domain-audio

---


# Domain: Audio Processing

> Building audio and speech applications with dora-rs

## Overview

Dora supports audio processing pipelines including:
- Speech-to-Text (STT)
- Text-to-Speech (TTS)
- Voice Activity Detection (VAD)
- Real-time audio streaming

## Common Audio Nodes

### Speech-to-Text (Whisper)

```yaml
- id: stt
  build: pip install dora-distil-whisper
  path: dora-distil-whisper
  inputs:
    audio: microphone/audio
  outputs:
    - text
  env:
    MODEL: base                 # tiny, base, small, medium, large
    LANGUAGE: en
```

### Text-to-Speech (Kokoro)

```yaml
- id: tts
  build: pip install dora-kokoro-tts
  path: dora-kokoro-tts
  inputs:
    text: llm/response
  outputs:
    - audio
  env:
    VOICE: af_bella
```

### Voice Activity Detection

```yaml
- id: vad
  build: pip install dora-vad
  path: dora-vad
  inputs:
    audio: microphone/audio
  outputs:
    - speech_segments
    - is_speaking
```

## Complete Voice Assistant Pipeline

```yaml
nodes:
  # Microphone input
  - id: microphone
    build: pip install dora-microphone
    path: dora-microphone
    inputs:
      tick: dora/timer/millis/100
    outputs:
      - audio
    env:
      SAMPLE_RATE: "16000"
      CHANNELS: "1"

  # Voice activity detection
  - id: vad
    build: pip install dora-vad
    path: dora-vad
    inputs:
      audio: microphone/audio
    outputs:
      - speech_audio
      - is_speaking

  # Speech to text
  - id: stt
    build: pip install dora-distil-whisper
    path: dora-distil-whisper
    inputs:
      audio: vad/speech_audio
    outputs:
      - text
    env:
      MODEL: base

  # Language model
  - id: llm
    build: pip install dora-qwen
    path: dora-qwen
    inputs:
      text: stt/text
    outputs:
      - response

  # Text to speech
  - id: tts
    build: pip install dora-kokoro-tts
    path: dora-kokoro-tts
    inputs:
      text: llm/response
    outputs:
      - audio

  # Audio playback
  - id: speaker
    build: pip install dora-pyaudio
    path: dora-pyaudio
    inputs:
      audio: tts/audio
```

## Custom Audio Node Example

### Microphone Capture

```python
# microphone_node.py
import numpy as np
import sounddevice as sd
from dora import Node

node = Node()

# Audio parameters
SAMPLE_RATE = 16000
CHANNELS = 1
CHUNK_SIZE = 1024

def audio_callback(indata, frames, time, status):
    if status:
        print(status)
    # Store audio data
    audio_buffer.append(indata.copy())

audio_buffer = []

with sd.InputStream(
    samplerate=SAMPLE_RATE,
    channels=CHANNELS,
    callback=audio_callback,
    blocksize=CHUNK_SIZE
):
    for event in node:
        if event["type"] == "INPUT" and event["id"] == "tick":
            if audio_buffer:
                # Combine and send audio chunks
                audio = np.concatenate(audio_buffer, axis=0)
                audio_buffer.clear()
                node.send_output("audio", audio.flatten())

        elif event["type"] == "STOP":
            break
```

### Speech Recognition

```python
# stt_node.py
import numpy as np
import whisper
from dora import Node

node = Node()
model = whisper.load_model("base")

for event in node:
    if event["type"] == "INPUT" and event["id"] == "audio":
        audio = event["value"].astype(np.float32) / 32768.0

        # Transcribe
        result = model.transcribe(audio, fp16=False)
        text = result["text"].strip()

        if text:
            node.send_output("text", [text])

    elif event["type"] == "STOP":
        break
```

### Text-to-Speech

```python
# tts_node.py
import numpy as np
from dora import Node
from TTS.api import TTS

node = Node()
tts = TTS(model_name="tts_models/en/ljspeech/tacotron2-DDC")

for event in node:
    if event["type"] == "INPUT" and event["id"] == "text":
        text = event["value"][0] if isinstance(event["value"], list) else str(event["value"])

        # Generate speech
        wav = tts.tts(text)
        audio = np.array(wav, dtype=np.float32)

        node.send_output("audio", audio)

    elif event["type"] == "STOP":
        break
```

## Audio Data Format

```python
# Audio as numpy array
# - Shape: (samples,) or (samples, channels)
# - dtype: np.float32 or np.int16
# - Sample rate: typically 16000 or 44100

# 16-bit integer audio
audio_int16 = np.array([...], dtype=np.int16)

# Float32 audio (normalized to [-1, 1])
audio_float32 = audio_int16.astype(np.float32) / 32768.0
```

## Performance Tips

1. **Use appropriate chunk sizes**
   - Small chunks (512-1024): Lower latency
   - Larger chunks (2048-4096): Better efficiency

2. **Use VAD to reduce processing**
   - Only process when speech detected
   - Saves compute resources

3. **Choose appropriate model sizes**
   - Whisper tiny/base: Fast, lower accuracy
   - Whisper medium/large: Slower, higher accuracy

4. **Buffer audio for batch processing**
   ```yaml
   inputs:
     audio:
       source: microphone/audio
       queue_size: 10  # Buffer multiple chunks
   ```

## Available Hub Nodes

| Node | Package | Purpose |
|------|---------|---------|
| dora-microphone | `pip install dora-microphone` | Audio input |
| dora-pyaudio | `pip install dora-pyaudio` | Audio output |
| dora-distil-whisper | `pip install dora-distil-whisper` | Speech-to-text |
| dora-kokoro-tts | `pip install dora-kokoro-tts` | Text-to-speech |
| dora-vad | `pip install dora-vad` | Voice activity |
| dora-outtetts | `pip install dora-outtetts` | TTS alternative |

## Related Skills

- **hub-nodes** - All pre-built nodes
- **dataflow-config** - YAML configuration
- **domain-vision** - Vision pipelines