Wavecap Whisper

WaveCap Whisper Model Tuning Skill

Use this skill to tune the Whisper speech-to-text model for optimal transcription quality.

Configuration Location

Whisper settings are in the whisper: section:

• User config: /Users/thw/Projects/WaveCap/state/config.yaml
• Default config: /Users/thw/Projects/WaveCap/backend/default-config.yaml

Model Selection

Primary Model

whisper:
  model: large-v3-turbo  # Model checkpoint

Backend Selection

whisper:
  backend: auto  # auto, mlx, faster-whisper

CPU Fallback Model

whisper:
  cpuFallbackModel: base  # Model when GPU unavailable

Set to null to use the primary model on CPU (slower but more accurate).

Decoding Parameters

Beam Size (search width)

whisper:
  beamSize: 8  # 1-15, higher = more consistent

Temperature (randomness)

whisper:
  decodeTemperature: 0.0  # 0.0 = deterministic
  temperatureIncrementOnFallback: 0.2  # Boost on low confidence

Condition on Previous Text

whisper:
  conditionOnPreviousText: false

Language Configuration

whisper:
  language: en  # ISO 639-1 code

Common codes: en (English), es (Spanish), fr (French), de (German), ja (Japanese)

Per-stream override:

streams:
  - id: spanish-feed
    language: es

Initial Prompts (domain vocabulary)

Global Prompt

whisper:
  initialPrompt: >-
    Emergency radio dispatch. Key terms: Adelaide, SITREP, SAPOL, SES, CFS.

Named Prompts (reusable)

whisper:
  prompts:
    sa_emergency: >-
      South Australia emergency dispatch. Terms: Adelaide, Noarlunga, SAPOL, SES, CFS.
    marine_vhf: >-
      Marine VHF radio. Terms: mayday, pan-pan, sécurité, vessel, nautical miles.

Assign to Stream

streams:
  - id: broadcastify-2653
    initialPromptName: sa_emergency

Concurrency

whisper:
  maxConcurrentProcesses: 2  # Parallel transcription jobs

View Current Settings

grep -A30 "whisper:" /Users/thw/Projects/WaveCap/state/config.yaml | head -35

Check Model Status

curl -s http://localhost:8000/api/health | jq

Tuning Scenarios

Maximum Accuracy (powerful hardware)

whisper:
  model: large-v3
  backend: auto
  beamSize: 12
  decodeTemperature: 0.0
  conditionOnPreviousText: true
  maxConcurrentProcesses: 1

Balanced Real-Time (recommended)

whisper:
  model: large-v3-turbo
  backend: auto
  beamSize: 8
  decodeTemperature: 0.0
  conditionOnPreviousText: false
  maxConcurrentProcesses: 2

Low Latency (weaker hardware)

whisper:
  model: small
  backend: auto
  cpuFallbackModel: tiny
  beamSize: 5
  decodeTemperature: 0.0
  maxConcurrentProcesses: 1

Apple Silicon Optimized

whisper:
  model: large-v3-turbo
  backend: mlx
  cpuFallbackModel: null  # Use primary on CPU too
  beamSize: 8
  maxConcurrentProcesses: 2

Prompt Engineering Tips

Effective Prompts

• Include domain-specific terms that Whisper might misspell
• Use proper capitalization for names and acronyms
• Keep prompts under 200 words
• Include example phrases if helpful

Example for Emergency Radio

whisper:
  initialPrompt: >-
    Emergency services radio dispatch for South Australia.
    Common terms: Adelaide, Adelaide fire out, Noarlunga, Aldinga, Para Hills,
    SITREP (situation report), SAPOL (SA Police), SES (State Emergency Service),
    CFS (Country Fire Service), MFS (Metropolitan Fire Service).
    Station identifiers: Sturt, Gawler, Metro, Blackwood.
    Radio protocol: Roger, Wilco, Over, Out, Copy that.

Apply Changes

launchctl stop com.wavecap.server && sleep 2 && launchctl start com.wavecap.server

Monitor Transcription Quality

Check confidence distribution

curl -s http://localhost:8000/api/transcriptions/export | \
  jq 'group_by(.confidence | . * 10 | floor / 10) |
      map({confidence: (.[0].confidence | . * 10 | floor / 10), count: length}) |
      sort_by(.confidence)'

Find low-confidence transcriptions

curl -s http://localhost:8000/api/transcriptions/export | \
  jq '[.[] | select(.confidence < 0.7)] | sort_by(.confidence) | .[:5] | .[] | {confidence, text}'

Tips

• large-v3-turbo offers the best speed/accuracy balance
• MLX backend on Apple Silicon is significantly faster
• Initial prompts help with domain-specific vocabulary
• Lower beam size for faster processing, higher for consistency
• Temperature 0.0 is best for transcription accuracy
• Monitor confidence scores to evaluate model performance