Super Transcribe

🎙️ Super-Transcribe — Unified Speech-to-Text

A self-contained transcription skill with two bundled backends that intelligently routes based on task requirements:

• 🦜 Parakeet (NVIDIA NeMo) — best accuracy (6.34% WER), ~3380× realtime, auto-punctuation, 25 European languages, NeMo diarization
• 🗣️ faster-whisper (CTranslate2) — translation, 99+ languages, initial prompting, advanced inference tuning

Lazy loading: each backend sets up its own venv on first use. No pre-configuration needed — just transcribe and the right backend installs itself.

⚡ First-Time Setup

Follow these 3 steps on first use. Goal: install only what this user's hardware needs — nothing extra.

Step 1 — Check readiness (instant, no downloads)

./scripts/transcribe --check --json

Key fields: gpu (name + VRAM), ffmpeg, backends status, estimated_install (download size for recommended backend), missing_optional (non-blocking items — do NOT eagerly install these).

Step 2 — Preview the install (no downloads)

./scripts/transcribe --quickstart --dry-run --json

Tell the user what will be downloaded before proceeding. Relay from the estimated field:

• Which backend and why it was chosen
• Total download size (setup + model on first use)

Example message: "Your system has an NVIDIA GPU, so I'd install the Parakeet backend for best accuracy. That's 5GB for setup plus ~1.2GB for the model on first use (6.2GB total). OK to proceed?"

For GPU systems where the user wants a smaller install, see Lean Install Option below.

Step 3 — Install (after user confirms)

./scripts/transcribe --quickstart --json

ok: true → backend installed. First transcription downloads model weights (~1-2 GB, cached permanently). ok: false → check errors array for what failed.

What Gets Installed

Quickstart installs exactly ONE backend — the best for detected hardware:

NOT installed by quickstart (deferred until actually needed):

• Second backend — auto-installs only when the user triggers a feature that needs it (--translate or non-EU language → faster-whisper; --fast/--multitalker → Parakeet)
• System deps — install separately only if the user needs them:
  • ffmpeg — only for non-WAV input (mp3/m4a/mp4/ogg). Install: sudo apt install ffmpeg
  • yt-dlp — only for YouTube/URL downloads. Install: pipx install yt-dlp
  • HuggingFace token — only for --diarize with faster-whisper. Setup: huggingface-cli login
• PyTorch for faster-whisper — deferred until --diarize is first used (saves ~2.8 GB on initial install)

Do NOT pre-install optional deps. Wait until the user actually needs a feature — the script auto-installs or clearly reports what's missing at that point.

Lean Install Option

If a GPU user wants a smaller download (~1 GB instead of ~6.2 GB):

./scripts/transcribe --setup faster-whisper

Trade-off: ~20× realtime speed (vs ~3380× with Parakeet), no built-in punctuation. Only offer this if the user explicitly asks for a minimal install or has limited bandwidth/disk.

Supplementary info: see Setup Details further down.

✅ Setup done. Everything below is usage reference — read on demand.

When to Use

Use this skill for any speech-to-text task. It replaces both the faster-whisper and parakeet skills as your single entry point.

Trigger phrases: "transcribe this", "speech to text", "what did they say", "make a transcript", "subtitle this video", "who's speaking", "translate this audio", "transcribe this podcast", "transcribe with parakeet", "transcribe with whisper", "best accuracy transcription", "transcribe in French", "diarize this meeting", "find where X is mentioned", "audio to text", "translate to English", "search transcript for", "when did they say", "at what timestamp", "add chapters", "detect chapters", "find breaks in the audio", "table of contents for this recording", "TTML subtitles", "DFXP subtitles", "broadcast format subtitles", "Netflix format", "ASS subtitles", "aegisub format", "LRC subtitles", "timed lyrics", "karaoke subtitles", "HTML transcript", "confidence-colored transcript", "separate audio per speaker", "export speaker audio", "transcript as CSV", "spreadsheet output", "podcast RSS feed", "per-file language", "remove filler words", "clean up ums and uhs", "transcribe left channel", "wrap subtitle lines", "character limit per line", "detect paragraphs"

Auto-Routing Logic

The router picks the backend automatically:

┌─────────────────────────────────────────────┐
│             --backend specified?             │
│         YES → use that backend              │
│         NO  ↓                               │
├─────────────────────────────────────────────┤
│     --fast or --multitalker?                │
│         YES → parakeet                      │
│         NO  ↓                               │
├─────────────────────────────────────────────┤
│     Needs faster-whisper-only feature?      │
│  (translate, initial-prompt, hotwords,      │
│   multilingual, non-EU language, advanced   │
│   inference tuning params)                  │
│         YES → faster-whisper                │
│         NO  ↓                               │
├─────────────────────────────────────────────┤
│      Needs parakeet-only feature?           │
│  (--long-form, --streaming, --no-align)     │
│         YES → parakeet                      │
│         NO  ↓                               │
├─────────────────────────────────────────────┤
│  Default: prefer Parakeet (best accuracy    │
│  + speed + auto-punctuation)                │
│  Fall back to faster-whisper if Parakeet    │
│  not installed                              │
└─────────────────────────────────────────────┘

Quick Reference

Basic (Auto-Routes to Best Available)

Routes to Faster-Whisper Automatically

Routes to Parakeet Automatically

⚠️ Agent Guidance — Keep Invocations Minimal

CORE RULE: The default command (./scripts/transcribe audio.mp3) is the fastest path. Add flags only when the user explicitly asks for that capability. The router handles backend selection automatically.

⚠️ Model preference: When the router selects faster-whisper, it uses distil-large-v3.5 by default — this is the preferred model (fastest, better accuracy than large-v3-turbo). Don't override unless the user asks.

Probe Mode (--probe)

Use --probe to check audio metadata before deciding whether to transcribe:

./scripts/transcribe --probe audio.mp3
# → {"file": "audio.mp3", "path": "/tmp/audio.mp3", "duration": 2714.5, "duration_human": "45m 14s", "format": "mp3", "channels": 2, "sample_rate": 44100, "bitrate": 128000, "size_mb": 43.2}

• Returns compact JSON with duration, format, channels, sample rate, bitrate, and file size
• No transcription — runs in milliseconds using ffprobe
• Use to implement the "ask before transcribing long files" flow:
  • Probe → duration > 10 min → ask user "This is 45 minutes, want me to transcribe?"
  • Probe → duration < 2 min → transcribe immediately
• Works with multiple files: ./scripts/transcribe --probe *.mp3

Agent Output Mode (--agent)

Use --agent for structured output the agent can parse directly:

./scripts/transcribe --agent audio.ogg
# → {"text": "Hello world", "duration": 4.2, "language": "en", "language_probability": 0.98, "processing_time": 0.8, "backend": "faster-whisper", "segments": 1, "speakers": null, "word_count": 2, "avg_confidence": 0.94, "file_path": "/tmp/audio.ogg"}

• Emits a single compact JSON line to stdout — no stderr noise (implies --quiet)
• Core fields: text, duration, language, language_probability, processing_time, backend, segments (count), speakers (list or null), word_count
• Agent-specific fields:
  • avg_confidence (0.0–1.0) — average word/segment confidence; use to gauge transcript reliability. Present for faster-whisper (from avg_logprob); absent for Parakeet (NeMo doesn't expose confidence scores)
  • file_path — absolute path of input file (for multi-file tracking)
  • output_path — absolute path of written output file (when -o is used)
  • summary_hint — {"first": "...", "last": "..."} for long transcripts (>400 chars); gives the agent a quick preview without reading the full text
• Combine with -o to also save full transcript to file: ./scripts/transcribe --agent -o /tmp/transcript.txt audio.mp3
• Use for quick voice message transcription where the agent just needs the text and metadata

Exit Codes

Standardized exit codes for agent error handling:

Chat Media Handling (OpenClaw Agents)

OpenClaw downloads media and provides local file paths in the message context (e.g. <media:voice path="/path/to/audio.ogg"> or <media:audio path="/path/to/file.mp3">).

⚠️ Do NOT auto-transcribe every voice message. Determine intent first:

When to Transcribe Immediately (No Confirmation Needed)

• User explicitly asks for transcription: "transcribe this", "what does this say", "make subtitles for this"
• User sends an audio/video file (not a voice note) with a transcription request
• User sends media in an ongoing transcription conversation (context makes intent clear)

When to Ask First

• User sends a voice message with no context — they're probably talking to you, not asking for a transcript. Respond to the content of their message instead, or ask: "Want me to transcribe this, or were you telling me something?"
• User sends a long audio/video file (>10 min) without explicit instructions — confirm before committing to a long transcription
• User forwards audio from someone else without comment — unclear if they want transcription or something else

Technical Notes

• Voice messages (.ogg opus) are auto-converted by ffmpeg — no special handling needed
• Video files work too — ffmpeg extracts the audio track automatically
• Use -q (quiet) for short voice messages to suppress stderr progress noise

Example Flows

User wants transcription:

User: Can you transcribe this? [sends audio file]
Agent: Transcribing now...
Agent: [runs ./scripts/transcribe -q /path/to/audio.mp3]
Agent: Here's the transcript: "..."

User is talking to the agent via voice:

User: [sends voice message saying "Hey, what's on my calendar today?"]
Agent: [transcribes silently to understand the request]
Agent: [responds to the calendar question, does NOT parrot back the transcript]

Ambiguous intent (use --probe to check duration first):

User: [sends 45-minute meeting recording, no comment]
Agent: [runs ./scripts/transcribe --probe /path/to/recording.mp3]
Agent: Got a 45-minute recording. Want me to transcribe it? I can also do speaker identification or subtitles if you need those.

Output Length Management for Chat

Transcripts can be very long. Follow these rules to avoid flooding the chat:

• Short audio (<2 min, <~2000 chars): Reply directly with the full transcript text
• Medium audio (2-10 min): Reply with full text; if it exceeds ~3000 chars, consider saving to file and offering a summary
• Long audio (>10 min): Always save to a file (-o /tmp/transcript.txt), share the file path, and provide a brief summary of the content
• Subtitle/data formats (SRT, VTT, ASS, CSV, JSON, etc.): Always save to file with -o, tell the user the path
• Search results (--search): Show directly — they're already concise
• Language detection (--detect-language-only): Show directly — single line output

When in doubt, save to file and summarize. Users can always ask for the full text.

Do NOT add flags unless explicitly requested:

• --backend — unless the user specifically requests a backend
• --diarize — unless the user asks "who said what" / "identify speakers"
• --translate — unless the user wants audio translated to English
• --format srt/vtt/ass/etc. — unless the user asks for subtitles in that format
• --long-form — unless the audio is confirmed >24 minutes
• --denoise/--normalize — unless the user mentions bad audio quality
• --initial-prompt — unless there's domain-specific jargon to prime
• --search — unless the user asks to find/locate a word in audio
• --detect-chapters — unless the user asks for chapters/sections
• --clean-filler — unless the user asks to remove filler words
• --word-timestamps — unless the user needs word-level timing
• --stream / --streaming — unless the user wants live/progressive output
• --clip-timestamps — unless the user wants a specific time range
• --temperature 0.0 — unless the model is hallucinating on music/silence
• --vad-threshold — unless VAD is aggressively cutting speech or including noise
• --min-speakers/--max-speakers — unless you know the speaker count
• --hf-token — unless the token is not cached at ~/.cache/huggingface/token
• --max-words-per-line / --max-chars-per-line — unless the user asks for subtitle wrapping
• --filter-hallucinations — unless the transcript contains obvious artifacts
• --merge-sentences — unless the user asks for sentence-level subtitle cues
• --channel left|right — unless the user mentions stereo tracks
• --detect-paragraphs — unless the user asks for paragraph breaks
• --speaker-names — unless the user provides real names (always requires --diarize)
• --hotwords — unless the user names specific rare terms
• --prefix — unless the user knows the exact words the audio starts with
• --detect-language-only — unless the user only wants language ID
• --stats-file — unless the user asks for performance stats
• --parallel N — only for large CPU batch jobs
• --retries N — only for unreliable inputs (URLs, network files)
• --burn-in — only when user explicitly asks to embed subtitles into video
• --keep-temp — only when the user may re-process the same URL
• --output-template — only when user specifies a custom naming pattern
• --timestamps (parakeet) — unless the user asks for word-level timestamps
• --fast — unless the user explicitly wants speed over accuracy or mentions low VRAM
• --multitalker — unless the user mentions overlapping speech or asks for better multi-speaker handling
• --resume <path> — unless the user is resuming a batch job that crashed/was interrupted
• --no-align — unless the user wants faster processing and doesn't need precise word timestamps
• --multitalker-diar-model / --multitalker-asr-model — only when user provides specific model names
• Multi-format (--format srt,text) — only when user explicitly wants multiple formats; always pair with -o <dir>
• --agent — use when you need structured JSON for parsing (e.g. silent transcription of voice messages to understand user intent); don't use when the user explicitly asked for a transcript (just show them the text directly)
• --probe — use to check duration before transcribing ambiguous long files; don't probe short voice messages (just transcribe them directly)

Overhead notes:

• Any word-level feature (faster-whisper) auto-runs wav2vec2 alignment (~5-10s overhead)
• --diarize adds ~20-30s on top

Search guidance:

• --search replaces the normal transcript output — it prints only matching segments with timestamps
• Add --search-fuzzy only when the user mentions approximate/partial matching
• To save search results to a file, use -o results.txt

Chapter detection guidance:

• Default --chapter-gap 8 (8-second silence = new chapter) works for most content; tune down for dense content
• --chapter-format youtube (default) outputs YouTube-ready timestamps; use json for programmatic use
• Always use --chapters-file PATH when combining chapters with a transcript output
• Batch mode limitation: --chapters-file takes a single path — in batch mode, each file's chapters overwrite the previous

Speaker audio export guidance:

• Always pair --export-speakers with --diarize
• Requires ffmpeg; outputs SPEAKER_1.wav, SPEAKER_2.wav, etc. (or real names if --speaker-names is set)

Language map guidance:

• Only use --language-map in batch mode when the user has confirmed different languages across files
• Inline format: "interview*.mp3=en,lecture*.mp3=fr" — fnmatch globs on filename
• JSON file format: @/path/to/map.json

RSS / Podcast guidance:

• Default fetches 5 newest episodes; --rss-latest 0 for all; --skip-existing to resume safely
• Always use -o <dir> with --rss — without it, all episode transcripts print to stdout concatenated

Output format for agent relay:

• Text transcript → show directly to user (summarise long ones)
• Subtitle formats (SRT, VTT, ASS, LRC, TTML) → write to -o file, tell user the path
• Data formats (CSV, TSV, JSON, HTML) → write to -o file, tell user the path
• Search results → show directly (human-readable)
• Chapter output → show directly or write to --chapters-file
• Stats output (--stats-file) → summarise key fields for the user
• Language detection (--detect-language-only) → print directly; it's a single line
• Multi-format (--format srt,text) → requires -o <dir>; tell user all paths written

Don't use this skill when:

• User wants text-to-speech (use the tts tool instead)
• User wants audio editing or music generation
• User wants to summarize a YouTube video without needing a raw transcript (use summarize skill)

Backend Comparison

Setup Details

For first-time setup, see ⚡ First-Time Setup at the top. This section is supplementary reference.

What Gets Installed (and What Doesn't)

Quickstart installs one backend — the best for the user's hardware. The second backend auto-installs later only if a feature needs it. Heavy optional deps are deferred until first use.

Why the size difference? Parakeet needs PyTorch+CUDA upfront (~5 GB, NeMo depends on it). faster-whisper uses CTranslate2 directly — PyTorch is only pulled in for diarization/alignment features.

Parakeet uses nemo_toolkit[asr-only] — this skips ~19 training-only packages (wandb, transformers, datasets, lightning, pandas, peft, etc.) saving ~400 MB vs [asr].

Optional Extras (Not Needed for Basic Transcription)

Manual Backend Setup

./scripts/transcribe --check              # Status check (human-readable)
./scripts/transcribe --check --json       # Status check (agent-parseable)
./scripts/transcribe --setup parakeet     # Best accuracy/speed (needs GPU)
./scripts/transcribe --setup faster-whisper  # Translation, 99+ languages, CPU-compatible
./scripts/transcribe --setup all --diarize   # Full install with diarization

Hard Requirements & Install Strategy

Hard requirements: Python 3.10+ only. Everything else is installed by --quickstart or auto-installs on first use.

Lean install strategy:

• faster-whisper: 300 MB base (CTranslate2, no PyTorch). PyTorch (2.8 GB) deferred until --diarize.
• Parakeet: ~5 GB base (NeMo + PyTorch CUDA). Uses nemo_toolkit[asr-only] — skips training-only packages.
• System deps: --quickstart does NOT auto-install system packages (ffmpeg, yt-dlp). It reports them as missing; install them separately if needed.

Auto-install behavior: Backends lazy-load on first use. Optional deps (like pyannote for diarization) auto-install when you use the feature that needs them. In --agent mode, setup emits structured JSON status to stderr.

Platform Support

GPU Troubleshooting

If setup didn't detect your GPU, manually install PyTorch with CUDA:

# For CUDA 12.x (in the backend's venv)
uv pip install --python .venv/bin/python torch --index-url https://download.pytorch.org/whl/cu121

WSL2 users: Ensure you have the NVIDIA CUDA drivers for WSL installed on the Windows side.

Options

Super-Transcribe Routing Options

--quickstart         One-command setup: install best backend for hardware (lean)
--quickstart --all   Install both backends (Parakeet + faster-whisper)
--quickstart --json  Same, structured JSON output for agent parsing
--quickstart --dry-run --json  Preview what would be installed with estimated sizes
--check              Check prerequisites + backend status (exit 0=ready, 2=not ready)
--check --json       Same check, compact JSON with estimated install sizes
--backend <name>     Force a backend: faster-whisper | parakeet | fw | pk
--backends           List available/installed backends and exit
--probe              Quick audio metadata (duration, format, channels) — no transcription
--setup <backend>    Pre-install a backend: faster-whisper | parakeet | all
--help-routing       Show detailed auto-routing logic
--fast               Use small 110M Parakeet model (quick, lower accuracy)
--multitalker        Multi-speaker pipeline with overlapped speech handling
--resume <path>      Resume batch processing from checkpoint file
--version            Show super-transcribe + backend versions

Shared Options (Work with Both Backends)

AUDIO                 Audio file(s), directory, glob, or URL
-f, --format FMT      text | json | srt | vtt | ass | lrc | ttml | csv | tsv | html
-o, --output PATH     Output file or directory
-m, --model NAME      Model name (backend-specific)
-l, --language CODE   Language code
--max-words-per-line  Subtitle word wrapping
--max-chars-per-line  Subtitle character wrapping
--batch-size N        Inference batch size
--skip-existing       Skip already-transcribed files
--resume PATH         Resume batch from checkpoint file (skips completed files)
--device DEV          auto | cpu | cuda
-q, --quiet           Suppress progress messages
--version             Print version info
--diarize             Speaker diarization
--min-speakers N      Min speakers hint for diarization
--max-speakers N      Max speakers hint for diarization
--speaker-names LIST  Replace SPEAKER_1, SPEAKER_2 with names
--export-speakers DIR Export each speaker's audio to WAV files
--search TERM         Search transcript for TERM
--search-fuzzy        Fuzzy/approximate search matching
--detect-chapters     Detect chapter breaks from silence gaps
--chapter-gap SEC     Min silence gap for chapter break (default: 8.0)
--chapters-file PATH  Write chapter markers to file
--chapter-format FMT  youtube | text | json (default: youtube)
--clean-filler        Remove hesitation fillers (um, uh, etc.)
--filter-hallucinations  Filter common hallucination patterns
--merge-sentences     Merge segments into sentence chunks
--detect-paragraphs   Insert paragraph breaks based on gaps
--paragraph-gap SEC   Min gap for paragraph break (default: 3.0)
--normalize           EBU R128 volume normalization
--denoise             High-pass + FFT noise reduction
--channel CH          left | right | mix (default: mix)
--burn-in OUTPUT      Burn subtitles into video file
--rss URL             Podcast RSS feed to transcribe
--rss-latest N        Latest N episodes from RSS (default: 5)
--stats-file PATH     Write performance stats JSON sidecar

Faster-Whisper Only Options

Model & Language:
  --revision REV        Model revision (git branch/tag/commit) to pin a specific version
  --initial-prompt TEXT  Prompt to condition the model (terminology, formatting style)
  --prefix TEXT         Prefix to condition the first segment (e.g. known starting words)
  --hotwords WORDS      Space-separated hotwords to boost recognition
  --translate           Translate any language to English (instead of transcribing)
  --multilingual        Enable multilingual/code-switching mode
  --hf-token TOKEN      HuggingFace token for private/gated models and diarization
  --model-dir PATH      Custom model cache directory (default: ~/.cache/huggingface/)

Output:
  --word-timestamps     Include word-level timestamps (wav2vec2 aligned automatically)
  --stream              Output segments as they are transcribed (disables diarize/alignment)
  --output-template TPL Batch output filename template ({stem}, {lang}, {ext}, {model})

Inference Tuning:
  --beam-size N         Beam search size; higher = more accurate but slower (default: 5)
  --temperature T       Sampling temperature or comma-separated fallback list
  --no-speech-threshold PROB  Mark segments as silence (default: 0.6)
  --no-vad              Disable voice activity detection
  --vad-threshold T     VAD speech probability threshold (default: 0.5)
  --vad-neg-threshold T VAD negative threshold for ending speech
  --min-speech-duration MS  Minimum speech segment duration in ms
  --max-speech-duration SEC Maximum speech segment duration
  --min-silence-duration MS Minimum silence before splitting (default: 2000)
  --speech-pad MS       Padding around speech segments (default: 400)
  --no-batch            Disable batched inference
  --hallucination-silence-threshold SEC  Skip silent sections where model hallucinates
  --no-condition-on-previous-text  Don't condition on previous text (auto-enabled for distil models)
  --condition-on-previous-text  Override auto-disable for distil models
  --compression-ratio-threshold RATIO  Filter segments above this ratio (default: 2.4)
  --log-prob-threshold PROB  Filter segments below avg log probability (default: -1.0)
  --max-new-tokens N    Maximum tokens per segment
  --clip-timestamps RANGE  Transcribe specific time ranges: '30,60' or '0,30;60,90'
  --progress            Show transcription progress bar
  --best-of N           Candidates when sampling with non-zero temperature (default: 5)
  --patience F          Beam search patience factor (default: 1.0)
  --repetition-penalty F  Penalty for repeated tokens (default: 1.0)
  --no-repeat-ngram-size N  Prevent n-gram repetitions (default: 0 = off)

Advanced Inference:
  --no-timestamps       Output text without timing info
  --chunk-length N      Audio chunk length for batched inference
  --language-detection-threshold T  Confidence threshold for language detection (default: 0.5)
  --language-detection-segments N  Segments to sample for detection (default: 1)
  --length-penalty F    Beam search length p

…(truncated)