Transcription

Transcription

Overview

Transcribe audio/video files (local or remote) using Whisper AI via transcribe-anything. Supports local files, YouTube URLs, and microphone input. Output formats: SRT, VTT, plain text, JSON.

Installation

pip install transcribe-anything

Backends install automatically in isolated virtual environments.

Usage

# Local file
transcribe-anything audio.mp3

# YouTube URL
transcribe-anything "https://www.youtube.com/watch?v=VIDEO_ID"

# With options
transcribe-anything audio.mp3 --model large-v3 --lang en --output_dir ./transcripts/

# GPU / device selection
transcribe-anything audio.mp3 --device cuda      # NVIDIA GPU
transcribe-anything audio.mp3 --device mlx       # Mac Apple Silicon (fastest on Mac)
transcribe-anything audio.mp3 --device groq      # Cloud API (fastest overall)

# Speaker diarization (requires HuggingFace token)
transcribe-anything audio.mp3 --device insane --hf_token YOUR_HF_TOKEN

Key Options

Backend Comparison

Output Files

Agent Usage Pattern

1. Identify input — local file path or URL
2. Select model — tiny/small for speed, large-v3 for accuracy
3. Select device — omit for auto; cuda for GPU, mlx for Apple Silicon
4. Run: transcribe-anything <input> --model <model> --output_dir <dir>
5. Return: path to output directory + detected language from .json

Batch Processing Large Audio Files

For audio files >30 minutes or processing multiple files:

# Batch process all audio files in a directory
for f in audio/*.mp3; do
  transcribe-anything "$f" \
    --model large-v3 \
    --output_dir "transcripts/$(basename "$f" .mp3)/" \
    --device cuda
done

# Process large files with chunking (split at silence boundaries)
# Install: pip install pydub
python3 -c "
from pydub import AudioSegment
from pydub.silence import split_on_silence
import os

audio = AudioSegment.from_file('long_audio.mp3')
chunks = split_on_silence(audio, min_silence_len=1000, silence_thresh=-40)

for i, chunk in enumerate(chunks):
    chunk_path = f'chunks/chunk_{i:04d}.mp3'
    chunk.export(chunk_path, format='mp3')
    os.system(f'transcribe-anything {chunk_path} --output_dir chunks/output/')
"

Performance targets:

WhisperX and Speaker Diarization

WhisperX extends Whisper with word-level timestamps and speaker diarization:

# Install WhisperX (used by transcribe-anything --device insane)
pip install whisperx

# Direct WhisperX usage for advanced control
python3 -c "
import whisperx
import json

# Load model
device = 'cuda'
compute_type = 'float16'
model = whisperx.load_model('large-v3', device, compute_type=compute_type)

# Transcribe
audio = whisperx.load_audio('audio.mp3')
result = model.transcribe(audio, batch_size=16)

# Align timestamps (word-level)
model_a, metadata = whisperx.load_align_model(language_code=result['language'], device=device)
result = whisperx.align(result['segments'], model_a, metadata, audio, device)

# Speaker diarization (requires HuggingFace token)
diarize_model = whisperx.DiarizationPipeline(use_auth_token='YOUR_HF_TOKEN', device=device)
diarize_segments = diarize_model(audio)
result = whisperx.assign_word_speakers(diarize_segments, result)

print(json.dumps(result['segments'], indent=2))
"

Speaker diarization output format:

{
  "segments": [
    {
      "start": 0.5,
      "end": 4.2,
      "text": "Hello, welcome to the meeting.",
      "speaker": "SPEAKER_00",
      "words": [{ "word": "Hello", "start": 0.5, "end": 0.9, "speaker": "SPEAKER_00" }]
    }
  ]
}

Requirements for speaker diarization:

• HuggingFace account + token (--hf_token)
• Accept model license: pyannote/speaker-diarization-3.1
• GPU strongly recommended (CPU is 10-50x slower)

Enforcement Hooks

Input validated against schemas/input.schema.json. See hooks/pre-execute.cjs for validation logic.

References

• Package: https://github.com/aj47/transcribe-anything
• Whisper paper: https://arxiv.org/abs/2212.04356

Memory Protocol (MANDATORY)

Before starting: Read .claude/context/memory/learnings.md for prior transcription task context.

After completing:

• Performance findings -> .claude/context/memory/learnings.md
• Issues encountered -> .claude/context/memory/issues.md

ASSUME INTERRUPTION: If it's not in memory, it didn't happen.