Video Subtitle Pipeline

Video Subtitle Pipeline

Overview

Three-step pipeline: Extract audio → Transcribe → Burn subtitles

Optimized for Apple Silicon using mlx-whisper (Metal-accelerated Whisper).

Prerequisites

# ffmpeg (media processing)
brew install ffmpeg

# mlx-whisper (Apple Silicon optimized transcription)
brew install pipx
pipx install mlx-whisper

Do NOT use pip install directly on macOS — it will fail with externally-managed-environment.

Step 1: Extract audio

ffmpeg -i input.mp4 -vn -acodec pcm_s16le -ar 16000 -ac 1 /tmp/audio.wav

Whisper expects 16kHz mono WAV for best results.

Step 2: Transcribe to SRT

mlx_whisper /tmp/audio.wav \
  --model mlx-community/whisper-large-v3-turbo \
  --language zh \
  --output-format srt \
  --output-dir /tmp/

Important CLI quirks:

• Use --output-format and --output-dir (with dashes, not underscores)
• Output file will be named after the input: /tmp/audio.srt

Model selection

Post-transcription cleanup

Always review the SRT file before burning. Common issues:

• Hallucinated repetition: Whisper may repeat words (e.g. "好好好好好") at silence boundaries — delete these entries
• Timestamp overlap: Some entries may have overlapping time ranges — adjust end times
• Segmentation: Merge overly short segments, split overly long ones (aim for 1–2 lines, < 42 chars per line)

SRT format reference:

1
00:00:01,000 --> 00:00:04,500
第一行字幕文本

2
00:00:05,000 --> 00:00:08,200
第二行字幕文本

Step 3: Burn subtitles (hardcoded)

ffmpeg -i input.mp4 -vf "subtitles=/tmp/audio.srt:force_style='\
FontName=PingFang SC,\
FontSize=22,\
PrimaryColour=&H00FFFFFF,\
OutlineColour=&H40000000,\
BackColour=&H40000000,\
Outline=2,\
Shadow=1,\
MarginV=35'" \
  -c:v libx264 -crf 23 -preset slow \
  -c:a aac -b:a 128k \
  -movflags +faststart \
  output_with_subs.mp4

Subtitle style parameters

Video encoding parameters

Soft subtitles (alternative)

If you need switchable subtitles (not burned in):

ffmpeg -i input.mp4 -i /tmp/audio.srt \
  -c copy -c:s mov_text \
  output_soft_subs.mp4

Note: browser <video> support for embedded subtitle tracks varies. For web delivery, prefer VTT sidecar files or hardcoded subtitles.

Checklist

• Audio extracted as 16kHz mono WAV
• Transcription complete, SRT file reviewed for hallucinations
• Subtitle timing verified (no overlaps, reasonable segment lengths)
• Subtitles burned with readable font, outline, and margin
• Output video tested for playback and readability
• File size acceptable for delivery method (compress further if needed)