# Captions And Transcription

> Generate, correct, time, style, and deliver captions for video: local Whisper-family transcription (openai-whisper, whisper.cpp, faster-whisper, WhisperX), SRT/WebVTT/ASS formats, line length and reading-speed limits, styling burned-in captions, and WCAG caption requirements. Use this skill when a video needs captions or subtitles, when transcribing narration or interviews, or when reviewing captions for accuracy and accessibility. Activate when: captions, subtitles, transcribe, transcription, whisper, whisper.cpp, faster-whisper, WhisperX, SRT, VTT, WebVTT, ASS subtitles, word-level timestamps, karaoke captions, closed captions, caption accessibility.

- Skill: `latestaiagents/captions-and-transcription` (Agent Skill)
- Install (CLI): `npx get-skill add latestaiagents/captions-and-transcription`
- Raw SKILL.md: https://api.skillmd.com/api/skills/latestaiagents/captions-and-transcription/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Web & Frontend
- Author: latestaiagents (https://skillmd.com/u/latestaiagents)
- Updated: 2026-11-14
- Page: https://skillmd.com/skills/latestaiagents/captions-and-transcription

---


# Captions and Transcription

**Machine transcription is a first draft. Captions ship only after a human-quality pass for accuracy, timing, and readability.**

## When to Use

- Any video with speech that will be published (captions are a WCAG requirement for prerecorded synchronized media)
- Sound-off social video where captions carry the message
- Creating word-timed "kinetic" captions for Remotion or HyperFrames compositions
- Auditing existing captions

## Workflow

### Step 1: Extract Clean Audio

```bash
# 16 kHz mono 16-bit WAV: the input format the whisper.cpp README asks for; fine for the others too
ffmpeg -i video.mp4 -ar 16000 -ac 1 -c:a pcm_s16le speech.wav
```

If the mix has loud music, transcribe the VO stem rather than the final mix.

### Step 2: Choose a Transcription Tool

All four run locally; audio does not leave the machine. Pick by environment, not by hype.

| Tool | Install | Strength | Licence |
|------|---------|----------|---------|
| openai-whisper | `pip install -U openai-whisper` (needs FFmpeg) | Reference implementation; CLI writes SRT/VTT directly | MIT |
| whisper.cpp | Build with CMake; download a ggml model | C/C++, no Python; CLI writes SRT/VTT | MIT |
| faster-whisper | `pip install faster-whisper` | CTranslate2 reimplementation; Python API with word timestamps and VAD filter | MIT |
| WhisperX | `pip install whisperx` | Word-level timestamps via forced alignment; optional speaker diarization | BSD-2-Clause |

Hosted speech-to-text APIs are also an option when local compute is unavailable; check the provider's data-handling terms before uploading user audio.

**openai-whisper CLI:**

```bash
whisper speech.wav --model turbo --language English --output_format srt --output_dir captions/
# Word-level timing and line control (these flags require --word_timestamps True)
whisper speech.wav --model turbo --word_timestamps True --max_line_width 42 --max_line_count 2 --output_format srt
```

Models: `tiny`, `base`, `small`, `medium`, `large`, `turbo` (English-only `.en` variants exist except for `large` and `turbo`). `--task translate` translates to English.

**whisper.cpp:**

```bash
cmake -B build && cmake --build build -j --config Release
sh ./models/download-ggml-model.sh base.en
./build/bin/whisper-cli -m models/ggml-base.en.bin -f speech.wav -osrt -ovtt -of captions/speech
```

`-ml N` sets a maximum segment length in characters; `-sow` splits on words.

**faster-whisper (Python):**

```python
from faster_whisper import WhisperModel

model = WhisperModel("small", device="cpu", compute_type="int8")
segments, info = model.transcribe("speech.wav", word_timestamps=True, vad_filter=True)

def ts(t: float) -> str:
    h, rem = divmod(t, 3600); m, s = divmod(rem, 60)
    return f"{int(h):02}:{int(m):02}:{int(s):02},{int((s % 1) * 1000):03}"

with open("speech.srt", "w") as f:
    for i, seg in enumerate(segments, 1):  # segments is a generator; iterate once
        f.write(f"{i}\n{ts(seg.start)} --> {ts(seg.end)}\n{seg.text.strip()}\n\n")
```

**WhisperX** (word alignment; diarization needs a Hugging Face token and accepting the pyannote model terms):

```bash
whisperx speech.wav --highlight_words True
whisperx speech.wav --diarize --hf_token "$HF_TOKEN"
```

### Step 3: Correct the Transcript

Machine output will contain errors. Fix, in order:

1. **Proper nouns and product names.** Keep a glossary from the brief and search-replace.
2. **Numbers, units, and code identifiers.** Speech models normalize these inconsistently.
3. **Hallucinated text in silence or music.** Delete captions over non-speech sections that contain invented words.
4. **Non-speech information.** Add speaker labels when the speaker is not obvious, and meaningful sounds: `[door slams]`, `[upbeat music]`. WCAG defines captions as covering both speech and non-speech audio needed to understand the content.

### Step 4: Fix Timing and Line Breaks

Reference limits from the Netflix Timed Text Style Guide (English, USA), a widely used published spec:

| Rule | Limit |
|------|-------|
| Characters per line | 42 |
| Lines per caption | 2 maximum; prefer 1 |
| Reading speed | Up to 20 characters/second (adult), 17 (children's) |
| Minimum duration | 5/6 second per event |
| Maximum duration | 7 seconds per event |

Line breaks: break after punctuation and before conjunctions or prepositions; do not split an article from its noun, an adjective from its noun, or a name across lines.

For short vertical social clips, shorter lines (a few words, one line) read better on phones; keep the same reading-speed ceiling.

### Step 5: Pick the Format

| Format | Use for | Notes |
|--------|---------|-------|
| SRT | Uploading to platforms, simple sidecars | `HH:MM:SS,mmm` timestamps (comma) |
| WebVTT | HTML `<track>` on the web | File starts with `WEBVTT`; `.` before milliseconds |
| ASS | Styled burn-in (fonts, colours, positions, karaoke) | Rendered by libass via FFmpeg `subtitles`/`ass` filters |

Convert with FFmpeg: `ffmpeg -i captions.srt captions.vtt` or `ffmpeg -i captions.srt captions.ass`.

### Step 6: Style Burned-In Captions

- High contrast: white or yellow text with an outline or a semi-opaque box behind it.
- Keep captions inside the safe area and clear of platform UI in vertical feeds (see `motion-design-principles`).
- Do not cover faces or the UI element being demonstrated; move captions to the top for those shots.
- Word-by-word highlight is fine for social clips; never let the effect slow reading or flash rapidly.
- FFmpeg's `force_style` `FontSize` is relative to the ASS script's `PlayResY`, not output pixels. An SRT converted by FFmpeg gets a small default script resolution (288 lines in our test with FFmpeg 8.1), so check a still before rendering the whole video.

Burn-in commands live in `ffmpeg-post-production`, Step 7.

**In Remotion or HyperFrames**, render captions as components from the word-timed JSON instead of burning with FFmpeg. Remotion documents `@remotion/captions` (import SRT/VTT, transcription options, display, export); HyperFrames ships caption guidance in its `/hyperframes` skill. Follow those official skills for the code.

### Step 7: Deliver

- Upload the sidecar (SRT/VTT) to platforms that accept it, even if captions are also burned in. Sidecars are searchable, resizable, and translatable.
- On the web, attach captions with `<track kind="captions" srclang="en" label="English" src="captions.vtt" default>`.
- Keep the corrected transcript alongside the video; it doubles as alt content and a script for translations.

## Caption QA Checklist

- [ ] Every spoken line captioned; nothing captioned during silence
- [ ] Names, brand terms, and numbers match the glossary
- [ ] Speaker identification and meaningful sounds included
- [ ] No caption over 2 lines or 42 characters per line; reading speed within limits
- [ ] Captions appear and disappear in sync with speech
- [ ] Burned-in captions readable on a phone and clear of platform UI
- [ ] Sidecar file attached where the platform supports it

## Best Practices

1. Transcribe the cleanest audio available (VO stem, 16 kHz mono)
2. Treat model output as a draft; always do a correction pass
3. Use word-level timestamps for kinetic captions, segment-level for standard captions
4. Ship a sidecar caption file even when burning in
5. Keep a project glossary of names and terms for search-replace
6. Never present an uncorrected machine transcript as final captions

## References

- WCAG 2.2 SC 1.2.2 Captions (Prerecorded): https://www.w3.org/WAI/WCAG22/Understanding/captions-prerecorded.html
- openai/whisper (install, CLI, models, licence): https://github.com/openai/whisper and CLI flags in https://github.com/openai/whisper/blob/main/whisper/transcribe.py
- whisper.cpp (build, models, `whisper-cli` flags): https://github.com/ggml-org/whisper.cpp
- faster-whisper: https://github.com/SYSTRAN/faster-whisper
- WhisperX: https://github.com/m-bain/whisperX
- Netflix Timed Text Style Guide, General Requirements: https://partnerhelp.netflixstudios.com/hc/en-us/articles/215758617-Timed-Text-Style-Guide-General-Requirements
- Netflix English (USA) Timed Text Style Guide: https://partnerhelp.netflixstudios.com/hc/en-us/articles/217350977-English-USA-Timed-Text-Style-Guide
- MDN, WebVTT format: https://developer.mozilla.org/en-US/docs/Web/API/WebVTT_API/Web_Video_Text_Tracks_Format
- FFmpeg `subtitles` and `ass` filters: https://ffmpeg.org/ffmpeg-filters.html
- Remotion captions: https://www.remotion.dev/docs/captions
- HyperFrames skills: https://github.com/heygen-com/hyperframes

