Local Narration Captions

Local narration and burned-in captions

A full offline voice + caption pipeline. Verified on Arch Linux, Python 3.14, CPU only (no CUDA). Nothing leaves the machine.

Why burned-in captions are not optional

Most short-form video is watched muted. For a large share of viewers the captions are the narration, not an accessibility extra. Generate them as subtitles rather than rendering text in your 3D scene: typography should not cost a re-render, and libass kerns and outlines far better than anything you would build on a renderer.

Install

python -m venv --system-site-packages .venv
.venv/bin/pip install kokoro-onnx soundfile faster-whisper

kokoro-onnx pulls onnxruntime. Both it and faster-whisper ship wheels that work on Python 3.14 (piper-tts also ships cp39-abi3 wheels if you prefer it).

Models (~350 MB):

curl -LO https://github.com/thewh1teagle/kokoro-onnx/releases/download/model-files-v1.0/kokoro-v1.0.onnx
curl -LO https://github.com/thewh1teagle/kokoro-onnx/releases/download/model-files-v1.0/voices-v1.0.bin

Watch out: on many distros /usr/bin/piper is the Logitech mouse configuration tool, not the TTS. Check piper --help before assuming.

Synthesis

from kokoro_onnx import Kokoro
k = Kokoro("kokoro-v1.0.onnx", "voices-v1.0.bin")
samples, rate = k.create(text, voice="bm_george", speed=1.0, lang="en-gb")

Measured ~4x realtime on CPU; model load ~0.4 s. Voices are af_*/am_* (American f/m) and bf_*/bm_* (British). Use lang="en-gb" with the b* voices.

Synthesise sentence by sentence, not in one pass. A sentence is the unit a narrator actually phrases, the gaps between them are where a viewer catches up, and you get real sentence boundaries as timestamps for free. Assemble with explicit silence: ~0.35 s lead-in, ~0.26 s between sentences, ~0.45 s tail.

Fitting a script to a fixed slot

Speed up the read rather than cutting the picture short. Up to about 1.12x is imperceptible; past that it sounds hurried and the real fix is a shorter script:

room = target_seconds - 0.5
n = synthesise(script, speed=1.0)
if n.duration > room:
    n = synthesise(script, speed=min(1.22, n.duration / room * 1.02))

Check this before rendering — a script that overruns cannot be fixed downstream. Synthesising all scripts up front and reporting the read speed each would need takes seconds and catches the problem while it is still editable.

Budget roughly 2.4 words/second including sentence gaps. 2.6 is too tight.

Word timings

Kokoro will not tell you when it said each word, and estimating from character counts drifts badly across a sentence. Transcribe the audio you just produced:

from faster_whisper import WhisperModel
model = WhisperModel("small", device="cpu", compute_type="int8")
segments, _ = model.transcribe(wav, word_timestamps=True, vad_filter=False, beam_size=1)

~3.6 s for 8 s of audio on CPU.

Use the recogniser only as a clock. The words come from your script, in order; the recogniser supplies when each landed. It will write "ninety" where your script says "90", and its comma guesses are not better than the writer's. Walk the script words forward through the recognised list matching on a normalised key (lowercase, strip non-alphanumerics), and when a word does not match, hang it off the previous word's end rather than dropping it.

Caption cards

Group words into cards of at most 3 words / ~26 characters. Break a card whenever the gap exceeds ~0.45 s or the previous word ended a sentence, so cards never straddle a sentence boundary. Then extend each card towards the next when the gap is only a breath, so captions do not flicker off between words.

ASS subtitles

ASS gives real outlines, shadows and fades, all of which survive compression better than thin text. Colours are &HAABBGGRR — alpha first, then BGR.

[Script Info]
ScriptType: v4.00+
PlayResX: 1080
PlayResY: 1920
WrapStyle: 0
ScaledBorderAndShadow: yes

[V4+ Styles]
Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding
Style: Caption,DejaVu Sans,92,&H00FFFFFF,&H00FFFFFF,&H00121216,&HA0000000,-1,0,0,0,100,100,1,0,1,7,4,2,70,70,330,1

[Events]
Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
Dialogue: 0,0:00:00.35,0:00:01.20,Caption,,0,0,0,,{\fad(70,70)}Why does a

• WrapStyle: 0 (smart wrap). WrapStyle: 2 means no wrapping at all — a long title then runs off both sides of the frame.
• Alignment 2 = bottom centre, 8 = top centre. MarginV is measured from that edge.
• Timestamps are H:MM:SS.cc — centiseconds, one digit of hours, no padding.
• Escape { and } in text; they are override-block delimiters.

9:16 safe areas (1080x1920)

• Top ~140-300 px: title kicker
• Middle: the subject
• Bottom: captions at MarginV ~330, clear of the platform's own UI

Loudness

Do loudness as two passes, on the audio alone. Measure first:

ffmpeg -i vo.wav -af loudnorm=I=-14.5:TP=-1.5:LRA=11:print_format=json -f null -

Parse the JSON from stderr ({...\"input_i\"...}), then apply with the measured values and linear=true. Single-pass loudnorm is an estimate and pumps.

Target -14 LUFS, -1.5 dBTP — what YouTube normalises to anyway. Normalise the voice to about -14.5 first, then mix the bed under it, then limit.

Music bed

A track pulled off the internet is a copyright claim waiting to happen on a channel meant to run unattended. Generate one: a few detuned sine partials (1x, 2x, 3x, 4x, 6x the root), each with its own slow amplitude LFO so it never settles into an obvious loop, through a one-pole lowpass at ~900 Hz, with a squared fade in and out. At about -34 dBFS RMS it registers as room tone, not music.

Final encode: one pass

Burn the subtitles in the same pass that encodes the frames. A second pass over finished H.264 re-compresses everything just to add text.

ffmpeg -y -framerate 30 -start_number 1 -i frames/%04d.png -i mix.wav \
  -vf "subtitles=captions.ass:fontsdir=/usr/share/fonts,format=yuv420p" \
  -c:v libx264 -preset medium -crf 18 -profile:v high -level 4.2 \
  -x264-params keyint=60:min-keyint=30:scenecut=0 \
  -c:a aac -b:a 192k -ar 48000 -af "alimiter=limit=0.8414" \
  -movflags +faststart -t 25.000 out.mp4

• Detect the frame-number padding — Blender pads to 4 digits, not 5. Read the first PNG's stem length rather than hardcoding %05d.
• Pass explicit -t and pad the audio to length. -shortest will silently truncate the video to the narration.
• format=yuv420p after the subtitle filter, or some players get a pix_fmt they will not decode.