Video Batch Qc

QC for batch-generated video

Generating video unattended means you will not watch all of it. Every check here exists because the corresponding failure reported success at the time.

The false-green failures

Stale output counted as this run's. A crashed render "passed" because the previous run's PNGs were still in the directory. Clear the output directory before every run and require an exact frame count afterwards. A batch runner that checks returncode == 0 and any(frames) will lie to you.

A test suite that logs and returns. A fatal handler that logs an error and then bare-returns lets the caller print "all tests passed" over a traceback. Re-raise, and sys.exit(1) at the top level.

A render that works and is black. Every frame written, zero exit code, nothing visible. Measure it.

Black-frame detection

ffmpeg -v error -i frame.png -vf signalstats,metadata=print:file=- -f null -

Parse lavfi.signalstats.YAVG (mean luma) and YMAX (peak). A dark-but-valid frame and a black one are easy to separate:

if ymax < 90:   fail("nothing bright in frame")
if yavg < 2.0:  fail("frame is essentially black")

Check peak as well as mean. A dark-background scene legitimately has a low mean; what it never has is a low peak. Real values from a neon-on-black render: mean 24-42, peak 187-241. A black frame: peak 17.

Sample the first, middle and last frame of each render — enough to catch a bad camera or a broken compositor, cheap enough to run on every episode.

For a whole video, sample every 12th frame:

ffmpeg -v error -i out.mp4 -vf "select='not(mod(n,12))',signalstats,metadata=print:file=-" -an -f null -

Contact sheets

The fastest way to judge a batch is to look at all of it at once.

ffmpeg -y -i a.png -i b.png ... -filter_complex "<parts>" -map "[out]" -frames:v 1 sheet.png

Per input, normalise the cell exactly or xstack silently overlaps cells:

[i:v]scale=W:H:force_original_aspect_ratio=decrease,
     pad=W:H:(ow-iw)/2:(oh-ih)/2:color=0x0b0b0f,
     pad=W:H+28:0:28:color=0x181820,
     drawtext=text='label':x=7:y=5:fontsize=18:fontcolor=0xE8E8F0[vi]

Then xstack=inputs=N:grid=CxR. Use grid=, not a hand-built layout= string — a layout string with a subtly wrong offset produces an image that looks plausible while hiding half the inputs. Pad the grid to a full rectangle with color=c=...:s=WxH:d=1 cells when the count does not divide evenly.

Loudness and container checks

Two-pass loudnorm for measurement:

ffmpeg -hide_banner -i out.mp4 -af loudnorm=I=-14:TP=-1.5:LRA=11:print_format=json -f null -

The JSON is on stderr; extract with re.search(r"\{[^{}]*\"input_i\"[^{}]*\}", stderr, re.S).

Gate on:

ffprobe -v error -print_format json -show_format -show_streams gives all of it in one call.

Fail per item, never per batch

An unattended batch must survive one bad item. Isolate each unit, record {slug, ok, error, metrics}, write a manifest, and return non-zero only at the end. Print the measured value on success too — "ok" tells you nothing about margin, and a run that is quietly drifting towards a threshold is worth seeing before it crosses it.

slug               dur   LUFS   luma  peak  caps     MB   status
eddy             25.0s  -13.2   42.4   241    22    9.0   ok

Check before you render, not after

Rendering is minutes per item; a typo in choreography should not cost that. Build the scene description only, and check it in seconds:

• does the declared duration match the sum of the beat sheet?
• are any coordinates non-finite or absurdly large? A diverged solver returns 1e245 and the renderer will build a scene from it without complaint. A recursive scan for not isfinite(v) or abs(v) > 1e6 catches it instantly.
• how many objects, and how large is the scene file?

The same logic applies to audio: synthesise every script up front and report what read speed it would need to fit its slot. A script that overruns cannot be fixed downstream.

Frame numbering

Blender pads frame numbers to 4 digits, not 5. Detect the padding from the first file rather than hardcoding %05d:

pngs = sorted(frames_dir.glob("*.png"))
pattern = str(frames_dir / f"%0{len(pngs[0].stem)}d.png")
start = int(pngs[0].stem)   # pass as -start_number

ffmpeg ordering rules worth remembering

• loudnorm goes last in an audio chain — anything after it undoes it.
• amix needs normalize=0 or it silently attenuates every input by 1/N.
• format=yuv420p goes after any filter that may output another pixel format.
• VAAPI (h264_vaapi) does not compose with CPU filters like subtitles without an explicit hwupload; for a single pass with burned-in text, libx264 -crf 18 -preset medium is simpler and the quality is better.

Watching a long batch: output buffering will lie to you

A batch render runs for hours, so you check its log. An empty log looks exactly like a job that has not started. Both of these produce one:

python render_all.py > run.log        # block-buffered: nothing until exit
python render_all.py | grep ok        # grep buffers too when not a tty

Python block-buffers stdout as soon as it is not a terminal, and so do most pipe stages. A batch that prints one line per item shows nothing at all until it finishes or its 4 KB buffer fills.

Fix it at the source, on every stage:

python -u render_all.py > run.log              # unbuffered
python render_all.py | grep --line-buffered ok # grep flushes per line
stdbuf -oL some_tool | tee run.log             # for tools with no flag

In the script itself, print(..., flush=True) on the per-item line is worth having whether or not the caller remembers -u.

Waiting on a batch without deadlocking yourself

Polling for "is it done" has two traps, and both cost a session.

A pgrep pattern matches the waiting command itself. The shell running until ! pgrep -f render_all; do sleep 5; done has render_all in its own command line, so pgrep finds it and the loop never exits. The same applies to pkill -f, which will happily kill the shell that issued it. Match on something the waiter does not contain — a PID captured with $!, a marker file the job touches on exit, or wait:

python -u render_all.py > run.log & pid=$!
wait "$pid"; echo "exit $?"

A completion marker beats a process check. Have the job write its own sentinel and poll for that instead — it survives the process being reaped, and it distinguishes "finished" from "crashed":

( python -u render_all.py; echo "EXIT=$?" >> run.log ) &
until grep -q '^EXIT=' run.log; do sleep 5; done