Mix

Living document — read this first

This file is a running record of the current best known settings, updated every time we improve on them. When the user says "mix," don't re-derive everything from first principles — apply the Quick Start settings below as the starting point, then iterate from there based on what they say needs work.

After any session where these settings change, update the Quick Start block below (not just the history/lessons further down) so next time starts from the improved version, not the old one. Keep the "Version history" section growing — one line per session, what changed and why — so we don't repeat a dead end.

Quick Start — current best settings (as of v15, Servier project, 2026-09-18)

Ask for / confirm: path to VO file, path to music file, target video duration (for padding). Then run, substituting paths:

VO="<path to VO file>"
MUSIC="<path to music file>"
DUR=<target duration in seconds, e.g. 300.312>

# 1. If multiple speakers: silence-detect + pitch-cluster + per-segment level balance first (see Step 3 below for the full method) — skip if single speaker.

# 2. VO clarity chain (adjust EQ measured_* values by re-running loudnorm measure first)
ffmpeg -i "$VO" -af "highpass=f=85,equalizer=f=180:width_type=o:width=1.5:g=-3,equalizer=f=2200:width_type=o:width=1.2:g=2.5,equalizer=f=3800:width_type=o:width=1.2:g=3.5,equalizer=f=10000:width_type=o:width=1.5:g=1.5,loudnorm=I=-18:TP=-1.5:LRA=7:linear=true:print_format=json" -f null - # first pass: get measured_I/TP/LRA/thresh, then re-run with those values baked in and write to vo_clear.wav

# 3. VO sidechain key (compressed copy, used ONLY as duck trigger, never heard)
ffmpeg -i vo_clear.wav -af "acompressor=threshold=0.1:ratio=10:attack=5:release=50:makeup=1" vo_key.wav

# 4. Music: smooth any internal jump (tune points to the actual jump timestamp found by RMS-enveloping the raw track), then normalize
ffmpeg -i "$MUSIC" -af "compand=attacks=0.3:decays=2.0:points=-90/-90|-32/-32|-24/-24|-18/-21|-14/-16|0/-3:gain=0,apad=whole_dur=$DUR,loudnorm=I=-16:TP=-1.5:LRA=7:linear=true:print_format=json" -f null - # measure, then re-run with measured values baked in -> music_norm.wav

# 5. THE DUCK — deep + slow + keyed off the leveled VO copy, not raw VO
ffmpeg -i music_norm.wav -i vo_key.wav -filter_complex "[0:a][1:a]sidechaincompress=threshold=0.015:ratio=20:attack=150:release=1800:knee=8:makeup=1[out]" -map "[out]" music_ducked.wav

# 6. Mix + master
ffmpeg -i music_ducked.wav -i vo_clear.wav -filter_complex "[0:a][1:a]amix=inputs=2:duration=longest:normalize=0[premix];[premix]alimiter=limit=0.9:level=disabled[out]" -map "[out]" premix.wav
# measure premix loudness, apply one corrective `volume=+/-XdB` to hit -14 LUFS, then alimiter=limit=0.8:level=disabled for TP safety, export WAV + MP3

Always verify after every step (see Gotchas, Step 8): tail decay intact (RMS per 0.5s over the last ~15s, no jump to digital silence), full duration preserved, no VO-vs-music level check skipped. Don't skip verification just because it's the "known good" recipe — different source files can break assumptions.

What this skill produces

A mastered stereo mix (WAV + MP3) where narration sits clearly forward, music keeps its full composed dynamics/arrangement everywhere VO isn't talking, and the duck under VO is deep (matches broadcast standard) but doesn't pump. Built entirely with ffmpeg + Python (librosa/numpy/scipy) — no paid plugins needed.

This came out of a real, iterative session (Servier corporate video, 5-min narration + music bed, two alternating narrators) — the numbers and gotchas below are hard-won, not theoretical. Read the whole file before starting — several of these fix problems that look like something else at first.

Step 0 — Environment check

Confirm before doing anything, don't assume:

which ffmpeg
python3 -c "import librosa; print('ok')"

Both were already present last time; if missing, ask before installing anything.

Step 1 — Diagnose before touching a single fader

Don't guess levels or apply "standard" fixes blind. Measure first:

• Duration/sanity: ffprobe -v error -show_entries format=duration -of default=noprint_wrappers=1 <file>
• Loudness of each raw file: ffmpeg -i <file> -af loudnorm=print_format=json -f null - → read input_i (integrated LUFS), input_tp (true peak), input_lra (loudness range).
• Multiple speakers? Don't assume one VO level fixes everything. Silence-detect to segment speech (silencedetect=noise=-35dB:d=0.35), then for each segment run pitch tracking (librosa.pyin, fmin=65 fmax=300) to get median F0, and simple 1D two-means clustering on F0 to split into speaker groups (male center ~120Hz, female ~185Hz is typical). Measure mean dBFS per group — a 2dB imbalance between speakers is common and invisible until you check.
• VO spectral balance: FFT the VO in speech-heavy chunks, band-average in {80-150, 150-300, 300-800, 800-2000, 2000-3500, 3500-6000, 6000-10000}Hz. A raw VO recording often has the 80-300Hz band 15-20dB hotter than the 2-6kHz "clarity" band — that imbalance, not overall loudness, is often why voice sounds unclear.
• Music's own dynamic arc: RMS-envelope the raw music track in 10s windows. Stock "building" tracks often have an abrupt 5-15dB jump somewhere (drums/section entering) that will read as "suddenly loud" against picture — find the timestamp before deciding how to handle it.
• The actual VO-vs-music relationship: measure both files' input_i in the same time window (e.g. where VO is talking). Don't assume music is quieter — in the reference session the raw music was 4.7 LU louder than VO in that window despite "feeling" balanced on a quick listen.

Step 2 — Industry reference numbers (verified via web search, not folklore)

• Music should sit -15 to -20 LU below voice while narration plays (W3C accessibility min. 20dB, general consensus).
• Documentary convention: VO peaks ~-6dBFS, music ~-20dBFS (source).
• Final delivery for YouTube: -14 LUFS integrated, -1 dBTP.
• A shallow transparent duck (3-5dB) pairs with fast timing (10-15ms attack, 70-300ms release). A deep duck (12-20dB, what real narration-forward mixing needs) needs slow timing (100-150ms attack, 1200-1800ms release) — it behaves like a gain-ride that engages once per speech passage, not a compressor chasing every syllable. Using fast timing at deep depth is what causes audible pumping — this was the single biggest wrong turn in the reference session, took several rounds to diagnose.
• Radio/commercial engineers key the ducking sidechain off a compressed/leveled copy of the VO, not the raw VO (source) — this makes the duck trigger evenly regardless of the VO performance's natural dynamics, and measurably reduces pumping (stdev of momentary loudness dropped ~25% in the reference session just from this change).

Step 3 — Build the VO chain

1. Per-speaker level balance (if multiple narrators): generate one chained volume=<gain>dB:enable='between(t,<start>,<end>)' filter per speech segment, gain computed to bring each speaker's mean dBFS to a shared target.
2. Clarity EQ (chain these on top):
   • highpass=f=85 — remove rumble
   • equalizer=f=180:width_type=o:width=1.5:g=-3 — tame boom/proximity
   • equalizer=f=2200:width_type=o:width=1.2:g=2.5 and equalizer=f=3800:width_type=o:width=1.2:g=3.5 — presence/consonant clarity
   • equalizer=f=10000:width_type=o:width=1.5:g=1.5 — a little air, radio-commercial convention
3. Normalize with two-pass linear loudnorm, not single-pass dynamic mode: measure first (loudnorm=print_format=json), then re-run with measured_I/measured_TP/measured_LRA/measured_thresh filled in and linear=true. Target -18 LUFS. Linear mode applies one constant gain (preserves natural delivery dynamics); dynamic mode adds unwanted automatic riding and — worse — has a lookahead-buffer bug right at end-of-file (see Gotcha 2 below).
4. Build a separate, more heavily compressed copy of this same VO to use only as the sidechain key later: acompressor=threshold=0.1:ratio=10:attack=5:release=50:makeup=1 on top of the leveled/EQ'd VO. Never play this copy — it's just the duck trigger.

Step 4 — Build the music chain

1. Smooth any abrupt internal jump (the drums-entering moment from Step 1) with a surgical compand curve, e.g. compand=attacks=0.3:decays=2.0:points=-90/-90|-32/-32|-24/-24|-18/-21|-14/-16|0/-3 — tune the mid-range control points to shave a few dB specifically off the section that jumps, leave everything else (especially the true intro and true climax) alone. A blanket compressor across the whole track will flatten the intended build — verify by re-measuring the RMS envelope after, not just trusting the settings.
2. Do NOT apply a flat/always-on volume cut to fix "music is too loud vs voice." This was the second major wrong turn: an always-on trim also guts the music in sections with no VO at all — including the emotional swell at the end that the video needed. Music should sit at its full composed level whenever VO is silent. All separation should come from the dynamic duck in Step 5.
3. apad=whole_dur=<video_duration> if music is a few tenths of a second shorter than VO/video.
4. Two-pass linear loudnorm to baseline -16 LUFS (same technique as VO).

Step 5 — The duck (this is the part that took many iterations to get right)

sidechaincompress=threshold=0.015:ratio=20:attack=150:release=1800:knee=8:makeup=1

• main input: the normalized music
• sidechain input: the compressed VO key from Step 3.4, not the listening VO
• This combo reliably lands 12-17dB of duck depth (matches the -15/-20 LU standard) while keeping momentary-loudness stdev in the smooth range (1.5-2.5) rather than the pumpy range (5-6) you get from fast timing at the same depth.
• Verify depth by comparing loudnorm input_i of the ducked music in a VO-active window vs. a VO-silent window (or the pre-duck baseline in that same window) — don't trust it by ear alone until you've confirmed numerically, then confirm by ear.
• Optional extra polish (diminishing returns, only if depth still isn't enough): split music into a vocal-collision band (bandpass ~1-4kHz) and a low-collision band (bandpass ~75-300Hz, matches VO's fundamental), duck each independently and a bit deeper, recombine with the untouched rest via amix ... normalize=0. Verify with a real FFT measurement that the collision actually exists before bothering (in the reference case, music bass was 30dB vs. VO's 19-23dB in the same band — that's a real collision, not assumed).

Step 6 — Mix and master

1. amix=inputs=2:duration=longest:normalize=0 then alimiter=limit=0.9:level=disabled — always pass level=disabled on alimiter, its default auto-levels and will silently undo any gain trim you apply around it.
2. Measure the premix, then apply one more linear gain trim (volume=+/-XdB) to land exactly on -14 LUFS, followed by alimiter=limit=0.8:level=disabled for true-peak safety margin (limit=0.85-0.9 often still leaves true peak close to -0.4 to -0.7 dBTP; tighten if needed to reliably clear -1dBTP).
3. Export WAV (pcm_s24le) for further editing + MP3 320k for quick review.

Gotchas that cost real time — check every one of these

1. Disk space. 24-bit 48kHz stereo WAVs at ~5 minutes are ~85MB each; a multi-stage pipeline with test variants fills a disk fast. df -h before and during a long session. Delete each intermediate the moment its consumer file exists — don't wait until the end.
2. Never reuse one filter-graph label as input to two different filter types in a single -filter_complex (e.g. feeding [vo] into both sidechaincompress and amix in the same graph). This caused a real bug: the last ~5 seconds of one stream silently zeroed out (not a fade — exact digital silence) because the two consumers desynced internally. Fix: render each stage to a temp file and feed files into the next ffmpeg invocation instead of chaining everything in one giant graph. Slower, but reliable — verify this by librosa-loading the tail of every intermediate and printing RMS per 0.5s; confirm no unexplained jump to -180dB.
3. Single-pass loudnorm in dynamic mode has an end-of-file lookahead artifact — can mangle or truncate the last couple seconds. Always use two-pass linear mode (measure, then linear=true with the measured values) for anything that will be heard, especially a final master step.
4. Verify the tail after literally every render, not just at the end. A composed track's natural fade-out is easy to accidentally destroy (compand attack/decay times interacting with a fast decay, amix dropout_transition misbehaving when two inputs are nearly-but-not-exactly the same length, etc.). One line of Python (RMS per 0.5s window over the last ~15s) catches all of these immediately.
5. amix's dropout_transition is for inputs that differ by many seconds, not fractions of a second. If your two inputs are within ~1s of each other, pad the shorter one to exactly match length with apad=whole_dur=<seconds> first, and don't rely on dropout_transition at all.
6. A metric like loudness-stdev is a proxy for pumping, not the final word. It caught real improvements in this session, but "sounds obviously ducked" (expected, normal for a 15-20dB narration duck) is different from "sounds mechanical/pumpy" (the actual complaint) — always get the user's ear on it, don't declare victory from a number alone.

Version history

• 2026-09-18 (v1–v15, Servier project): first full build of this method. Landed on: per-speaker leveling, VO clarity EQ (85Hz highpass + 180Hz cut + 2.2k/3.8k presence + 10k air), music jump-smoothing via targeted compand (never a blanket compressor), no static volume trims on music (kills the no-VO swell sections), deep duck (~14dB) via sidechaincompress keyed off a separately-compressed VO copy with slow timing (150ms/1800ms attack/release) rather than fast. Two-pass linear loudnorm throughout, never single-pass dynamic (end-of-file artifact risk). Landed at -14.23 LUFS / -0.92 dBTP with a duck that's deep but not pumpy.

Open threads to pick up next time

• Parallel compression on VO (blend a heavily-compressed aux with the clean VO) to make voice denser/more forward without more duck — flagged as the next lever if v15-equivalent still isn't enough.
• Multiband duck was built once (vocal + low-collision bands) but superseded by the single deep broadband duck + leveled sidechain key — worth revisiting in combination with the leveled key if a future project's music has stronger frequency-specific collisions than pure level separation can fix.
• Haven't tried sourcing/requesting a composer-delivered "narration bed" (thinner arrangement) alternate mix, which is how real productions get depth without leaning on a single automatic compressor.