Video Ad Concept Editor

Video ad concept editor

Turn a folder of raw clips into finished 9:16 social ads. The core idea that makes this skill different from naive cutting:

Audio and video are not married. Each clip is a voiceover block AND a visual block, and they can be separated, re-ordered, and re-paired. A 44s set of six clips is not "44s of material" — it is ~6 movable VO lines plus ~18 distinct visual beats. That is what makes multiple genuinely different concept versions possible from the same footage.

Tools (all local, no network)

Workspace layout: one folder per project; assets/ raw in, work/ intermediates, output/ deliverables.

The method

0. Lock the design system BEFORE you cut

Do this first, every project. Without it you will drift: pick white-bordered text in one cut, invent a gold in the next, and ship a campaign with two visual languages. Decide once, write it to a file, reference it everywhere.

Derive the palette from the footage — never invent it. Sample dominant and accent colours across every clip:

from PIL import Image; from collections import Counter
im = Image.open(frame).convert("RGB")
q = Counter(((r//32)*32,(g//32)*32,(b//32)*32) for r,g,b in im.getdata())
dom = q.most_common(3)                                     # base/background
sat = lambda c:(max(c)-min(c))/max(c) if max(c) else 0
acc = max((c for c in q if sat(c)>.35 and max(c)>110), key=q.get, default=None)

Take the muted source tones and saturate them into UI accents. Worked example (founder-story ad): footage gave base #202020 in every clip, warm #806040 in 5/6 (wood, signage, tungsten), cool #406080 in the code screen only → tokens --bg:#0F1216, --primary:#F0B429 (warm, saturated), --secondary:#4EA1FF (cool, tech shots only). The payoff is visible: the gold CTA matched the gold BOUTIQUE sign already in frame, so it read as part of the scene, not pasted on.

Write tokens to TWO mirrored files — the pipeline has two renderers and they must agree:

• work/brand.css → :root{} vars for the HTML/Lottie graphics (1080x1920)
• work/brand.sh → shell vars + a br_text drawtext builder for ffmpeg (720x1280)

Templates live at new-website/work/brand.{css,sh} — copy and re-derive per project. source work/brand.sh then $(br_text work/txt/cta.txt "$BR_FS_CTA" "$BR_Y_CTA" "gte(t,26)" "$BR_PRIMARY").

Non-negotiables:

Type: Segoe UI Black (seguibl.ttf) for statements/CTA, Segoe UI for captions.

Retro-fit anything built before the tokens existed. Drift is not hypothetical: the first graphic here was authored pre-tokens and hardcoded #0b0e13 (vs --bg #0F1216) and 96/104px type (vs the 84px statement size) — invisible in isolation, wrong beside everything else. Grep each graphic for raw hex (#[0-9a-fA-F]{6}) and expect zero hits outside neutral placeholder greys. Verify tokens actually resolved by sampling the rendered PNG, not by reading the CSS — and account for overlays when you do (a corner reading #080a0c was #0F1216 under a 55% vignette, i.e. correct).

1. Inventory BEFORE cutting (this is the whole skill)

Transcribe every clip. Do not cut a frame before you know what is said.

ffmpeg -y -i clip.mp4 -vn -ac 1 -ar 16000 -c:a pcm_s16le c1.wav
whisper-cli.exe -m ggml-base.en.bin -f c1.wav -nt        # plain text
whisper-cli.exe -m ggml-base.en.bin -f c1.wav -ml 1      # word-level

The transcripts reveal the story order, duplicate takes, and VO defects (stutters, broken grammar). The filenames lie; the transcripts don't.

Then look at every clip — contact sheet, ~7 frames across each clip:

for t in 0.4 2.0 3.6 5.2 6.8 8.4 9.6; do
  ffmpeg -y -ss $t -i clip.mp4 -frames:v 1 -vf "scale=170:-1" f_$t.jpg
done
ffmpeg -y -i f_0.4.jpg ... -filter_complex "[0:v]...[6:v]hstack=inputs=7" row.jpg

A single 10s clip usually holds 3–4 distinct visual beats (wide → close-up → detail), not one. Catalogue them. Dead-air tails you'd cut for silence are often perfectly good silent b-roll for laying other VO under.

Write the result as two inventories: VO lines (text + in/out) and visual beats (clip + in/out + description).

2. Find speech boundaries with energy, not word timestamps

whisper -ml 1 word timings drift (seen reporting a word ending at 11.72s in a 10.01s file). Trust them for order, not for cuts. Get real boundaries from:

ffmpeg -i c.wav -af "silencedetect=noise=-35dB:d=0.3" -f null -

If a clip has a music/ambience bed, silencedetect finds nothing — fall back to an RMS scan (python wave + 50–100ms windows) and read the troughs. This is also how you locate a stutter to cut: the duplicated phrase is an energy burst with silence troughs either side.

Always verify a speech cut by re-transcribing the result, not by eye.

3. Design concepts, not trims

Three versions of the same edit is a trim. Three concepts means each tells a different story: different VO subset, different order, different opening hook. Proven patterns:

• Problem-led — drop the origin entirely; make it about the client. (problem → proof → results → CTA)
• Credibility-led — the founder's origin. (hook → turn → proof → CTA)
• In medias res — open on the payoff, then backfill why/how. Strongest hook. (payoff → problem → turn → CTA)

Rules: every version needs a different first line (the first 3s decide everything), and every version ends on the CTA. Lift a short CTA sting (~1.2s, just "Let's build together.") to close versions that don't include the CTA scene.

4. Treat weak or reused visuals

When VO is good but the visual is weak or already used in another version:

4b. Motion graphics — replace weak b-roll instead of treating it

Generic AI b-roll ("a boutique", "hands typing") illustrates a line. A graphic can argue it — a search returning 0 results shows invisibility in a way no photo of a shop can. Use graphics for the abstract/conceptual lines and keep real footage for the human ones (hook, CTA). Rule of thumb: face on camera for emotion, graphics for concepts.

Pipeline (already installed, no network):

node C:\Users\user\Desktop\video-ad\tools\html2mp4\render.mjs <html> <outDir> <frames> <fps>

• Renders at 1080x1920 via playwright-core + channel:'msedge', then downscale to 720x1280 (scale=720:1280:flags=lanczos) — a downscale, so it's crisp.
• render_alpha.mjs renders transparent PNGs for overlays on top of footage.
• Frames → video: ffmpeg -framerate 24 -i out/f_%04d.png ...

The HTML must expose window.seek(t) and be fully deterministic. Drive every property from t in JS. Do not use CSS animations/transitions or Date.now() — the renderer steps frame-by-frame and anything self-animating will tear or freeze. Call window.seek(0) at the end of the script so frame 0 is correct.

Skeleton that works:

const clamp=(v,a,b)=>Math.min(b,Math.max(a,v));
const inv=(t,a,b)=>clamp((t-a)/(b-a),0,1);      // progress across a..b
const outC=p=>1-Math.pow(1-p,3);                 // ease-out
window.seek = function(t){
  const p = outC(inv(t, 0.15, 0.70));            // beat timed to a spoken word
  el.style.opacity   = p;
  el.style.transform = `translateY(${(1-p)*44}px) scale(${0.82+0.18*p})`;
};
window.seek(0);

Time every beat to a word, using boundaries from the RMS scan (step 2) — not guesses. Frames = ceil(duration * fps). Working example: new-website/work/mg_a2.html (search → "0 RESULTS NEARBY" → GREAT SERVICE / ZERO VISIBILITY), rendered 219 frames @24fps for a 9.10s VO line.

Design notes that held up: dark premium base (#0b0e13), one accent (gold #f0b429), a radial vignette, staggered pop-ins (~0.26s apart), and ghosting elements out with saturate() + blur() rather than plain opacity.

Lottie icons (LottieFiles) — the user's preferred source

https://lottiefiles.com/free-animations/<tag> (health, restaurant, spa, notification, growth, business, icons…). Only take from the FREE list — the same pages also serve premiumAnimations / packs from iconscout (price: 5), which are paid.

Extract the free list with structured data rather than scraping the DOM (it's React Router, cards render lazily):

const ld = window.__reactRouterDataRouter.state.loaderData;
const key = Object.keys(ld).find(k=>k.includes('free-animations'));
ld[key].animations.map(a => ({
  name: a.name,
  json: 'https://assets-v2.lottiefiles.com/' +
        (a.variants.find(v=>v.type==='json' && v.isOptimized) || a.variants.find(v=>v.type==='json')).path,
  author: a.user && a.user.username,        // keep for attribution
  fr: a.meta.frameRate, frames: a.meta.frames, layers: a.meta.layers
}));

Pick small and simple: <15kb, <10 layers = a clean icon. Big multi-layer files are illustrations and will fight the footage.

Two traps:

1. Chromium blocks file:// XHR, so lottie.loadAnimation({path}) fails silently in the renderer. Wrap each JSON as window.LOT_name = {...}; in a .js file and load via <script src> (scripts aren't CORS-restricted), then pass animationData.
2. Icons carry their own palettes — recolour them, don't just re-badge them. Checked for real: clinic was #ffffff (invisible on a white badge), chart shipped 13 red/green/blue/yellow fills (straight off-brand), spa was #272b2b. A badge colour cannot fix both directions at once. Walk the JSON and rewrite fl/st (and gf/gs gradient stops, and animated c.k keyframes — k[0] numeric = static, objects = keyframed with s/e arrays) to brand tokens. Working recolor(data, {fill, stroke}) in new-website/work/mg_a4.html. Rule that worked: icons on a white badge → ink #0F1216; a chart on the dark base → fills --primary, strokes white. Deep-clone first (JSON.parse(JSON.stringify())). Audit any file with: fills/strokes are {ty:'fl'|'st', c:{k:[r,g,b,a]}}, floats 0–1.
3. Judge icons at full resolution. In a 200px contact sheet the recoloured icons looked like empty white circles and read as broken — at 1:1 the heartbeat, fork & knife and lotus were all crisp. Crop the badge region before concluding anything.

Install locally: npm install lottie-web in tools/html2mp4, copy build/player/lottie.min.js next to the HTML. Drive deterministically — anim.goToAndStop(frame, true) inside seek(), never autoplay:

const a = lottie.loadAnimation({container: el, renderer:'svg', autoplay:false,
                                loop:false, animationData: window.LOT_clinic});
// inside seek(t): map wall-clock to the icon's own frame rate
a.goToAndStop(Math.min((t - t0) * ICON_FR, ICON_FRAMES - 1), true);

Licensing: LottieFiles free animations are "Free to use under the Lottie Simple License" — usable in a client ad, but you may not resell/redistribute the file itself. Record name + author per asset in a credits file next to the project. Do not lift assets from sites without a clear licence, and never rip elements off a design gallery into a paid deliverable — rebuild the style in HTML/CSS instead.

5. Build in ONE ffmpeg pass

Trim → concat → grade → text → encode in a single filter_complex from the source clips. Never cut a finished render into shorts — that's a second generation of loss. Re-use of one input needs split/asplit.

Audio chain (social spec, verified):

highpass=f=85,afftdn=nr=12:nf=-30,deesser=i=0.4,
loudnorm=I=-14:TP=-1.5:LRA=11,alimiter=limit=0.95

Video encode: -c:v libx264 -crf 17 -preset slow -pix_fmt yuv420p -movflags +faststart Audio encode: -c:a aac -b:a 192k -ar 48000

Put a 20–60ms afade in/out on every segment to kill join clicks: afade=t=in:d=0.03,afade=t=out:st=<dur-0.08>:d=0.08

6. Verify — always, every time

1. Re-transcribe the render. Confirms the story reads and nothing clipped. Transcribing a long file drops words — if the CTA looks missing, transcribe just the tail before believing it.
2. A/V drift: -map 0:v -f null - vs -map 0:a -f null - must match.
3. Loudness: re-run loudnorm in print_format=summary; expect ≈ -14 LUFS / -1.5 dBTP.
4. Look at frames at every text overlay and every treated shot.

Gotchas (all learned the hard way — trust these)

• delogo is unusable for AI watermarks. It leaves a vertical smear and, over busy shots, an obvious blurred rectangle — worse than the watermark. Use crop+rescale instead. For Veo/Gemini 720x1280 clips the ✦ sparkle sits at x 580–622, y 1137–1179 (fixed in every clip; semi-transparent so it looks absent over bright shots — it isn't):

  crop=637:1133:41:0,scale=720:1280:flags=lanczos,unsharp=5:5:0.4:5:5:0.0
  

  Costs ~13% zoom + mild upscale. Note SynthID (invisible) survives this — only the visible mark goes. Removing it is against Google's terms; it's a disclosure question, so surface it and let the user decide rather than doing it silently.

• tpad does NOT work after trim/setpts — it silently pads nothing and you get audio longer than video. Build freeze/end cards as a real looped image input (-loop 1 -t 2.5 -i card.png) plus anullsrc audio, and concat them.

• -shortest will silently truncate a padded stream. Prefer explicit -t.

• drawtext needs an explicit fontfile — fontconfig is unavailable and drawtext fails with "Cannot load default config file".

  drawtext=fontfile='C\:/Windows/Fonts/seguibl.ttf':textfile='work/txt/cta.txt':
  fontcolor=white:fontsize=52:borderw=3:bordercolor=black@0.75:x=(w-tw)/2:y=h*0.70:
  enable='between(t,8.0,9.1)'
  

  Apostrophes break the filter parser. Use textfile= with a UTF-8 file and a curly ’ (U+2019) — never an inline '.

• mjpeg can't encode frames from a looped-PNG/blurred card ("Non full-range YUV is non-standard"). Export those frames as .png for inspection.

• AI retake convention: two clips with the same prompt name and different timestamps = the later one is the fix. Compare transcripts — the earlier take usually has broken grammar. Drop the earlier VO, but its visuals are still usable as silent b-roll.

• Check for new clips mid-session. Users generate more while you work, and delete rejects. Re-list assets/ before a final build.

• SVG scale() transforms about origin (0,0), not the element's centre. A plain scale(0.7) on a <g> drags it toward the top-left. Scale about a point with translate(cx-cx*s, cy-cy*s) scale(s). (CSS transforms don't have this problem — transform-origin handles it, but SVG presentation attributes ignore it.)

• Two texts sharing one y must not cross-fade. A→out 7.95-8.15 while B→in 8.10-8.42 stacks both for 50ms — one visibly wrong frame. Leave a real gap (A clears 8.04, B starts 8.10). Shipped undetected in the dark cut; only caught by sampling a frame inside the window. Grab a frame in every hand-off window.

• Anchor an underline/rule to the WORD's rect, not the block's. bottom:-14px on a wrapped two-line block draws under the last line — i.e. the wrong word. Measure live: ul.top = wordRect.bottom - blockRect.top + 4. Never hardcode x/width.

Deliverable conventions

9:16 720x1280 @ 24fps. Name by concept, not number: v1_invisible.mp4, v2_selftaught.mp4, v3_nevergetsold.mp4. Keep the watermarked original alongside the cleaned one. Ads land at ~15s or ~30s.

Honest limits — say these out loud

Veo/Gemini source is 720x1280 @ ~1.9 Mbps. You cannot make it sharper than the source. High CRF + light unsharp is the ceiling; upscaling to 1080x1920 inflates the file without adding detail. If a user asks for "very clear", say this rather than pretending.

Runtime math: N seconds of unique VO cannot become 3×30s of non-overlapping ads. Decoupling + treatments stretch visuals a long way, but if the user wants three truly distinct concepts, the honest answer may be "generate ~5 more clips" — propose the missing scenes (offer/pricing, testimonial) rather than silently shipping three near-identical cuts.