Remotion Cut

Automatic cut with Remotion

/cut writes the cut list for a human editor. This skill executes it: raw footage in, rendered reel out. The script stays the single source of truth — nothing is cut that the script does not call for.

Requirements

• A local environment (Claude Code) in a checkout of this repo — the skill needs the file system, ffmpeg and Node ≥ 20. Uploading it to claude.ai does nothing: there is no footage there.
• The remotion/ template from this repo, copied per brand (see its README), npm install run once.
• A shoot-ready script (from /script) or a finished cut list (from /cut). Without one of the two, stop and ask — guessing the story from footage produces a montage, not a video.

Raw footage is read-only. Every ffmpeg call writes to a new file; nothing overwrites the source.

1 · Survey the footage

ffprobe every source file: resolution, fps, duration, audio. Note rotation and odd frame rates — they decide the normalize command in step 5.

2 · Find the takes

Speech lives between silences. Detect the silences:

ffmpeg -i raw.mp4 -af silencedetect=noise=-35dB:d=0.45 -f null - 2>&1 | grep silence_

-35dB / 0.45s are starting values. A quiet room wants -40dB; a speaker who pauses mid-sentence wants a longer d. If the regions come out absurdly long or short, tune these first — everything downstream depends on them.

The speech regions between the silences are the take candidates. When running ffmpeg inside a shell loop, append < /dev/null — ffmpeg reads stdin and will otherwise swallow the loop input.

3 · Transcribe the regions

Extract each region as 16 kHz mono WAV and transcribe with word timestamps:

ffmpeg -ss <start> -to <end> -i raw.mp4 -ar 16000 -ac 1 work/region-07.wav < /dev/null
node scripts/transcribe.mjs work/region-*.wav --lang <language>

The result is a text with word timestamps per region — the map of the whole shoot: every attempt, every false start, every aside.

4 · Select the takes

Match regions against the script beats. Rules:

• The last clean take per beat wins. People warm up; attempt four beats attempt one. Earlier takes only when the last one has a technical fault.
• Complete and clean means: full wording of the beat, no re-start, no direction from off-camera, no laugh in the middle — the transcript shows all of this.
• Cut edges sit on real speech pauses, measured in the audio — the silencedetect timestamps, not estimates. In on the first word, out after the last one.
• A beat the footage does not cover is reported, not papered over.

Show the selection before cutting: beat → region timecode → transcript wording. This is the moment for the user to swap a take, and it costs one message instead of a re-render.

5 · Build the clips

Trim each selected take and normalize to the composition format:

ffmpeg -ss <in> -to <out> -i raw.mp4 \
  -vf "crop=ih*9/16:ih,scale=1080:1920" -r 30 \
  -c:v libx264 -crf 18 -preset slow -c:a aac \
  public/clips/01-hook.mp4 < /dev/null

Name clips by position and beat (01-hook, 02-stat, …). Adjust the crop to the actual source geometry from step 1 — the example assumes landscape or oversized portrait material centered on the subject.

6 · Write the timeline

Fill src/edit/timeline.ts: one Shot per clip with sourceIn (raw-footage timecode — keeps every cut traceable), duration, and the zoom values.

Zoom carries the edit. With a static camera it is the only perspective change there is:

• Base is 1.00. Every cut changes the focal width, or it reads as a jump cut.
• Step in on numbers and climaxes, step back out for theses — the thesis gets air, the number gets pressure.
• The person must stay inside the safe zone at every zoom level. With a centered subject that caps the zoom around 1.14; tighter only when the framing verifiably allows it — a two-state scheme (1.00 / 1.30, alternating on every cut) is fine when the head measures with 150 px to spare.
• zoomTo ≈ zoom + 0.015 — a drift you feel, not one you see.
• Consecutive cuts building to one point may climb (1.06 → 1.13 → 1.20); the release afterwards goes wide.

Cut without silences

If the brief asks to remove every pause, breath and stall, don't hand-pick: build it as a pipeline, because you will re-cut it several times.

1. Extract each clip's audio as 16 kHz mono and compute RMS per 10 ms window in plain Python (wave/array — the bundled ffmpeg has no astats).
2. Silence = below −40 dB for ≥ 0.10 s. Cut only silences ≥ 0.30 s; keep 0.05–0.07 s after the last sound and 0.03–0.06 s before the next. Clip margins 0.03–0.06 s front, 0.06–0.08 s back.
3. Write a plan (JSON) with the kept windows per clip, then cut the sub-clips from the raw footage at sourceIn + offset and generate the timeline file from the plan. Split clips get -a, -b, -c suffixes.
4. Keep manual decisions as per-clip overrides in the plan (a longer tail for a soft sentence ending, a fixed window) and re-cut only those parts.
5. Pass audioFade={1} to Shot — with 0.06 s margins a 2-frame fade lands on the first sound.
6. Never treat a quiet voiced block right after a word as a breath. It is the word ending; only real silence goes.

7 · Overlays

Overlays sit on the spoken word, not on the cut — the number appears the moment it is said. At the earliest 0.3 s after the cut, never two at once, the last one ends before the video does. Exception: a comment trigger holds to the last frame, because reels loop — half a second of empty freeze frame reads as a mistake, not as an ending.

Whether text runs dark or light depends on the footage behind it — check the actual frame, not the default (onLight on the stat overlay).

8 · Subtitles

Transcribe the final clips, not the raw regions — the timings must match what is rendered:

for f in public/clips/*.mp4; do
  ffmpeg -i "$f" -ar 16000 -ac 1 "work/$(basename "$f" .mp4).wav" < /dev/null
done
node scripts/transcribe.mjs work/*.wav --lang <language> --merge public/subtitles.json

Then the correction pass: names, technical terms and numbers Whisper gets wrong, fixed in public/subtitles.json. /cut's subtitle correction template lists the candidates. Subtitle colors and type come from theme.ts — from FORMATS.md, never invented. If the foundation does not specify them, ask.

9 · Calibrate the theme

Once per brand, before the first render:

• theme.ts — colors from BRAND.md / FORMATS.md.
• fonts.ts — display and subtitle family.
• SUBJECT_OFFSET — measure the face height in a clip frame; the formula is in theme.ts. Without it the head slides up on every zoom-in.

10 · Check and render

npm run dev                                # Studio — user checks the cut
npx remotion render Reel out/<name>.mp4    # final render

Before handing over, verify against the script: total length vs. planned length, every beat present, overlays on their words, no subtitle under a card. After a cut without silences also: transcribe the rendered audio in full and read it against the script (word endings!), measure the sample jump at every edge for clicks, measure the head position at 75 % of every part, and list the remaining pauses with timestamps. Report deviations with numbers ("planned 47 s, rendered 52 s — beat 3 runs 4 s long"), not as "roughly fine".

What this skill does not do

It does not choose the story — that is the script's job. It does not publish. It does not touch raw footage. And it does not replace the check in Studio: the last look before rendering belongs to a human.