Resolve Autocut

Resolve Autocut

Turn raw gameplay recordings into edited videos with DaVinci Resolve. You do the judgment (thresholds, which moments matter, titles); the bundled scripts do the grunt work (silence detection, cutting, rendering). Run each script with skill_run_script — pass name: "resolve-autocut", a script path, and args. Every script returns one JSON object; check ok and stop on ok:false, reporting the error.

Requires DaVinci Resolve Studio running with a project open for the timeline/render steps. The ffmpeg steps (analyze, captions, shorts, thumbnails, QC, normalize) work without it.

The core pipeline (what the human mostly wants)

1. Analyze — detect silence → keep-list, and loudness peaks → highlight moments: scripts/analyze.py <video.mp4> --out cutlist.json Tune per game if cuts feel wrong: --db -35 (silence threshold), --min-silence 0.6, --pad 0.15 (breathing room around cuts). Louder games → lower --db (e.g. -30). Two-track gaming recordings (mic and game as separate audio streams, OBS-style): pass --mic-stream N --game-stream M — a segment is kept when there's speech or in-game action, and cut only when both are dead, so a loud fight without commentary survives. .mkv track titles auto-detect; .mp4/.mov usually lose titles, so pass the flags — and check two_track and warning in the result: a warning means it fell back to single-track and you should re-run with explicit streams. --action-sense 0.35 tunes how loud (vs the game's own baseline) counts as action; lower protects more. Highlight moments carry a source tag (mic = reaction, game = action) — great for picking shorts. Per-game presets: when the human approves thresholds for a game, remember them (agent memory) keyed by the game — db, pad, min_silence, action_sense, mic/game streams — and start the next video of that game from those instead of the defaults.
2. Build the timeline in Resolve from the keep-list (adds highlight markers): scripts/build_timeline.py --cutlist cutlist.json --timeline "Auto Cut" Source is .mkv/.webm? Resolve won't import it reliably — remux to .mov first (scripts/remux.py <in.mkv>) and run the pipeline on the result. See Resolve-friendly source. On the first run for a new game, STOP HERE and ask the human to eyeball the cuts — bad thresholds eat punchlines. Re-analyze with adjusted --db/--pad if needed.
3. Render the reviewed timeline (only after the human approves): scripts/render.py --outdir <drafts_dir> --name <VideoName> --preset "<preset>"

The payoffs (add these on top)

• Captions — scripts/transcribe.py <video> --outdir captions --model base → .srt + .json. Add to the timeline with build_timeline.py --srt captions/<name>.srt. Add --words for word-level timestamps (required by the guardian pass). pyenv shims can hide the whisper binary — set WHISPER_CMD to the env's real path (e.g. ~/.pyenv/versions/<env>/bin/whisper).
• Guardian pass (kid-channel safety) — auto-bleep profanity, flag PII, before anyone reviews: scripts/guardian.py <video> --transcript captions/<name>.json --pii-file pii.txt Transcribe with --words first, and prefer --model base or better — bleep accuracy is capped by transcript accuracy. Built-in EN+TR wordlists (--lang en,tr); extend per channel with --extra-words. The PII file is hashed: each line sha256(term) label, so the sensitive terms never exist in plaintext — add entries with guardian.py --pii-file pii.txt --add "<term>" --label school (never write terms by other means), and matches are reported as the label only (school @ 4:12). PII is flagged, never auto-muted unless --mute-pii — a human decides what an identity leak means. Matching is exact on the transcript: if a name was flagged with an odd spelling, --add that variant. --mode mute silences instead of a 1 kHz tone. When anything matches, a redacted transcript (<name>.redacted.json — profanity → [bleep], PII → [label]) is written: every later text step (titles, chapters, shorts --captions) must use it, or the raw term can end up burned into the video or in the model's context. Writes <name>.guarded.<ext> + <name>.guardian.json; run it on the source before build_timeline so cuts happen on the guarded file (duration is unchanged, so cutlist timings stay valid), and lead your report with the safety summary (bleeps applied, PII flags with timestamps). Two-track sources: bleep the mic — guardian.py --stream <mic> edits only that stream and passes the game track through untouched; transcribe the same stream (transcribe.py --stream <mic>), since whisper alone reads only the first stream.
• Shorts / Reels — vertical 9:16 (1080x1920) clips at the loudest moments: scripts/shorts.py <video> --cutlist cutlist.json --count 3 (or --at <sec> for exact spots). Combine loud moments with transcript keywords ("no way", "let's go", laughter) to pick the best. Recorded a facecam alongside? Add --overlay cam.mp4 to composite it into a corner of each short, cut to the same moment (both sources assumed start-aligned at 0). Tune with --overlay-scale 0.30 (Shorts read best ~0.3-0.4 of the narrow frame), --overlay-corner, --overlay-margin. The short's audio comes from the base <video> track — point that at whichever file carries the commentary. Moments are picked best-first by score from the cutlist (two-track cutlists tag each with mic/game — reactions vs action). Add --captions captions/<name>.json (a --words transcript — the mic one on two-track sources) to burn big word-by-word captions in the vertical style; --caption-size 96, --caption-pos 0.70 tune it. Caption text is rendered with Pillow and composited as image overlays, so it works on slim ffmpeg builds that lack ass/subtitles/drawtext (current Homebrew bottles do).
• Cold open — prepend the best moment as a ~5s teaser before everything (gaming retention 101): scripts/hook.py --cutlist cutlist.json → cutlist.hook.json; build the timeline from that file instead. --pick game|mic chooses action vs reaction, --at forces a moment, --duration/--lead shape the teaser. The moment plays again in context later — that's the point.
• Punch-ins — add --punch-in --zoom 1.08 to build_timeline.py to zoom on loud moments.
• Overlay / picture-in-picture — composite a second clip onto a corner, cut in lockstep with the base track: build_timeline.py --cutlist cutlist.json --overlay clip.mp4. Works for a facecam, reaction cam, or webcam bug recorded alongside the gameplay. The overlay is assumed to start aligned at 0 and is clamped if it ends early. Tune with --overlay-scale 0.25, --overlay-corner bottom-left|bottom-right|top-left|top-right, and --overlay-margin 0.03.
• Titles / description / tags / chapters — read captions/<name>.json and the analyze result; write these yourself from the transcript. Chapters = timestamps of topic/scene changes.
• Thumbnails — scripts/thumbnails.py <video> --cutlist cutlist.json --count 5, then pick the best frame and propose a caption.
• Loudness — scripts/normalize.py <video> --lufs -14 (YouTube target) before rendering, if the source is quiet/inconsistent.
• Intro / outro / music — pass --intro intro.mov --outro outro.mov to build_timeline.py.
• QC — scripts/qc.py <video> flags black frames, freezes, and dead audio; run it on the draft before telling the human it's ready.
• Big capture? Make a proxy first — a raw recording is often far more bitrate than a YouTube edit needs (e.g. a 9 GB gameplay file). scripts/proxy.py <in> transcodes it to a compact 1080p H.264 .mp4 (usually a fraction of the size) that's also Resolve-friendly, so it replaces the remux step. Analyze/edit on the proxy; publish from it, or relink to the original in Resolve for the final export (cuts are frame-accurate). Tune with --height 1080 and --crf 21 (lower = better/larger).
• Resolve-friendly source — DaVinci Resolve won't reliably import .mkv/.webm (often no audio). Remux to a .mov first: scripts/remux.py <in.mkv> — copies the video stream when it's already edit-friendly (H.264/HEVC/ProRes/…) and converts audio to PCM. For VP9/AV1 sources add --transcode (defaults to ProRes; --video-codec h264|dnxhr to choose). The ffmpeg steps (analyze, captions, shorts, thumbnails, QC) read .mkv directly — only the Resolve steps need this.
• No DaVinci Resolve Studio? Export a timeline to import. The free version can't be scripted (so build_timeline.py / render.py won't connect), but it can import a timeline. Build the cut — including a synced second-track overlay — as FCPXML and import it by hand: scripts/export_fcpxml.py --cutlist cutlist.json [--overlay cam.mp4 --overlay-scale 0.25] → in Resolve, File > Import > Timeline. The cuts and both tracks come in ready at the overlay's scale; Resolve ignores FCPXML transform position on import, so the overlay lands centered — set its Transform position once, then Paste Attributes (Transform) to the other overlay clips. Then render.

Finding the parts a recording got wrong

scripts/qc.py <video>

Black frames, frozen frames, dead audio — and dropped frames, which is the one you cannot see coming. A capture that could not keep up (a busy machine, an unresponsive app) writes the previous frame again rather than skipping a slot, so the container stays flawless constant-rate while the picture stops moving. Every timing check passes and the video still judders, in patches, usually not the whole recording.

A duplicated frame is bit-identical to the one before it. A person merely sitting still never is — sensor noise guarantees two genuine captures differ — and that is the whole discriminator. The report gives a share of the recording, the seconds of frozen picture, and the windows worth scrubbing to:

"dropped_frames": {
  "frames": 73813, "duplicated": 3417, "share": 0.0463, "seconds_lost": 113.9,
  "windows": [{"start": 165.0, "end": 180.0, "share": 0.311, "frames": 141}, …]
}

It decodes every frame, which is the slow part of a QC pass — about 25 seconds for 40 minutes of 720p30. --no-dropped-frames skips it. --dropped-window and --dropped-flag set how coarsely windows are reported and how bad one has to be to appear.

What it will not catch: frames that are all unique but were captured unevenly and stamped onto a regular grid — motion that judders without repeating. That needs a motion-smoothness measure, which is far more expensive and much easier to get wrong, so it is not attempted here.

Cheaper checks that do not work, in case they look tempting: frame timestamps show nothing (the stream is perfect CFR); encoded frame size misses stalls that land inside busy motion, where duplicates hide among large frames; and mpdecimate over-reports badly, because it drops frames that merely look alike — a still subject reads as a total freeze.

Dropping the overlay where the camera stalled

scripts/trim_overlay.py --xml timeline.fcpxml [--min-fps 22] [--out trimmed.fcpxml]

Reads a two-track timeline, finds the stretches where the overlay capture was not keeping up, and removes the overlay from every split those stretches touch. The split keeps its gameplay — full frame, no picture-in-picture — and the cut itself is untouched. The input timeline is never modified; a new file is written.

It does not look for frozen runs, because there are none to find. A stalling capture misses one frame slot at a time and repeats the previous picture to fill it. Measured on a real 40-minute facecam: 3,417 repeated frames in 3,409 separate runs, median run length one frame, never longer than three. Cutting those individually would strobe the picture in and out over a thousand times. What a viewer notices is the rate collapsing for a stretch — whole seconds delivering 18 of 30 frames — so that is what is measured: effective frame rate over a sliding second.

Two faults, because neither test finds the other.

Frozen (--min-fps, default 22) — the capture missed a slot and repeated the previous picture, so motion is exactly zero. Counted per second, that is an effective frame rate. Works even when nothing in shot is moving.

Slow (--min-advancing, default off) — the picture keeps changing but arrives in lurches: a few frames carry all the movement and the rest are dead. Motion never reaches zero so the frozen test never fires, yet it reads on screen as juddering. Measured relative to each second's own motion — how many of its frames carry any — so a subject simply sitting still is not mistaken for a stalling camera, and seconds with nothing moving are skipped entirely, because nothing that still can judder.

Healthy footage delivers 18–24 advancing frames per second; a stalling camera drops to 1–5. Both dials on the same 40-minute facecam:

The slow test escalates quickly, so it is off unless you ask. On the one recording judged on screen so far, both 5 and 10 were rated good by the person whose video it was — 65 and 103 of 337 splits respectively. So the whole 5-10 band is usable and the choice inside it is preference, not correctness: 5 keeps more face and lets more judder through, 10 the reverse. Watch the result before trusting a number, because how much missing face beats how much judder is a taste call this cannot make for you.

Above roughly 15 it stops being a judgement call: two thirds of the timeline loses the overlay, which is a different edit rather than a cleaner one. Re-running against the same input is free; it is only ever read.

Point it at the capture, never at a copy — and it enforces this for you. A re-encoded file cannot be judged: compression perturbs a repeated frame just enough to break the byte-for-byte match the measure rests on. On a real pair the capture showed 4.6% repeated frames and its conformed copy showed 0.6% — not a small error, the finding inverted. Since the exporter conforms differing-rate overlays by default, the timeline's own overlay is usually the wrong file to look at, so any derived name (.conform60, .exact30) is walked back to the original automatically. --overlay-source is optional and gets the same treatment.

The result reports the moments it found, which splits lost their overlay, and how many seconds of face remain — check it before importing, and run verify_fcpxml.py on the output as usual.

Combining two videos on one timeline

Gameplay plus a facecam, or any capture plus a second angle. The cutlist drives the cut; the overlay is placed at the same source timestamp on a second track, so both are cut at the same moments and stay in step.

scripts/analyze.py      --video gameplay.mp4                       # -> cutlist.json
scripts/export_fcpxml.py --cutlist cutlist.json --overlay facecam.mp4 --overlay-scale 0.25
scripts/verify_fcpxml.py --xml gameplay.fcpxml                     # verdict: clean

Then in Resolve: File > Import > Timeline.

Two rules, and only two.

1. The two recordings must start together. The overlay is placed at the gameplay's source time, which assumes both cameras were rolling from the same instant — true whenever one capture wrote both files (an OBS Source Record filter, say). They may end at different times; the overlay is clamped and the difference ignored. If they genuinely started apart, this skill has no offset for that: trim the later one first so both begin at the same moment.
2. Their frame rates do not have to match, but must be stated exactly. 60 against 30 is a clean 2:1 and stays locked. 59.94 against 30 is not — see below. The exporter handles this for you and verify_fcpxml.py fails the export if anything is left ambiguous.

Everything else is automatic, and you pass no flags for any of it: each asset is declared with its own real format and dimensions, every referenced file is rewritten onto an exact frame grid first (lossless, seconds), and an overlay whose rate differs from the timeline is re-encoded to match.

That last step is a lossy encode and the slow part — roughly 2 minutes for 40 minutes of 720p, cached next to the source and reused until it changes. It is on by default because two files at one rate is the only arrangement verified on screen; see below for the reasoning and how to skip it.

Do not skip the verify step. Every sync bug this pipeline has ever had was invisible in the XML and only showed up twenty minutes into a Resolve import. verdict: clean is the gate.

Only one overlay is supported — two video tracks total. A third angle would need a second lane and a position per lane, which the exporter does not emit today.

Resolve ignores FCPXML transform position on import, so the overlay lands centered: set its Transform position once, then Paste Attributes (Transform) to the other overlay clips.

Check the export before you import it

scripts/verify_fcpxml.py --xml timeline.fcpxml

Reads the FCPXML back and checks it against the media it points at: that every clip starts where the previous one ended, that each overlay sits at its parent's left edge and names the same source moment, that every in-point is a whole frame of the file it addresses, and that each asset's declared format is the format that file actually has. verdict: clean means the timeline is in sync as written. Run it whenever you export — the errors it finds are invisible in the XML and only show up as drift after a twenty-minute import.

Why frame rates must be stated exactly

Two clips do not need the same frame rate in principle — 60 against 30 is a clean 2:1. What they must do is state their rates exactly: 59.94 against 30 is not clean, and that is a difference in metadata, not in footage. In practice the exporter still conforms a differing-rate overlay to the timeline, because only the one-rate arrangement has been verified on screen — see below.

This is the bug that made a facecam look progressively late, and it is worth understanding because no amount of correct arithmetic in the timeline fixes it. OBS writes MP4s on a 16000-tick timebase, and 16000 divides by neither 60 nor 30. A 60fps frame should be 266.667 ticks, which cannot be written, so the deltas alternate 272 and 256. An NLE does not average those — it takes the common one and snaps to the nearest broadcast rate, and 16000/272 is 58.82, which snaps to 59.94. The same file's 30fps sibling has a modal delta of 528 ticks → 30.303 → snaps correctly to 30.000.

So one clip is read 0.1% slow and the other is read right. Their ratio becomes 1.998 where the truth is 2.000, and 0.1% of a 41-minute recording is 2.4 seconds of drift that grows from zero — indistinguishable by eye from two devices with different clocks, and unfixable by any sync ratio, because the recordings were never out of sync.

export_fcpxml.py therefore rewrites every file it references onto an exact frame grid by default. It is lossless: the video bitstream is unwrapped and re-muxed byte for byte, the audio is copied untouched, the original start offset is preserved, and only the container's timing table changes. A 14 GB capture takes under a minute; a file that is already exact is skipped. New files are written next to the originals (<name>.exact60.mp4), so budget the disk. --no-exact-fps turns it off, which is almost never what you want.

The exporter also declares each asset with its own format — announcing a 30fps 720p facecam as 60fps 1080p invites the NLE to conform it and computes the PiP scale against a frame size it does not have — and snaps the cut points to the grid the two files share (for 60 and 30, every even gameplay frame). A 30fps overlay has no frame at 11.25 s, so a cut there would otherwise leave the NLE to round, landing the overlay up to half a frame off on a different side of each cut. Moving the cut to the nearest moment that is a whole frame of both files costs at most half a frame of silence, on a boundary chosen because nobody was talking.

Check what an NLE will read off any capture:

ffprobe -v error -select_streams v:0 -show_entries stream=avg_frame_rate,time_base -of csv media.mp4

60/1 with a timebase divisible by 60 is healthy. A ratio like 73832000/1230533 is the bug above, even though it equals 60.0000 — it is the container admitting it could not write the rate exactly. verify_fcpxml.py reports the same thing as read_as and says so in plain language.

Why a differing-rate overlay is re-encoded by default

When the overlay's rate is not the timeline's, it is re-encoded to match — each frame duplicated across the two slots it now occupies. On by default, and it costs a lossy encode (~2 min for 40 minutes of 720p), so it is worth being straight about why.

Exact frame grids fix the metadata, and that is what actually caused the drift described above. What they cannot fix is how the NLE conforms a clip whose rate is not the timeline's, because an FCPXML has no way to state it. Two files at one rate is the only arrangement verified end to end by a human watching the result. A 30fps overlay on a 60fps timeline is exact arithmetic and should hold — every check here passes on it — but that has never been confirmed on screen, and Resolve's conform behaviour is precisely what made this hard to find in the first place.

So the default is the tested arrangement rather than the elegant one. On a picture-in-picture scaled to a quarter of frame, the extra generation is invisible; a drift you find twenty minutes into an import is not.

--no-overlay-conform takes the lossless path instead: seconds instead of minutes, no quality cost, and the conform left to the NLE. Reasonable if you will check the result — and if you do, say whether it held, because that is the one piece of evidence this recommendation is missing.

The conform is verified rather than assumed: frames are pulled from the original and the copy at the same timestamps and compared against deliberately misaligned ones, and a conform that moved the picture in time is refused. Reported as conform: "encoded, timing verified (+17.2dB)". Footage too still to tell says inconclusive instead of claiming a pass.

If the overlay still drifts out of sync

The overlay is placed at the same source time as the gameplay, because two files written by one capture share one clock. Check the far end of a long edit before trusting it: play the last minute and watch the lips.

Run verify_fcpxml.py first, every time. A rate the NLE reads wrong, a declared-format mismatch, or a misanchored overlay all look exactly like clock drift and none of them are — this is the mistake that cost several rounds of chasing a correction for a problem that did not exist. Only if the verdict is clean and it still drifts did the two devices really run at different rates, and only then does --overlay-sync-ratio apply. Prefer a ratio measured off a real sync point — find one moment near the start and one near the end that you can identify in both, and divide:

ratio = (overlay_time_at_end - overlay_time_at_start) / (gameplay_time_at_end - gameplay_time_at_start)

--overlay-sync-ratio 0 derives it from the two file durations instead. Reach for that last. Two recordings also differ in length when one simply stopped a moment before the other, and a ratio cannot tell that apart from a clock difference — so on footage that was merely stopped a moment apart it manufactures drift. A 0.7 s difference across a 50-minute recording becomes a 224 ppm ramp: 80 ms of lag six minutes in, two thirds of a second by the end.

A silent overlay track is worth noticing here too: if the facecam has no audio, the voice is the gameplay mic, so a drift shows up as lip-sync and no fixed offset will fix it.

The unattended loop (watch folder)

For a recurring "drop a recording, review a draft later" workflow, drive the pipeline from a watch folder with scripts/next_drop.py — it owns the state so no file is processed twice and a file still being recorded is never grabbed (two-scan size stability + --min-age):

• next_drop.py --drop <dir> --claim → the next stable, unprocessed recording (or next: null — then do nothing and end quietly).
• Run the full pipeline on it, write a REPORT.md next to the outputs (safety summary first), then --done <file> --note "…" — or --fail <file> on a blocker, and don't retry failed files without being asked.
• Process one recording per run; the next run takes the next file. Never publish — the loop produces drafts and reports, a human ships.

Wire it to the agent's heartbeat (HEARTBEAT.md + heartbeat.enabled) for a true evening loop.

Rules

• Pass absolute paths for videos and outputs. Put working files (cutlist.json, captions/, shorts/, thumbs/) in the space workspace or the footage folder — never inside the skill dir.
• Long jobs: transcribe.py and render.py can be slow. Use a small whisper --model for long videos, and render.py --wait-seconds 0 to queue a long render without blocking.
• A timeline is not done until verify_fcpxml.py says clean. Run it after every export_fcpxml.py and put the verdict in your report. The failures it catches — a rate the NLE will misread, an asset declared with the wrong format, an overlay off its parent's start — are all invisible in the XML and only surface as drift after a long import. Never reach for --overlay-sync-ratio before the verdict is clean; a misread frame rate looks identical to a clock difference and no ratio fixes it.
• Never auto-publish. Produce drafts, report what you did (cuts made, duration before/after, markers, shorts, QC issues), and let the human approve the final render/upload.
• Kid-channel footage: guardian first. Run the guardian pass before building the timeline and put its result (bleeps applied, PII flags) at the top of your report. Never skip it because the footage "seems clean" — the pass is cheap and the transcript already exists for captions.
• If Resolve isn't running, the timeline/render scripts return a clear error — do the ffmpeg steps, then ask the human to open Resolve.