# Video Pipeline

> End-to-end video editing pipeline. Takes a raw talking recording and produces a publish-ready video with fast cuts, word-by-word captions, and optional multicam b-roll. Use when the user says "edit this video", "process this recording", "cut this video", "/video", or drops a video path with intent to publish. Requires parakeet-transcribe, video-edit, ffmpeg, and bun (see the video-alchemy repo).

- Skill: `kingbootoshi/video-pipeline` (Agent Skill)
- Install (CLI): `npx skillmds@latest add kingbootoshi/video-pipeline`
- Raw SKILL.md: https://api.skillmd.com/api/skills/kingbootoshi/video-pipeline/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: DevOps & Infra
- Author: kingbootoshi (https://skillmd.com/u/kingbootoshi)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/kingbootoshi/video-pipeline

---


# video-pipeline

Take `raw.mp4` from "hit record and talk" to `final.mp4` with burned-in
word-by-word captions. The edit is data: a cut list in JSON, captions in JSON,
b-roll timing in JSON. You are the editor; the human gives notes in plain
language and you convert each note into a one-line change and re-run.

## Pipeline

```
raw.mp4
  ├─ 1. transcribe        parakeet-transcribe → word-level ms timestamps
  ├─ 2. propose cuts      read the FULL transcript, write cuts.json
  ├─ 3. CHECKPOINT        show the human the proposed cuts, wait for "go"
  ├─ 4. cut               video-edit cut → cut.mp4 + remapped captions
  ├─ 5. render            Remotion → burned-in captions (+ b-roll)
  └─ 6. revise            notes → edit cut list → re-run 4-5
```

## Step 1 — Transcribe

```bash
parakeet-transcribe raw.mp4 -v
```

Produces `raw.json` (words with `start_ms`/`end_ms`), `raw.srt`, `raw.vtt`.
Create a project folder; keep every artifact in it.

## Step 2 — Propose cuts (the creative step)

Read the transcript IN FULL first. Then build the removal list.

**Cut on sight:** cold-open throat clearing, false starts, stutter stacks,
leading fillers ("So", "Um", "Like"), restatements, self-corrections, abandoned
threads, tangents, progress-report filler ("still processing..."). When two
sentences say the same thing, kill the weaker one entirely.

**Never cut:** personality phrasing, intentional emphasis, live reactions and
discoveries.

**Gap squeeze (automatic):** scan ALL tokens (including punctuation tokens -
they carry real duration) for silences > 1.5s; remove each gap leaving ~0.42s
after the last token and ~0.16s before the next. This alone often removes
30-40% of runtime invisibly.

**Boundary safety (prevents eaten words):**
- Run silence detection over all tokens INCLUDING punctuation. A `.` after the
  final word owns real time; starting a cut inside it deletes the whole word
  group.
- If a boundary lands inside any token, snap the cut start FORWARD past the
  token (never backward) and the cut end forward to the token's end.
- Never resume a kept segment on a filler word - extend past up to 3 leading
  fillers.
- Verify zero token straddles before cutting. Assert it programmatically.
- Watch contradiction seams: cutting from "I'll do X" straight to footage of Y
  makes the video argue with itself - cut the announcement too.

For heavy edits (keeping < half the source), write a KEEP list of surviving
ranges and generate the removals as its complement - clearer to review and to
revise.

Write `cuts.json`:

```json
{
  "video": "/abs/path/raw.mp4",
  "remove": [
    {"start_s": 12.0, "end_s": 18.5, "reason": "false start"},
    {"start_s": 45.2, "end_s": 52.1, "reason": "tangent"}
  ]
}
```

## Step 3 — CHECKPOINT

Summarize for the human: total cuts, time removed, estimated final length, the
5 most aggressive cuts with reasons, any borderline calls flagged for their
judgment. Wait for approval before rendering. Never skip this.

## Step 4 — Cut

```bash
video-edit cut raw.mp4 cuts.json --out-dir . --transcript raw.json
```

Outputs `cut.mp4`, `captions.json` (remapped to the new timeline),
`captions.srt`, `report.md`. Read `report.md` for the output duration and lint
findings.

Then fix transcription mishears in the caption files only (audio is already
correct). Keep a small find→replace map for the video's domain terms and verify
each fix landed.

## Step 5 — Render

Copy the `remotion-template/` from the video-alchemy repo into the project:

```bash
cp cut.mp4 remotion/public/source.mp4
cp captions.json remotion/src/captions.json
```

- Probe the source: `ffprobe -v error -select_streams v:0 -show_entries stream=width,height,r_frame_rate -of csv=p=0 cut.mp4`
- Set `WIDTH`/`HEIGHT`/`FPS` in `src/Root.tsx` to EXACTLY the probed values -
  no scaling, no stretching.
- Set `TOTAL_S` in `src/Main.tsx` from report.md's output duration.
- Caption font size ≈ height × 0.058.
- `bun install && bun run build` → `out/final.mp4`.

Before the full render, render 3-4 still frames at key moments
(`bunx remotion still Main out/check.png --frame=N`) and LOOK at them - verify
captions sized right, cutaways landing, nothing colliding.

## Multicam b-roll (when a second camera exists)

If the second camera captured the speaker's voice: transcribe it too, align 3-4
shared phrases between the transcripts, and derive the constant offset
(`broll_time = main_time + offset`; agreement within ~150ms across several
phrases confirms it). Extract cutaway clips with ffmpeg, map their positions
through the cut list onto the edited timeline, and list them in
`src/broll-spec.json`. Cutaways render muted over the continuous main audio.

## Step 6 — Revision loop

Every note maps to a small data change:

- "jarring cut at M:SS" → adjust that boundary to end after the sentence's
  punctuation token, in real silence
- "kill the tangent about X" → one more remove entry (find it in the transcript)
- "it drags in the middle" → hunt restatements and progress filler there
- "start on the second sentence" → move the first keep boundary

Re-run steps 4-5 after each round. Rebuild b-roll timing whenever the cut list
changes - insert positions are derived from it.

## Rules

- Never skip the cut checkpoint.
- Never cut against a transcript mishear - check the words around every
  boundary.
- Verify with evidence: probe durations, render check frames, measure audio
  levels (`ffmpeg -af volumedetect`) rather than assuming.
- Keep every generator script (`build_cuts.py` etc.) in the project folder so
  the edit is reproducible end to end.