# Video Editing > Non-destructive video editing in Clips — the editsJson model, trim / split / cut / speed / blur, transcript-based editing, and ffmpeg.wasm export. Use when building the editor UI, adding a new edit operation, or wiring the export pipeline. - Skill: `builderio/video-editing` (Agent Skill) - Install (CLI): `npx skillmds@latest add builderio/video-editing` - Raw SKILL.md: https://api.skillmd.com/api/skills/builderio/video-editing/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: DevOps & Infra - Author: Builder.io (https://skillmd.com/u/builderio) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/builderio/video-editing --- # Video Editing ## When to use Reach for this skill any time you modify how recordings are edited: new edit operations, the timeline UI, transcript-driven cuts ("remove the ums"), the preview overlay, or the export flow. Editing **must** be non-destructive — the recording blob is immutable after upload. ## Data model touched - **`recordings.edits_json`** — a JSON column with the edit document. Shape: ```json { "trims": [{ "startMs": 0, "endMs": 3200, "excluded": true }], "cuts": [{ "startMs": 12000, "endMs": 13500 }], "speed": [{ "startMs": 0, "endMs": 60000, "rate": 1.5 }], "blurs": [{ "startMs": 0, "endMs": 90000, "box": { "x": 10, "y": 10, "w": 200, "h": 80 } }] } ``` - **`recordings.chapters_json`** — JSON array of `{ startMs, title }`. - **`application_state.editor-draft`** — in-progress editor state the user is previewing (recording id, playhead, preview playback speed, zoom, and edits JSON). Persisted edit operations still write directly to `edits_json` through actions. ## Rules 1. **Non-destructive.** Never re-encode on edit. The original webm/mp4 stays intact at `recordings.video_url`. Edits only change the JSON. 2. **Single source of truth.** The player renders edits at playback time — read `edits_json`, compute the virtual timeline, and skip excluded ranges via `HTMLVideoElement.currentTime` seeks. Do not fork the edit model for the editor vs the player. 3. **Export is explicit.** The user must click Export to render a new file. That call goes through `export-video` and kicks off ffmpeg.wasm (or server-side ffmpeg if the recording is long). 4. **Append, don't rewrite.** Prefer pushing a new entry into `edits_json` over editing an existing one, so undo/redo can reverse a single edit without ambiguity. ## Operations | Operation | `apply-edit` args | What it does | | --------- | ----------------------------------------------------------- | ----------------------------------------------------------- | | Trim | `--type=trim --startMs=0 --endMs=30000` | Exclude the first 30 seconds from playback | | Cut | `--type=cut --startMs=12000 --endMs=13500` | Remove a middle range — the timeline collapses | | Split | `--type=split --atMs=` | Put a cut marker at the playhead (no range removed) | | Speed | `--type=speed --startMs --endMs --speed=1.5` | Speed a range up (or down — `0.5` works too) | | Blur | `--type=blur --startMs --endMs --x --y --w --h` | Apply a blur rectangle to a time range | All of these append to `edits_json`. The action validates non-overlapping ranges per type (except `speed`, which compounds) and throws on bad input. ## Transcript-based editing Users love editing by text — click a filler word in the transcript, remove it. The editor uses `recording_transcripts.segments_json` to map words to `{ startMs, endMs }` ranges; clicking a segment creates a `cut` edit for that range. "Remove the filler words" is **not** solved inline — it delegates to the agent. See the `ai-video-tools` skill. The agent analyzes the transcript, proposes a list of cuts, writes them to `editor-draft` for review, and the user one-click approves. ## Export ```ts // actions/export-video.ts export default defineAction({ schema: z.object({ id: z.string(), format: z.enum(["mp4", "webm"]).default("mp4") }), run: async ({ id, format }) => { await assertAccess("recording", id, "editor"); const rec = await getRecordingOrThrow(id); const edits = JSON.parse(rec.editsJson || "{}"); // Short clips (< 2 min) render client-side via ffmpeg.wasm. // Longer ones are queued for server-side ffmpeg. return { exportId: enqueueExport({ recordingId: id, edits, format }) }; }, }); ``` For **short recordings**, load ffmpeg.wasm in a web worker, feed it the source blob + the edit list, and return the rendered file. For **long recordings**, enqueue a background job and have the UI poll `export-status` every 2s. ## Player integration The player is the canonical renderer for edits. Given `edits_json`, compute a `VirtualTimeline`: ```ts interface VirtualTimeline { // Maps virtual ms -> source ms, skipping cuts/trims. toSource(virtualMs: number): number; // Total virtual duration after edits applied. durationMs: number; // Ranges to skip on playback: the player listens for timeupdate and seeks past them. excludedRanges: { startMs: number; endMs: number }[]; } ``` Never call `video.currentTime` with a raw segment index — always go through `toSource`. ## Rules - The edit UI writes to `editor-draft` on every change, and only writes to `edits_json` on Save (so Cmd+Z is cheap and the DB stays clean). - Never mutate `edits_json` from `db-exec`. Use `apply-edit` or `reset-edits`. - Speed edits compound (`1.5 × 2 = 3x`) — validate the result is in `[0.25, 4]`. - Blur coordinates are in **source resolution**, not display pixels. Always normalize against `recordings.width` / `recordings.height`. ## Related skills - `ai-video-tools` — filler-word removal and chapter generation propose edits via the agent. - `real-time-sync` — when the agent writes `edits_json`, the player must reflect it; `edits_json` is part of the `recordings` row which is already on the sync list. - `video-sharing` — exports honor the share's `enableDownloads` flag. - `storing-data` — why edits live in a JSON column rather than a separate table. --- ## Editor implementation reference ### Concrete `editsJson` shape (editor team) The editor writes and reads this exact shape. Any new edit operation MUST preserve this shape so the player, editor, and export pipeline stay in sync. ```ts interface EditsJson { version: 1; // Ripple-style trim ranges. `excluded:true` ranges are skipped during // playback and collapsed during export. `excluded:false` entries where // `startMs === endMs` are SPLIT MARKERS used by the editor UI — they // never affect playback. trims: Array<{ startMs: number; endMs: number; excluded: boolean }>; blurs: Array<{ id: string; startMs: number; endMs: number; x: number; y: number; w: number; h: number; // normalized 0..1 intensity: number; }>; thumbnail?: | { kind: "url"; value: string /* absolute URL */ } | { kind: "frame"; value: string /* timeMs as string */ } | { kind: "gif"; value: string /* JSON: { url, startMs, durationMs } */ } | null; // Provenance for stitched recordings — set by `stitch-recordings`. stitchedFrom?: string[]; } ``` Comment `videoTimestampMs`, reaction `videoTimestampMs`, and `recording_transcripts.segmentsJson` timestamps all refer to **original** video time. The player converts to edited time; never assume the two are equal. ### Timestamp helpers `app/lib/timestamp-mapping.ts` is the single source of truth: | Helper | Returns | | -------------------------------------- | ----------------------------------------------------------------------- | | `parseEdits(raw)` | A fully-populated `EditsJson` from the DB column (empty on bad input) | | `serializeEdits(edits)` | Stringified JSON ready to write back | | `getExcludedRanges(edits)` | Sorted, non-overlapping excluded ranges | | `getKeptRanges(durationMs, edits)` | Ordered "kept" ranges in original time — the export pipeline iterates | | `originalToEdited(ms, edits)` | Maps original → edited (playback) timeline | | `editedToOriginal(ms, edits)` | Maps edited → original (used when seeking the underlying `