Roto

Reference-plate adaptation

Some motion design is not worth re-deriving. When a reference video already has the exact gradient ramp, camera drift, cursor path, and easing the user wants, re-animating it from scratch produces a worse approximation at ten times the cost. This workflow keeps the reference's pixels as the moving background and replaces only the lettering.

The trade: you inherit the reference's motion exactly, and you give up the ability to change it. Timing, camera, and layout are frozen at the reference's frame grid. Only what you clear can be re-authored. If the user wants different pacing or a different layout, this is the wrong workflow - route to /general-video.

When this workflow applies

Phase 0 - Rights gate (blocking)

Run this before decoding a single frame. The output of this workflow is derived from someone else's artwork, and the derivation is visible in every frame.

Establish, and record the answers in THIRD_PARTY.md:

1. Who made the reference, and how the user obtained it. A public tweet is not a license.
2. What permission exists. The user owns it, commissioned it, licensed it, or has the author's explicit go-ahead. "It's for internal use" and "it's just a study" are not permission to publish.
3. The soundtrack, separately. Audio is almost never covered by permission to adapt the visuals: the original designer licensed that track for their piece, not for yours. This is a decision to put to the user, not one to make silently in either direction - a silent render is as much a surprise as an unlicensed one. See § Choosing the soundtrack below.
4. Fonts. Reference fonts are not redistributable. See Phase 5's font policy.
5. Trademarks in the reference. Remove the original brand identity; do not silently transplant it.

If any answer is missing, say so plainly and stop. Offer the alternative: build an original composition in the same style through /general-video, which needs no rights to the reference.

If the user confirms permission, proceed and write THIRD_PARTY.md from the template in references/attribution.md. Credit the original designer in the README even when permission is explicit - the motion is theirs.

Project layout

The scripts are project-local, so a published adaptation carries the exact code that built it. Copy them in once:

mkdir -p <project>/{scripts,assets,docs,reference}
cp <skill>/scripts/*        <project>/scripts/
cp <skill>/templates/regions.example.json <project>/regions.json
cp <skill>/templates/adapt.example.json   <project>/adapt.json
cp <skill>/templates/index.template.html  <project>/index.html
cp <skill>/templates/film.template.js     <project>/film.js

Then the project looks like this, and every command below runs from its root:

reference/original.mp4     the supplied reference, never a render input
regions.json               Phase 2 + 3 config: clear regions and trackers
adapt.json                 Phase 5 config: dimensions, fps, audio, output
assets/plates/*.png        text-cleared frames, one per authored frame
assets/tracks.js           measured per-frame positions
soundtrack.json            which track, under which licence, with what credit
index.html  film.js        the composition
assets-manifest.json       pinned asset hashes
docs/                      frame-analysis.md, contact-sheet.jpg, analysis.json

The Python scripts declare their own dependencies inline, so uv run needs no environment setup.

Choosing the soundtrack

Ask the user, once, before Phase 5. Never default silently.

node scripts/soundtrack.mjs list

That prints two things. First a catalogue of sources - Musopen, Incompetech, Pixabay, Free Music Archive, ccMixter, Freesound, the Internet Archive, Uppbeat, Openverse - with each one's licence, whether attribution is required, and whether the licence is set per track. Then a shortlist of ready-to-fetch tracks, each already verified and hash-pinned:

node scripts/soundtrack.mjs fetch odyssey

downloads it, checks it against the pinned SHA-256, transcodes to AAC if needed, trims it to the composition duration, wires the clip, and writes the credit. Present the realistic options and let the user pick:

For anything not on that shortlist, wire it with one command, which copies the file into assets/, probes its true duration, sets the clip in index.html and audioDuration in adapt.json, and writes the attribution record:

node scripts/soundtrack.mjs use <source> --file <path> --title "..." --artist "..." --url "..."
node scripts/soundtrack.mjs use silent
node scripts/soundtrack.mjs status

It refuses a per-track-licensed source without the track URL you read the licence on, and refuses an attribution-required source without the credit details - attribution you cannot reconstruct later is attribution you will not give. Paste the printed line into THIRD_PARTY.md.

Three things bite if you wire audio by hand instead:

• Codec. FFmpeg will mux Vorbis or Opus into an MP4 without complaint, and the result plays in neither Safari nor QuickTime. -c:a copy succeeding is not evidence the audio is usable. Anything that is not already AAC or MP3 should be transcoded once, into the asset.
• Length. A five-minute track under a fifteen-second piece leaves the container running long after the last video frame. Trim the asset rather than passing -shortest at encode time, so the asset and the output's audio stay identical and the packet-hash check still means something.
• Ownership. The audio must be a declared clip the framework plays, never something film.js calls play() on; and the clip duration must be the real stream duration, which routinely outruns the visual loop by a fraction of a second.

Phase 1 - Analyze the reference

uv run scripts/analyze_reference.py reference/original.mp4 --out docs/

This decodes every frame, reports dimensions and fps, searches for a loop point by mean absolute RGB difference, and emits:

• docs/frame-analysis.md - a scene-band table: frame range, timecode, and what the reference does in that range. Fill in the behavior column yourself after looking at the contact sheet; the script proposes cut points from inter-frame difference spikes, it does not understand the content.
• docs/contact-sheet.jpg - a grid of sampled frames.
• docs/analysis.json - machine-readable bands, loop length, and per-frame difference series.

Loops are the big win. Many social-format references are one visual loop repeated. If the analyzer reports a loop, you build and clear only the first cycle and modulo the frame index at draw time - half the plates, half the tracking, identical output.

The scene bands you settle on here become the branches of your draw function in Phase 4 and the region sets in Phase 2. Get them right before moving on. Read references/motion-tracking.md § Reading a band table for what makes a good band boundary.

Phase 2 - Clear the text into plates

A plate is one reference frame with the original lettering removed and everything else intact. Plates are PNG, lossless, one per frame of the loop.

Author regions.json: for each scene band, the rectangles to clear and the strategy for each. Start from templates/regions.example.json. Then:

uv run scripts/build_plates.py --config regions.json --out assets/plates/

Strategy choice is the whole craft of this phase. The four that cover almost everything:

Full decision procedure, parameters, and the hard cases are in references/plate-clearing.md.

The builder also reports a coverage audit: bands that declare no clear regions, and frames that match no band at all. Read it. A band nobody cleared keeps the reference's own typography, and since the draw loop has no branch for it either, that lettering renders into your output looking like finished design that simply is not yours. The audit cannot know whether a band legitimately has no text, so confirm each one against the contact sheet rather than dismissing it.

Verify every plate before trusting it. Run the builder with --contact-sheet and look. A seam you cannot see at 100% will strobe at 24 fps because it moves with the camera. Common tells: a horizontal band edge where linear-blend met a curved gradient, and ghosting where the cleared rectangle was too tight and left antialiased text pixels at the border. Pad clear rectangles by several pixels beyond the visible glyph bounds.

Phase 3 - Measure the motion

The plates move. Your text has to move with them, and eyeballing per-frame positions does not survive a camera drift. Measure them from the source frames instead.

Trackers are declared per band in the same regions.json and emitted by the same builder into assets/tracks.js as window.referenceTracks - one object per frame, keyed by frame index.

Scan on a row or column that stays inside the element for the whole band, and away from the text you are about to clear. Measure on the original frame, never the cleared plate - clearing destroys the thing you are tracking. references/motion-tracking.md has the scanline selection procedure and how to sanity-check a track for jitter.

Phase 4 - Compose

One canvas, sized to the reference, sitting in a HyperFrames composition. Read /hyperframes-core for the data-* timing contract before writing the HTML; templates/index.template.html is a minimal conforming shell.

templates/film.template.js is the draw loop. Its shape:

async function drawFrame(time) {
  await ready;                                   // fonts resolved before first paint
  const f = Math.round(time * FPS) % LOOP;       // modulo the loop
  const track = window.referenceTracks[f];
  ctx.drawImage(await plate(f), 0, 0);           // plate first, always
  if (f < 22)        { /* band 1 typography */ }
  else if (f < 72)   { /* band 2 */ }
  // ...
}
window.addEventListener('hf-seek', e => e.detail.waitUntil?.(drawFrame(e.detail.time)));

Four rules that make it render correctly rather than merely look right in preview:

• Every frame is drawn from scratch. No state carried between frames, no accumulation. The renderer seeks; it does not play. A frame must be reproducible from its index alone.
• waitUntil is mandatory. Plate loading and font loading are async. Without handing the promise back to the seek event, the renderer captures a blank or stale canvas.
• Cache plates, but bounded. An LRU of about four entries. Holding 180 decoded 1920x1080 bitmaps exhausts memory mid-render; holding none re-decodes every frame.
• Composite text off-screen when it needs its own mask. Gradient-masked reveals and wipes need destination-in on a scratch canvas, not on the main context, or you erase the plate.

references/composition-pattern.md covers the cache, the off-screen mask pattern, gradient text fills, per-character typing reveals, and blur-in transitions.

Phase 5 - Render deterministically

Two renders of the same composition must produce the same file. That is what makes the adaptation checkable by someone else, and it is what catches an accidental edit.

node scripts/hashes.mjs make          # after assets are final
node scripts/render.mjs               # check -> lint -> render -> encode -> verify

render.mjs runs the full chain: verify asset hashes, lint the composition, render a PNG sequence single-worker with GPU off, encode H.264 with an explicit BT.709 conversion, copy the audio packets rather than re-encoding, then probe the result for frame count, dimensions, rate, full decode, and an unchanged audio hash. See /hyperframes-cli for the render flags themselves.

Font policy. Never commit a font you did not license for redistribution. Pin its SHA-256 in the manifest and download it at setup time behind an explicit --accept-font-license flag, the way references/deterministic-render.md § Fonts describes. A font hash mismatch must fail the build, not silently substitute - a substituted font changes every glyph in the render.

What determinism does and does not promise. Pinned dependencies and hashed assets make the composition reproducible. Browser version, OS font rasterization, and FFmpeg build still move individual pixels across machines. Promise a reproducible process and ship the verification report; do not promise a byte-identical MP4 on every platform.

Guardrails

• Do not use the reference video as render input. It is the source for plates and the artifact you compare against. If it ends up in the render path, you have transcoded someone's video and called it your own.
• Do not regenerate plates casually once the manifest is written. Different OpenCV or libpng versions change PNG bytes. The committed plates are the baseline; hash mismatch is the detector working, not a bug to paper over.
• Do not claim procedural recreation. The output preserves source pixels. Say so in the README. Changed typography also means the frame can never be pixel-identical to the reference - do not set that as the success bar.
• Account for every frame. Before rendering, check each band either clears something or demonstrably has no source lettering. The failure is silent in both directions: nothing errors, and the frames look designed.
• Stop after two failed clears of the same region. If a region will not clear cleanly, the honest options are to move the new text elsewhere, cover the region with an opaque designed element, or tell the user this reference is not adaptable. Do not keep tuning thresholds.

Delegation

Worked example

references/case-study-astra.md walks the ChatGPT adaptation this workflow was extracted from: 180 plates from a 7.5-second loop, four clearing strategies, four trackers, five draw bands, and a byte-identical re-render on the recorded environment.