Rotoscope
Produce a measured 1:1 replica of a reference clip whose pixels come from traced paths and brush
parameters only. Never replay or embed source bitmaps. Read
reference/method.md first: the defaults, why they hold, and the
diagnostics in the order to read them.
Run every script as python3 ${CLAUDE_PLUGIN_ROOT}/scripts/pydeps.py run --data-dir "${CLAUDE_PLUGIN_DATA}" -- <script> ...; it never installs a package, and a missing tool or
package stops it with exit 2 and one remedy line (/animation:setup checks them all). Below, $R
stands for ${CLAUDE_PLUGIN_ROOT}/skills/rotoscope/scripts/.
Work directory
One work directory per reference clip, outside the plugin; it grows to hundreds of MB.
| Path |
Written by |
Holds |
src/dNNN.png, d/index.json |
extract.py --video |
distinct drawings; size, duration, [k, pts, t1] |
d/dNNN.json |
extract.py |
T-layer paths with holes, tone layers, colors, applied overrides |
overrides.json |
you, fit.py |
per-drawing parameters (below); survives re-extraction |
out/<tag>/ |
measure.py, review.py |
replicas, heatmaps, table-*.md, review-*.md, crops/ |
learnings.md |
measure.py, you |
one line per run, plus your findings |
Override file
Tool-neutral JSON that carries every hand decision. Entries apply in file order to each drawing
whose k (an index or [k0, k1]) holds it; later entries win, and brush merges key by key.
{"overrides": [
{"k": [0, 30], "brush": {"bias": 0.18, "blur": 0.7}, "why": "sparse line art, replica too thin"},
{"k": [203, 217], "sharp": [2.0, 0.6], "levels": [40, 80, 170, 215]},
{"k": [64, 74], "tint": true}
]}
brush keys: bias, blur, grain, tones (see roto.js). sharp is the unsharp mask
[amount, sigma] or null; levels are the tone gray levels; both retrace. tint adds
warm-paper layers. why is for the reader; the tools ignore it. Geometry never goes in this file:
a fix that needs hand-edited paths is a finding to report, not an override.
Loop
- Extract:
$R/extract.py <work> --video <clip>. After an override change,
$R/extract.py <work> --apply retraces only drawings whose sharp or levels changed.
- Measure:
$R/measure.py <work> [--only K0-K1] [--tag name] --playwright-core '${user_config.playwright_core}'. It exits 0 only when every drawing meets the measure.py
target (reference/method.md, Target). Run long ranges in the background.
- Fit:
$R/fit.py <work> [--only K0-K1] --playwright-core '${user_config.playwright_core}'
bisects bias per drawing from the replica-only / source-only balance and appends the result to
overrides.json; apply it with --apply. It fits bias only: blur, sharpening and tone levels
are yours, guided by the diagnostics in method.md. On shfred0, fitting every drawing from the
default bias passes them all at a mean XOR 0.3% below the hand fits.
- Review:
$R/review.py <work> <tag> adds the manual checks as columns (largest one-color
cluster and whether it is a blob, frame-edge XOR, tone-tile difference, paper color) and writes
one 1:1 crop per drawing of its hottest window. Read every crop, not only the flagged ones: a
passing number can hide a visible defect such as a tint. Never downscale a crop or review a GIF.
- Record: append to
<work>/learnings.md what failed, the cause, the change and the before and
after numbers.
Repeat 2-5 by shot or drawing range until the table passes and the crops show no one-color
cluster.
A film of the replica: ${CLAUDE_PLUGIN_ROOT}/scripts/render.py $R/roto.js <frames dir> --fps <the source's frame rate> --root <work> --encode mp4 --playwright-core '${user_config.playwright_core}'
writes the frames, render.json and <frames dir>.mp4 beside them.
Retro
At the end of a clip, read learnings.md next to earlier runs' learnings. A finding that recurs
across shots or clips with the same fix (the same override on most drawings, the same diagnostic
leading to the same change) becomes a default: change the constant in extract.py or
roto.js, re-run the regression, and record the finding with its numbers in method.md. A one-off
stays in the clip's override file.
Regression
The shfred0 study ships as a fixture (fixtures/shfred0.overrides.json, parameters only; the clip
is not shipped). Given that clip and an empty work directory, $R/regress.py <shfred0.mp4> <work>
extracts with the fixture, renders and measures every drawing, checks the encoded replica against
the woodcut-ink pack, and exits 0 only when every drawing and the replica pass.
$R/regress.py --synthetic <work> needs no clip: it renders, encodes and re-traces
fixtures/synthetic.js and checks the render, encode and decode contracts. Run both after any
change to the scripts.
Next
/animation:learn-style <work dir> <pack dir>
It measures this work directory's traces and drawings into a style pack.
Gotchas
renderFrame(t) and renderDrawing(k) return Promises; a capture that does not await them
saves the canvas before the drawing lands.
- Chromium's canvas
blur() filter does nothing below a floor (reference/method.md, Rendering,
holds the dated record), so roto.js blurs in JS; gauss() must not add 0.5 before writing to a
Uint8ClampedArray, which already rounds.
- A drawing JSON is one line of 1-4 MB. Change parameters through the override file and
--apply,
never by editing the JSON.
- Gray XOR and SSIM cannot see color. Read
paper_err and the crops for tints.
- An all-ink or all-paper frame has a floor near 0.02%; a few hundred border pixels decide it, so
judge it by the ratio and the edge column, not by the XOR alone.
1---2name: rotoscope3description: "Drawing-by-drawing copy of a reference animation clip: decode it into distinct drawings, trace each to vector paths, render them through the ink.js brush engine, measure each against its source, and fit brush overrides until every drawing passes. Use when: 'rotoscope this', 'trace this clip', 'copy this animation one to one', 'replicate this video exactly', 'match the reference frame by frame', 'measure the replica against the source'. Not for an original film or a style study without a reference clip."4argument-hint: "<reference video> <work dir>"5user-invocable: true6disable-model-invocation: false7metadata:8 source: https://skillmd.com/skills/melodic-software/rotoscope9 workflow-stage: implement10 summary: Trace, render, measure and fit a 1:1 replica of a reference animation11---1213# Rotoscope1415Produce a measured 1:1 replica of a reference clip whose pixels come from traced paths and brush16parameters only. Never replay or embed source bitmaps. Read17[`reference/method.md`](reference/method.md) first: the defaults, why they hold, and the18diagnostics in the order to read them.1920Run every script as `python3 ${CLAUDE_PLUGIN_ROOT}/scripts/pydeps.py run --data-dir21"${CLAUDE_PLUGIN_DATA}" -- <script> ...`; it never installs a package, and a missing tool or22package stops it with exit 2 and one remedy line (`/animation:setup` checks them all). Below, `$R`23stands for `${CLAUDE_PLUGIN_ROOT}/skills/rotoscope/scripts/`.2425## Work directory2627One work directory per reference clip, outside the plugin; it grows to hundreds of MB.2829| Path | Written by | Holds |30|---|---|---|31| `src/dNNN.png`, `d/index.json` | `extract.py --video` | distinct drawings; size, duration, `[k, pts, t1]` |32| `d/dNNN.json` | `extract.py` | T-layer paths with holes, tone layers, colors, applied overrides |33| `overrides.json` | you, `fit.py` | per-drawing parameters (below); survives re-extraction |34| `out/<tag>/` | `measure.py`, `review.py` | replicas, heatmaps, `table-*.md`, `review-*.md`, `crops/` |35| `learnings.md` | `measure.py`, you | one line per run, plus your findings |3637## Override file3839Tool-neutral JSON that carries every hand decision. Entries apply in file order to each drawing40whose `k` (an index or `[k0, k1]`) holds it; later entries win, and `brush` merges key by key.4142```json43{"overrides": [44 {"k": [0, 30], "brush": {"bias": 0.18, "blur": 0.7}, "why": "sparse line art, replica too thin"},45 {"k": [203, 217], "sharp": [2.0, 0.6], "levels": [40, 80, 170, 215]},46 {"k": [64, 74], "tint": true}47]}48```4950`brush` keys: `bias`, `blur`, `grain`, `tones` (see `roto.js`). `sharp` is the unsharp mask51`[amount, sigma]` or `null`; `levels` are the tone gray levels; both retrace. `tint` adds52warm-paper layers. `why` is for the reader; the tools ignore it. Geometry never goes in this file:53a fix that needs hand-edited paths is a finding to report, not an override.5455## Loop56571. Extract: `$R/extract.py <work> --video <clip>`. After an override change,58 `$R/extract.py <work> --apply` retraces only drawings whose `sharp` or `levels` changed.592. Measure: `$R/measure.py <work> [--only K0-K1] [--tag name] --playwright-core60 '${user_config.playwright_core}'`. It exits 0 only when every drawing meets the `measure.py`61 target (`reference/method.md`, Target). Run long ranges in the background.623. Fit: `$R/fit.py <work> [--only K0-K1] --playwright-core '${user_config.playwright_core}'`63 bisects `bias` per drawing from the replica-only / source-only balance and appends the result to64 `overrides.json`; apply it with `--apply`. It fits bias only: blur, sharpening and tone levels65 are yours, guided by the diagnostics in `method.md`. On shfred0, fitting every drawing from the66 default bias passes them all at a mean XOR 0.3% below the hand fits.674. Review: `$R/review.py <work> <tag>` adds the manual checks as columns (largest one-color68 cluster and whether it is a blob, frame-edge XOR, tone-tile difference, paper color) and writes69 one 1:1 crop per drawing of its hottest window. Read every crop, not only the flagged ones: a70 passing number can hide a visible defect such as a tint. Never downscale a crop or review a GIF.715. Record: append to `<work>/learnings.md` what failed, the cause, the change and the before and72 after numbers.7374Repeat 2-5 by shot or drawing range until the table passes and the crops show no one-color75cluster.7677A film of the replica: `${CLAUDE_PLUGIN_ROOT}/scripts/render.py $R/roto.js <frames dir> --fps <the78source's frame rate> --root <work> --encode mp4 --playwright-core '${user_config.playwright_core}'`79writes the frames, `render.json` and `<frames dir>.mp4` beside them.8081## Retro8283At the end of a clip, read `learnings.md` next to earlier runs' learnings. A finding that recurs84across shots or clips with the same fix (the same override on most drawings, the same diagnostic85leading to the same change) becomes a default: change the constant in `extract.py` or86`roto.js`, re-run the regression, and record the finding with its numbers in `method.md`. A one-off87stays in the clip's override file.8889## Regression9091The shfred0 study ships as a fixture (`fixtures/shfred0.overrides.json`, parameters only; the clip92is not shipped). Given that clip and an empty work directory, `$R/regress.py <shfred0.mp4> <work>`93extracts with the fixture, renders and measures every drawing, checks the encoded replica against94the woodcut-ink pack, and exits 0 only when every drawing and the replica pass.95`$R/regress.py --synthetic <work>` needs no clip: it renders, encodes and re-traces96`fixtures/synthetic.js` and checks the render, encode and decode contracts. Run both after any97change to the scripts.9899## Next100101`/animation:learn-style <work dir> <pack dir>`102103It measures this work directory's traces and drawings into a style pack.104105## Gotchas106107- `renderFrame(t)` and `renderDrawing(k)` return Promises; a capture that does not await them108 saves the canvas before the drawing lands.109- Chromium's canvas `blur()` filter does nothing below a floor (`reference/method.md`, Rendering,110 holds the dated record), so `roto.js` blurs in JS; `gauss()` must not add 0.5 before writing to a111 `Uint8ClampedArray`, which already rounds.112- A drawing JSON is one line of 1-4 MB. Change parameters through the override file and `--apply`,113 never by editing the JSON.114- Gray XOR and SSIM cannot see color. Read `paper_err` and the crops for tints.115- An all-ink or all-paper frame has a floor near 0.02%; a few hundred border pixels decide it, so116 judge it by the ratio and the edge column, not by the XOR alone.