Taste Application

Taste Application

The second half of the pipeline. taste-distillation measures references into a style pack; this generates against that pack and cuts the result.

Execution and Delivery Contract

The original implementation ships here in scripts/; no separate ito-video checkout is required. Install scripts/requirements.txt for local processing. Install scripts/requirements-live.txt only for provider execution. Live uploads, submissions and downloads require explicit TASTE_FORGE_ALLOW_LIVE=1 in addition to credentials; set it only for the user's authorized run. --dry-run remains credential-free and produces labelled placeholders. An ambiguous provider timeout is not retried as a new paid job. Inspect the provider request before deciding whether another submission is warranted.

Use existing completed takes without any provider calls:

python scripts/pipeline.py --genre example --root stylepacks \
  --takes media/take-a.mp4 media/take-b.mp4 --duration 12 --fps 30 \
  --out out/review-v1.mp4

--duration is a best-effort cadence target, not an exact runtime. Complete shots may produce a shorter or longer edit; the assembler does not duplicate clips or add padding to meet the target. The output manifest retains actual duration and adds duration_contract with requested and actual seconds, shortfall, overrun, and the cadence_target policy. Differences of at least one output frame are warned explicitly. No exact-duration mode is provided; when an exact runtime is required, inspect the receipt and revise or reject the cut before delivery.

The pipeline keeps each run's graded shot files because the editable FCPXML and EDL reference them. It refuses output collisions and reports timeline export failures. A completed render is not a saved editor project or creative approval. Preserve source assets and versioned project checkpoints before and after live edits; record the saved path and digest separately from the in-memory timeline receipt.

Clone-ready Fal graphs and their offline input compiler are documented in workflows/README.md. Compile brief and style direction before submission; do not assume a disconnected schema field affects a model prompt. Validate current endpoint fields against the actual provider before a paid run.

For Blender, use the full textured source GLB; retopology is a separately named derivative and never replaces that source. blender_prop.py supports explicit --width 1920 --height 1080 --fps 30 --receipt receipt.json and preserves pack-derived rim lighting and material textures. Saving a scene is distinct from rendering it; inspect the receipt's state and packed images.

For Resolve overlays, use taste.resolve.apply_placements with injected objects from an explicitly selected, versioned target. Set source_end_mode to the convention verified on that host. Do not assume an inclusive source end across Resolve versions. The adapter allocates overlapping effects above preserved tracks and checks every placement immediately and again after all appends. Composite integers must match the installed API; for the verified Studio 21 host, Screen is 5, not the historical builder's incorrect 22. The receipt proves in-memory placement only. Save the project, verify its checkpoint, then inspect the exact rendered output before reporting delivery.

When to Activate

• "make a video in this style" / "apply the pack" / "supplement this footage"
• Assembling generated clips into something with real edit rhythm
• Minting 3D props from a look and getting them back into the video
• Verifying that a finished piece actually matches its reference

Division of Labour

The model supplies content, motion, framing and lighting structure. The pack supplies colour and rhythm. This is measured, not stylistic preference — see taste-distillation for the numbers. Practical consequences:

• The generation prompt contains zero colour language. Add "Colour: none. Render neutral. Grading is applied afterwards."
• Keep two separate flags: --brief (what HAPPENS: subject, action, place) and --style-steer (how it LOOKS). Merging them leaks style words into the scene ("teal" becomes a teal object) and subject words into the grade.
• The model responds to local, checkable rules far better than global ones. "Backgrounds pure black and unlit; subjects blowing toward white" works; "extreme contrast" does not.

Generate TAKES, Not Shots

The obvious reading of "match the cadence" is one generation per shot. It is economically absurd. A reference averaging 0.78s/shot against an endpoint with a 4-second floor turns a 10s piece into 12 calls, 48 generated seconds for 10 used (21% efficiency), and twelve unrelated clips stitched into what should read as continuous.

Editors roll a longer take and cut inside it. Grouping shots into ~5s takes: 3 calls, 13 generated seconds, 77% efficiency, and consecutive shots that actually belong to each other because they came from the same generation.

Tell the model what shape you want, or it renders a slow locked-off push and six cuts inside it read as a stutter:

"Filmed as ONE continuous take with no hard cuts inside it. It will be cut into 6 pieces of roughly 0.8s in the edit, so the framing, subject and light must keep changing throughout — any 0.8s window has to stand alone as its own shot."

Cutting Rules

• Re-encode, never stream-copy. Stream copy only cuts on keyframes, which at 0.78s mean shot length rounds every boundary to the nearest GOP — destroying the exact thing the pipeline exists to preserve.
• When supplementing existing footage, use its own shot boundaries. Slicing a base video into contiguous pieces and playing them in order just reassembles the original: every "cut" lands mid-shot and is invisible. Measured, a 20-shot assembly registered only 12 detected cuts. Detect real boundaries and take every Nth so consecutive picks are guaranteed discontinuous. That moved a cut measurement from 1.29s to 0.83s against a 0.78s target.
• Sample shot lengths from the reference's distribution, not from its mean, so the cut inherits rhythm variance instead of flattening to even clips.

Grading Rules

• Direct measurement beats a baked LUT when you have the clip: measure it, match its L* CDF, apply zone chroma. grade_clip_direct reached MAE 1.58 and contrast 34.0 against 34.7.
• Anchor, do not CDF-match, when the clip's histogram is unlike the reference's. Forcing a 68%-black generated clip onto a busy reference histogram lifted the entire background out of black: background preservation fell to 26.0% (CDF) versus 69.0% (anchor), while MAE and contrast both still looked excellent. Default to anchored tone.
• Batch size matters. Grading 48 frames at once OOM-killed the process; 6 is safe.

Recovered fal Platform Behavior

These observations and endpoint examples came from the recovered workflow. Recheck current endpoint metadata; they are not guarantees about every future provider version. The bundled graph templates record the separately verified workflow inputs, including explicit generated-audio control.

These cost real time to discover. Check them before designing a graph.

Endpoint picks

Seedance is ~4x Kling o3's price for the same 5 seconds. It earns that on multi-reference fidelity (up to 50 mixed image/video/audio refs) and does not earn it when conditioning on a single still.

Model IDs and prices drift. Verify against fal.ai/models before promising any of them.

The 3D Branch

python mint3d.py --genre <name> --from-stills 4 --retopo --render
python mint3d.py --genre <name> --prompt "a cracked chrome visor" --render

• Generate a clean plate first; do not lift from reference stills. The endpoint's stated input requirement is simple background, single object, object >50% of frame. Reference reels are the opposite of that — collages, wide shots, several subjects, burnt-in graphics — and they produce sculpted noise. Text → single-object plate → mesh costs ~$0.15 extra and is the difference between a usable mesh and a discarded one.
• Multi-view is named per-angle fields, not a list. input_image_url (front, required), then back_image_url, left_image_url, right_image_url, left_front_image_url, right_front_image_url, top_image_url, bottom_image_url. There is no input_image_urls and no multi_view flag — inventing them degrades every mint to single-view while appearing to work. A wrong angle label is worse than omitting the view, because the model trusts it.
• Address the GLB by key, not by position. The response carries a thumbnail PNG and a model_urls block alongside model_glb; taking the first URL works only until the keys reorder, and a preview PNG downloads fine — nothing fails until Blender refuses to open it.
• Request PBR maps. Without them the mesh lights like painted cardboard.
• Retopologise if anyone will edit or rig it. Generated meshes are dense and chaotic.
• Render locally to close the loop. Once a turntable is frames, it is footage, and every downstream stage already handles footage — grade it, cut it, screen it as an element, or upload it as a conditioning reference. Use Blender when a binary is on PATH; keep a dependency-light software rasteriser as the default, because a headless GL context is the single most common thing missing from a container and a renderer that only works on a workstation is not part of a pipeline.

Verify, Then Believe

Every other stage claims a result. Check it, and check distribution shape, not just moments:

• background — share of frame below L*10 vs the pack's figure. Compare to the reference, not to the source clip: a generated source at 68% black is blacker than any reference in a 24–55% band, so "preserve the source's blacks" demands the wrong thing and equally excuses a lifted grade.
• chroma_mae — per-zone a*/b* error, using the median (matching how the pack's targets were measured; a mean here compares a skew-sensitive statistic to a robust one and reports a definition mismatch as an error).
• contrast / black_point / white_point
• banding — empty L* histogram bins between occupied ones. Counting total empty bins does not work: a legitimately dark clip has empty highlight bins.
• cadence — detected mean shot length vs the reference's.

Watch the units trap: OpenCV changes Lab convention with dtype. On float32, L* is 0–100 and a*/b* are signed; on uint8, L* is 0–255 and a*/b* are biased +128. Mixing them reports chroma errors in the hundreds.

Full Chain

python pipeline.py --genre <name> --refs a.mov b.mov \
  --brief "what happens" --duration 12 \
  --base-video existing.mp4 --out out/FINAL.mp4

mint → distill → (mint3d) → apply → forge → verify. Stages 1–3 are cached, so iterating on briefs never re-measures anything. --dry-run stubs every network call: the plan, prompts, track layout and manifest all still get exercised.

Anti-Patterns

Borrowed Footage Carries the Capture App's UI

The single worst defect found in a delivered cut: the finished video shipped someone else's like button, view counter and comment bubble, because the base footage was a screen recording and nothing cropped them out.

content_mask does not solve this and its bounding box makes it worse. Temporal variance keeps interface chrome, because chrome animates - the heart pulses, the counter ticks - so the mask marks it as moving content. Measured on three references, the mask bbox kept 100% of the width every time while the interface sat plainly in the right-hand margin.

The separating signal is the temporal median, not the variance. Footage moves, so the median of many frames averages into mush with almost no edge energy; chrome sits at fixed coordinates, so its edges survive intact. Sobel energy on the median frame lights up on chrome and goes quiet on content - measured, a right-hand column read 0.23 against an interior background of 0.03 on one reference and 0.31 against 0.15 on another.

Two implementation details that cost a cycle each:

• Trim past the innermost outlier in each outer band, not inward from the edge. Walking in while the current line is hot stops immediately, because the outermost lines are letterbox - flat black, zero edge energy - and the chrome sits inside that at 90-95% of width. The naive version trimmed 1% of frame while the like button stayed in shot.
• Scale to cover, not pad, when portrait source lands in a landscape cut. Padding 9:16 (narrower still after the UI crop) into 16:9 left ~60% of frame as black bars, one shot was nearly an empty rectangle, and it poisoned the background metric because bars are pure black. Covering loses the sides, which is the right trade for centre-framed material.

Overlay Plates Are Elements, Not Washes

A glow plate is ~4% covered by construction. Composite it at frame size and you get a small bright dot parked mid-shot - it reads as a sticker, and it was visible in a delivered cut as an unexplained coloured blob.

• Tighten each plate to its alpha bounding box first. That raises coverage from ~4% to 15-35% and hands size control to the caller instead of inheriting whatever fraction of the source frame the element happened to occupy.
• Choose wash vs element by coverage. Diffuse plates (<10% after tightening) work stretched full-frame at low opacity; concentrated ones want to be scaled to 35-70% of frame width and placed.
• Vary placement, scale and rotation per shot from a seeded RNG, so the cut stays reproducible but no two stamped shots share a mark. One plate in one spot every Nth shot reads as a watermark.
• Resolve element geometry in Python, not in ffmpeg expressions: pad() rejects a negative offset and cannot pad below its input size, so an oversized or off-frame element kills the whole filtergraph.

Downstream Handoff (Resolve / Blender)

Always ship an editable timeline beside the mp4. The flattened video is a viewing copy and the one thing a colourist cannot work with — every cut is baked in and the shots are no longer separable. forge.py writes FCPXML 1.9 and a CMX3600 EDL referencing the individual graded shot files, so the piece lands as a timeline that can be re-cut and re-graded.

Validate the export, do not assume it: check that asset-clip offsets equal the running sum of prior durations (no gaps), that every ref resolves to a declared asset, that every media-rep src exists on disk, and that the total matches the mp4. A timeline that imports but drifts is worse than one that fails loudly.

Two things that silently destroy the work:

• Project frame rate must be set before import. Resolve locks timeline fps on first timeline creation and conforms the cadence silently. At a sub-second mean shot length that conform is visible.
• The delivered shots are already graded. look.cube is a normalising LUT for new material and for matching — applying it to the supplied shots double-grades them. Node 1, nothing before it, corrections after.

Ship a handoff doc with the measured targets in it (scripts/HANDOFF-TEMPLATE.md is a filled example): the zone chroma table, the cadence distribution including rhythm variance — an editor who matches the mean but not the variance produces something that reads completely differently — and an explicit "do not" list.

For Blender, scripts/blender_prop.py derives its lighting from the pack: black world with transparent film (so renders composite with no keying), key plus rim (a key alone lets the silhouette die against black), rim colour converted from the pack's peak-chroma zone via Lab→linear sRGB so the prop picks up the same cast the footage is graded to. View transform Standard, not AgX/Filmic, and no grading in Blender — AgX applies its own tone curve before the LUT ever sees the pixels, and grading twice compounds.

Bundled Code

scripts/ in this skill is a working implementation, not pseudocode. It has no project-specific assumptions: point it at any reference videos and it produces a pack.

pip install -r scripts/requirements.txt
export FAL_KEY=...            # only needed for the stages that call fal

Every network call is stubbed under TASTE_FORGE_DRY_RUN=1 or --dry-run, so the plan, prompts, track layout and manifest can be inspected without spending.