# Orchestration

> The master program for producing or editing a video end to end — read this at the START of any video task (after video-router), then follow the gates and the per-line steps. Trigger for "make / edit / cut / caption / dub / animate a video"; it sequences the compose / generate / edit lines and the approval gates. Do NOT trigger for a single low-level operation (just transcribe a file, just probe a clip) — call that operation directly.

- Skill: `orkas-ai/orchestration` (Agent Skill)
- Install (CLI): `npx skillmds@latest add orkas-ai/orchestration`
- Raw SKILL.md: https://api.skillmd.com/api/skills/orkas-ai/orchestration/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Productivity
- Author: Orkas-AI (https://skillmd.com/u/orkas-ai)
- Updated: 2026-09-10
- Page: https://skillmd.com/skills/orkas-ai/orchestration

---


# orchestration

You are producing a short video. Run this as a TIGHT program — lean turns — but STOP at every GATE so the creative decisions stay the user's. Only the technical/assembly steps between gates run unattended. The CLI surface is `ovs ...` (an MCP server mirrors it 1:1; use whichever your host exposes). All work for one deliverable lives under a single project dir, e.g. `project/`.

Read `gate-control` once before the first user gate. It is the single authorization policy across all lines. After every gate reply, resumed approval, post-gate revision, or exhausted visual-QA result, run `ovs gate transition` and follow its one returned action; line sections below define artifacts and production steps, not a competing approval state machine.

## Checkpoint protocol — how every GATE works (there is no special form UI)

1. **Show the artifact in chat** so the user can actually see it — script/plan as markdown, images inline, a draft video as its output file path — plus one line of "what I'll do next" and any cost/QA note.
2. **State the options** for that gate and **WAIT for the user to reply**. Do not run the next production step in the same turn as the gate.
3. **On reply, resolve the choice** with `ovs gate transition`: approve → continue only with the returned operation; revise → redo only the authorized scope, re-show, and re-gate only when the resolver says so; abort → stop. Never pass a gate without explicit user confirmation, and never ask again for an unchanged artifact whose approval is already recorded.

## 1. Route + lock (read `video-router`)

Classify and LOCK the line (no silent switching):
- **COMPOSE** — explain / teach / animate / motion-graphics / kinetic text, no source footage → `ovs draft` (+ optional `ovs image` / `ovs video` imagery, optional `ovs speak` narration).
- **GENERATE** — "footage of / a scene of / cinematic / a presenter or avatar speaking / talking-head" → AI footage via `ovs video` (+ `ovs image` for the subject, `ovs speak` for voice), assembled with `ovs edit`.
- **EDIT** — the user supplied real clips to cut / join / subtitle / localize → `ovs edit` (+ `ovs transcribe` for transcript-driven work).
- **AUTO (end-to-end)** — the deliverable spans MORE THAN ONE axis. Run the cross-modal orchestration (read `stage-plan`, then `stage-assemble`); the lock is the plan's `delivery_promise`.

## 2. GATE A — Proposal (all lines)

Show: the brief you inferred (line, aspect, duration, language) for the user to correct, plus 1–3 differentiated concepts (each: hook + look + rough length; for GENERATE add the shot count and that each clip is a billable call; for AUTO also state the proposed delivery promise — source_led / motion_led / compose_led / hybrid — and the rough segment mix). Options: pick a concept / adjust the brief / new direction. STOP.

## 2.5 Craft standard (ALL lines — read `video-craft`)

Before scripting / storyboarding / composing / generating, hold the output to `video-craft`: a hook in the first seconds, one idea per beat, readable type inside safe zones, restrained easing, muted-friendly captions, ducked audio, the right aspect for the platform. Bake these into the script/shotlist/composition — don't leave them to chance.

For COMPOSE or AUTO compose segments, also apply `frontend-design` before writing HTML. If the user supplied a DESIGN.md, brand guide, screenshot, existing app UI, Figma notes, or an explicit named style, apply `design-system-importer`. After a rendered draft, use `composition-design-review` only when its trigger applies and treat non-blocking findings as Gate D notes.

## 2.6 Narration voice

When the piece has voiceover, run `ovs speech-capabilities` and copy its executable route/model/voice/format into the Gate B plan together with the BCP-47 video language and a natural speed. Do not invent a voice id. Before Gate B, run `ovs narration fit --text ... --target ...`; revise over/under text internally before any paid synthesis. After `ovs speak`, probe the produced audio and run the same fit with `--measured`; retime scenes from the measured duration without silently shortening the approved target. If no TTS provider is configured, tell the user and explicitly choose silent delivery or wait for configuration.

TALKING-HEAD note: if a GENERATE clip already returned lip-synced built-in speech, THAT is the voice — do NOT synthesize a narration over it (a fresh TTS track desyncs from the mouth). Use `ovs speak` only for a silent clip, or for COMPOSE / EDIT / off-screen voiceover.

---

## COMPOSE line

3C. Script + storyboard (ONE step) → `project/script.md` + `project/shotlist.json`; include the design thesis inputs that `frontend-design` will put in manifest `art_direction`.
4C. **GATE B** — Script + storyboard sign-off. Show `script.md` + a shotlist summary. Options: approve / revise / change direction. STOP.
5C. (optional) Narration: after the free fit passes, `ovs speak` once → `project/assets/narration.mp3`, probe/measure it, retime the composition within the approved target, and add it as an `<audio>` track (see `stage-compose`). For a STANDALONE compose deliverable only; in AUTO the assembler mixes narration and compose segments render SILENT.
6C. (optional) Visual assets via `ovs image` / `ovs video` → `project/assets/`. Skip for pure typographic explainers. **If any asset is billable: GATE C first** — state the count + that they're billable; options approve & generate / adjust / skip. STOP, then generate.
7C. Compose (`stage-compose`) → `project/composition/composition-manifest.json` v2 and `project/composition/index.html`; prepare once and reconcile after manifest timing/audio changes. Use the optional HTML Preview Gate from `stage-compose` only when render rework is likely expensive.
8C. QA + draft: `ovs draft project/composition --out project/render/draft.mp4 --quality draft --report project/render/draft-report.json --findings project/composition/qa/check.json`; repair only concrete blockers within the bounded repair budget.
9C. **GATE D** — Draft review. Show the draft path + report/check/craft/design-review findings. Options: approve → high export / revise. STOP, then run `ovs draft project/composition --out project/render/video.mp4 --quality high --report project/render/final-report.json --findings project/composition/qa/final-check.json` once.

## GENERATE line (follow `stage-generate`; for recurring characters / a story, ALSO `stage-consistency`)

3G. Plan shots → `project/shotlist.json` (each: prompt, motion, duration, which characters are visible + camera angle). Keep the count tight (cost). For a STORY / long script: first build the global character bible + scene plan per `stage-consistency`.
4G. **GATE B** — Script + shotlist sign-off. Options: approve / revise / change direction. STOP.
5G. Character anchors (talking-head / any recurring character) — per `stage-consistency`: generate ONE locked front portrait per character with `ovs image` → `project/characters/` (or, for a cameo, use the user-uploaded photo AS the front portrait); record in `project/characters/bible.json`. NEVER regenerate a locked portrait.
6G. **GATE C** — Pre-generation confirm. State "N shots + M portraits ≈ X billable generations"; show the locked portrait(s) for approval. Options: approve & generate / adjust portrait / reduce scope. STOP.
7G. Generate each shot — per `stage-consistency`: pick references (angle-matched portrait of each visible character + the most recent prior frame), write an explicit ref→element prompt, `ovs video` image-to-video → `project/assets/shot-N.mp4` (KEEP the clip's built-in lip-synced audio; call `ovs speak` ONLY when the clip came back silent). After each shot, `ovs edit extract-frame` its last frame → `project/frames/` to carry forward.
8G. Assemble: `ovs edit concat` the shots → `project/render/draft.mp4`; add captions/lower-thirds via a small composition (`stage-compose`) + `ovs edit overlay` / `ovs edit burnsubs`. Captions/overlays are VISUAL-ONLY — preserve each clip's built-in audio through assembly.
9G. **GATE D** — Draft review. Show the assembled draft + craft findings. Options: approve / revise. STOP, then finalize → `project/render/video.mp4` (carry talking-head audio through UNCHANGED; captions/loudness only).

## EDIT line (read `stage-edit`)

3E. Ingest — `ovs edit probe` each clip for durations/resolution.
4E. Plan → `project/plan.json` (the segments EDL + a `tracks.narration` track of timed lines, so each narration line / caption stays separately re-editable). GROUND the plan on the clip's ACTUAL content BEFORE writing any narration:
   - spoken audio → `ovs transcribe raw/clip.mp4 --out project/transcripts/clip.json` (`--model large-v3` for non-English), pick timecodes from the transcript.
   - SILENT / screen-recording / slideshow → prefer `ovs ocr` for per-timecode on-screen text; if local OCR reports a runtime/install/recognition failure, extract frames across the clip with `ovs edit extract-frame` and read them yourself. Then write each narration segment to match the slide in its window. Do NOT narrate from prior knowledge of the topic.
   - If OCR/frame reading is unavailable, STOP and ask the user for the on-screen beats. Never invent narration.
5E. **GATE B** — Edit plan sign-off. Show the plan (segments / order / subtitles / localization; for highlights, the chosen moments + timecodes). Options: approve / revise / change selection. STOP.
6E. Execute: `ovs edit trim` → `project/cuts/`; `ovs edit concat` → `project/render/draft.mp4`; optional `burnsubs` / `overlay`. Adding narration to footage that ALREADY has audio: `ovs edit mix` — choose on purpose: keep the original under the voice (`--on-existing-audio mix`) or drop it (`--on-existing-audio replace`). It rejects by default when the base already has audio, so decide explicitly. Localization/dubbing: transcribe → translate → `ovs speak` → `ovs edit mix --on-existing-audio replace` + burn captions.
7E. **GATE D** — Draft review. Show the edited draft + mix coverage (when narration was added) + the loudness check (`ovs edit loudness` vs ~−14 LUFS / −1 dBTP). Options: approve / revise. STOP, then finalize with `ovs edit normalize-loudness` when needed → `project/render/edited.mp4`.

## AUTO end-to-end line (read `stage-plan`, then `stage-assemble`)

Ingest every supplied clip from evidence (probe + transcribe/OCR-or-frame-reading/extract-frame), author ONE cross-modal `project/plan.json`, `ovs plan validate` and fix every error, **GATE B** on the timeline (`ovs plan summarize`), **GATE C** only if the plan has billable `generate` segments with the exact count and exact `media_kind`/duration/ratio/resolution/audio/reference settings, then assemble per `stage-assemble` (produce each segment via its line, mix narration ONCE, music ducked, burnsubs, normalize loudness). At **GATE D** run `ovs plan promise-check --probe-produced` plus a draft review, then finalize.

---

## plan.json as the editable record (all lines) — keep follow-up edits cheap

Once a draft exists, keep `project/plan.json` faithful so a later tweak only re-touches one piece (never the whole video): (1) every produced segment carries its real output under `produced_path` + `status:"done"`; (2) narration is `tracks.narration` whose lines each carry their own `produced_path`, so one line can be re-voiced alone; (3) captions are DATA in `tracks.captions.lines` ({text, start_sec, target_sec}) — NOT burned into the picture — so a typo is a one-line edit re-burned at assemble; (4) set top-level `"draft": "render/draft.mp4"`.

**Local follow-up edits — make the minimal targeted change; never redo the whole video.** Once a `plan.json` is present and the user asks to change ONE local thing (a segment's narration / caption / text, a trim, volume / speed, a single shot swap), edit ONLY the matching entry in `plan.json` and re-produce ONLY what it touched (`ovs speak` for that one line, `ovs draft` for that one compose segment, or `ovs edit` for that one cut), then re-assemble. Do NOT re-author the whole EDL and DO NOT regenerate a segment whose `status` is `done` that the user did not touch. Fall back to a full re-plan only when the request genuinely restructures the timeline.

## Deliver (all lines)

Run the `video-craft` pre-publish review on the FINAL file (readable / timed / on-message / consistent / audio [talking-head: confirm the built-in lip-synced voice is intact] / platform; fix any blocker first). Present the final output file path + a one-line summary. End the turn.