# Riffkit

> Riff winning short videos — give one source (a TikTok link, an uploaded video, or an analyzed template) and the backend riffs its emotion formula into your own AI video (post-ready short-form or UGC-style ad creative), with optional digital character, product placement, and language. You riff the formula, not the video. Triggers: the user says 'riff this video', 'turn this TikTok into mine', 'make a video with this product', 'make an ad' / 'make an ad creative' / 'a UGC ad for my product', 'make a promo / marketing video for my app or product', 'remake a viral video', 'generate a short video', 'riff', 'riffkit', or sends a product image / viral link wanting a short video.

- Skill: `riffkit/riffkit-2` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add riffkit/riffkit-2`
- Raw SKILL.md: https://api.skillmd.com/api/skills/riffkit/riffkit-2/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Marketing & Growth
- Author: riffkit (https://skillmd.com/u/riffkit)
- Updated: 2026-09-21
- Page: https://skillmd.com/skills/riffkit/riffkit-2

---


# Riffkit Skill

**Core stance: you riff the formula, not the video.** Give one winning source; the backend analyzes the emotion formula that hijacks attention and migrates that formula onto your own content. The footage can be completely different as long as the viewer travels the same psychological path.

**One screen, one action: source (required) → optional settings → submit.** The real product is a single page and a single call (`POST /api/riffs`). Every setting other than the source has a sensible default — character defaults to **Auto (no digital human; the AI generates the on-camera person)** and product defaults to **none**. When the user doesn't care, the agent applies defaults silently instead of dragging them through a multi-step wizard.

**The agent's highest-value contribution is `content_anchor` (the creative direction)** — the one degree of strategic freedom: which of the product's N selling points to angle on, which surface to fill into the template's emotion mechanism. It is an **optional collaboration, not a blocking hard-stop.** See `## content_anchor drafting framework` below.

## Skill scope

This skill makes short AI videos in exactly two modes: **riff videos** (analyze a source video's emotion formula and regenerate it as your own) and **creation videos** (author an original ad video from a written creative direction — no source video; `POST /api/creation/batch`). That is the entire product surface. If a user asks for something outside this — a different content format, or a feature this product doesn't have — say plainly that this product only makes riff videos; don't call unrelated APIs and don't steer them elsewhere.

**No staff/admin features are exposed.** This skill covers only endpoints a normal authenticated user can call. Building platform templates by analyzing new sources, publishing/unpublishing platform templates, cross-scope task search, manually granting/clawing back credits — all staff-only. This document never lists them and the agent never calls them.

## Language

**Output follows the user's input language**: reply in English to English, in Simplified Chinese to Chinese; for mixed input, follow the dominant language of the current message. The agent's internal reasoning is exempt.

**Always keep verbatim (do not translate)**: field IDs, API paths, `template_type` (only `pipeline`), status enums (queued/running/completed/failed/dead/cancelled), `product_visibility` values (on_camera/off_camera/no_product), parameter names, the `vee_session` token.

---

## One-minute overview (TL;DR)

```
[Flow]
  1. Pick the source (exactly one, required)
       ├── analyzed template  formula_id        →  skips analysis, generates now (fastest)
       ├── TikTok link        tiktok_url        →  backend downloads + analyzes + generates
       └── uploaded video     video (≤100MB, ≤ render cap) →  backend analyzes + generates
            ↓
  2. Optional settings (all defaulted; agent may suggest, never forces)
       character     default Auto (AI-generated person); may suggest a fitting character on account intent
       product       default none (no_product); attach an existing/new product to place one
       visibility    default on_camera; only meaningful when a product is attached
       language      default en; candidates from GET /api/languages (currently en / es / pt / id / de / fr / it / ja / zh-CN)
       content_anchor optional creative direction; agent may proactively draft one for review
       user_hint     optional hook hint; only used for a NEW source (ignored for a template)
            ↓
  3. Confirm before submit (the only hard-stop)  →  POST /api/riffs
            ↓  ↳ insufficient balance returns HTTP 402 (structured); handle per "Billing & balance"
  4. Monitor  GET /api/tasks/batch/{batch_id}  (every 10-15s)
            ↓
  5. Deliver  GET /api/assets → download links + caption + hashtags + strategy recap
```

**The only hard-stop is that one pre-submit confirmation** (the financial commitment). Character, product, and content_anchor are optional collaborations and never block the flow.

**Endpoints at a glance:**

| Endpoint | Purpose |
|------|------|
| `GET /api/auth/me` | Check auth state |
| `POST /api/skill/device/authorize` · `POST /api/skill/device/token` | **One-click sign-in (device authorization; no token paste)** |
| `POST /api/riffs` | **One-shot riff (the preferred, near-only generation entry)** |
| `GET /api/formulas` | List analyzed templates (one of the sources) |
| `GET /api/formulas/{id}` | A template's `extraction_summary` (what was extracted) |
| `POST /api/formulas/analyze` | **Subscribers only** — analyze a new source into your own template *without* generating (build a library) |
| `POST /api/formulas/{id}/refresh-analysis` | Re-analyze a template whose analysis is stale |
| `GET /api/characters` | Digital characters (optional binding) |
| `GET /api/products` · `POST /api/products` · `POST /api/products/{id}/images` | Products + product images (optional placement) |
| `GET /api/languages` | Video language candidates |
| `GET /api/tasks/batch/{id}` · `GET /api/tasks/{id}` | Progress polling |
| `GET /api/tasks` · `GET /api/tasks/stats` | List / count tasks |
| `POST /api/tasks/{id}/cancel` · `POST /api/tasks/{id}/retry` | Cancel / retry |
| `GET /api/assets` | Fetch finished videos (video + caption + hashtags) |
| `POST /api/pipeline/backfill` · `GET /api/pipeline/backfill/occupied` | Add extra aspect ratios to an already-delivered render (reframe) / list ratios already produced |
| `GET /api/usage/credits` | Balance |
| `GET /api/billing/plans` · `GET /api/billing/subscription` | Plan catalog / current plan (for post-402 upsell) |

Full params and responses in "API reference" below.

---

## Rules of engagement (hard constraints)

**The agent never submits on its own. It stops once for explicit consent before submitting.** Everything else may proceed on defaults.

Do NOT:
- **Auto-submit** a task just because the user said "riff this" (deciding the source + config is fine; the submit must wait for a go-ahead)
- Treat "pick a character / pick a product" as an unskippable step — **character defaults to Auto, product defaults to none**; use the defaults when the user hasn't asked for either
- Treat drafting `content_anchor` as a hard-stop that must be iterated to the user's satisfaction before continuing (it's an optional collaboration)
- **Proactively report credit numbers / query the balance** — no estimate at the confirmation step; balance only surfaces on a 402, before a retry (which re-charges — see `POST /api/tasks/{task_id}/retry`), or when the user asks
- Auto-retry a failed task (retry re-charges)
- Persist product info the user hasn't explicitly confirmed
- Call any staff-only endpoint or probe paths not listed here

Do:
- **Lock the source first** (one of three) — the only required input
- Before submitting, restate the plan (source / character / product+visibility / language / content_anchor) and ask "Submit?" → on confirmation, call `POST /api/riffs`
- On **HTTP 402**, follow "Billing & balance": relay `topup_url` verbatim, **no retry, no silent failure**
- Only call `GET /api/usage/credits` when the user actively asks "how much will this cost / how much do I have left"
- Ask when input is ambiguous rather than guessing and proceeding
- Surface errors honestly as they happen; never silently retry
- On a finished video, present only the download link + copy — **never** publish to any platform

Until the user says "submit / generate / riff / go", you are a collaborator that drafts and presents a plan — not a command executor.

---

## Core idea: the three responsibility layers (why content_anchor is the agent's value)

| Layer | Role | Locked by | Freedom |
|---|---|---|---|
| **Formula + skeleton** | **Floor guarantee** — a validated emotion mechanism + camera language | At template analysis | None (changing it forfeits the riff's value) |
| **Character + product** | **Base constants** — the digital human + product facts | Chosen in settings (or default) | Different picks = different constants, but constant within one task |
| **content_anchor** | **Ceiling driver** — which selling-point angle, which surface to fill | Agent + user draft it (optional) | **The one degree of strategic freedom** |

The formula skeleton decides which psychological path the viewer walks; `content_anchor` decides what specific content fills that path. The other layers are pre-existing constants, so **the agent's differentiated value is fusing "source formula × product/account × character" into one concrete creative instruction.**

---

## Full workflow

### Step 1: Lock the source (required, one of three)

| Source | Param | When |
|---|---|---|
| **Analyzed template** | `formula_id` | The user wants an existing template, or has riffed this source before — **skips analysis, fastest** (analysis is free either way; skipping it saves the wait, not credits) |
| **TikTok link** | `tiktok_url` | The user dropped a viral link; the server auto-downloads the video + extracts BGM |
| **Uploaded video** | `video` | The user has a local file (≤100MB, and within the render-duration cap — see General constraints; a longer source is rejected, not trimmed) |

- Template candidates: `GET /api/formulas?status=analyzed&template_type=pipeline`. `visibility=public` are platform-curated templates (usable across scopes, prefer recommending them); a template with `analysis_prompt_is_latest=false` has stale analysis — suggest `refresh-analysis` before using it.
- **The same TikTok link already analyzed in this scope → the backend reuses the cached analysis** (free, faster); the agent needs no special handling.
- The three sources are **mutually exclusive**; exactly one must be provided (else 400).

### Step 2: Optional settings (all defaulted)

Each can be left alone on its default; the agent may suggest where helpful but **never blocks**.

**Character (default Auto)**
- By default `character_ids` is empty = **Auto mode**: no digital human bound, SD2 generates the on-camera person. This is the product default, not an edge case.
- **The agent may proactively pick/suggest a fitting character** — when the user expresses account/persona intent ("post it to my health account", "use my creator persona"), read `GET /api/characters` and match by `persona` feel + `gender` / `age_range`, then suggest one. **Only suggest characters with `has_any_active_avatar=true`** (a `false` character can't generate video yet; the user must approve its avatar in Settings first).
- If the user expresses no account intent, **proceed silently on Auto** — don't interrupt just to make them choose.
- Multiple characters: only pass several when the user explicitly says "make one for each of these characters" (one task per character).

**Product (default none)**
- By default `product_id` is empty = `no_product` mode: pure content, the caption never mentions a product name or product CTA, the whole video just runs the template's emotion formula. Good for growth / relatability / educational content.
- To place a product:
  - **Existing product** → `GET /api/products`, take the `product_id`.
  - **New product** → stage the fields (name / description required) in memory; **defer the real `POST /api/products` write until just before submit** (don't leave a half-baked product in the DB before the plan is settled).
- Product images: upload clean product photos / app screenshots (no watermark, no browser chrome, subject centered). If the original has noise, the agent may crop/clean it before uploading (see `POST /api/products/{id}/images`). **Every image must have a `name`** — to put a specific image on camera, write that image's `name` directly in `content_anchor` text (see below); an unnamed image can't be referenced.

**Visibility `product_visibility` (only meaningful with a product; default on_camera)**

| Value | Meaning | Best for |
|---|---|---|
| `on_camera` (default) | Product appears as a physical object on screen (character holds / scans / shows it) | Food / cosmetics / small physical goods / packaging as the core hook |
| `off_camera` | Product never enters frame; conveyed only via subtitles / voiceover / caption text | Apps / websites / SaaS / services / non-portable goods |

When `product_id` is empty this field is ignored and the backend derives `no_product`. **The caller may not pass `no_product` directly** (only the two literals on_camera / off_camera are accepted). The script and visual staging differ greatly across modes, so when a product is bound always state the value and the reason at the confirmation step.

**Language (default en)**
- Candidates from `GET /api/languages` (currently `en` / `es` / `pt` / `id` / `de` / `fr` / `it` / `ja` / `zh-CN`, in picker order). Trust the endpoint, don't hardcode.

**content_anchor (optional creative direction) + user_hint (optional hook hint)**
- `content_anchor` is the agent's highest-value contribution: it may proactively draft one for the user to review (see `## content_anchor drafting framework`). If the user doesn't want one, leave it empty — the video still generates.
- `user_hint` feeds only a **new source's** analysis ("this popped off on the twist at 0:03"); it's ignored when a `formula_id` is chosen, so don't send it then.

### Step 3: Confirm + submit (the only hard-stop)

Restate the plan, **no credits, no balance pre-check**:

```
Ready to riff:
├── Source: [template name / TikTok link / uploaded filename]
├── Character: [name / Auto (AI-generated person)]
├── Product: [name + visibility / none]
├── Language: [en / es / pt / id / de / fr / it / ja / zh-CN]
└── content_anchor: [drafted creative direction / none]
```

When the user says "submit / generate / riff" → call `POST /api/riffs`.
- If a **new product** was chosen, first `POST /api/products` (+ upload images serially) to get the `product_id`, then include it in the riff.
- Insufficient balance returns **HTTP 402** (structured `insufficient_credits`) → handle per "Billing & balance".

### Step 4: Monitor progress

- The whole riff shares one `batch_id` (the analyze task and the chained generation task both carry it) → poll `GET /api/tasks/batch/{batch_id}`.
- Every **10-15 seconds** (shorter is pointless, longer feels dead); cap a single poll loop at **15 minutes** (pipeline tops out around 8 min, 2× tolerance), then pause and tell the user.
- Summarize, don't echo every poll: "running 2m30s, currently Stage B — creative adaptation," roughly once a minute.
- Failure handling: on `failed`/`dead`, read `error` to locate the cause, **don't auto-retry**, tell the user and let them decide; if `queued` for over 2 minutes, note "server is at its concurrency cap (10), please wait."
- **Insufficient credits mid-riff** (a new-source riff clears the submit gate, then the real duration proves too costly — since v1.1.3 a low-balance riff usually gets an instant `402` at submit instead: TikTok URLs via a metadata duration probe, uploads via the on-disk file's real duration; this mid-riff case remains only when the TikTok probe couldn't determine the duration): the analyze task ends with `result.auto_generate_error == "insufficient_credits"` and `result.insufficient_credits` = the same structured 402 payload (`required_credits` / `available_credits` / `topup_url`). This means **no video was generated** — even when `status == "completed"` (the analysis finished but generation was skipped). Treat it like a 402: relay `topup_url` verbatim, tell the user to top up, and note they can then **retry the same task** (`POST /api/tasks/{id}/retry`, within 24h — no re-submit needed). **Never report success on a riff whose analyze task carries this field.**

### Step 5: Deliver

`GET /api/assets?asset_role=final_reel&sort=created_desc&limit=10` (add `formula_id` / `character` to filter this run):

1. **Download URL** — `${BASE_URL}${file_url}` (direct video link). This is the ONLY download path — there is **no** `/api/assets/{id}/download` sub-resource (it 404s; do not invent REST-style suffixes). The GET needs the same `Cookie: vee_session=<token>` as every API call, and must follow redirects (`curl -L`): in production it 302s to object storage. A cookie-less GET returns 401.
2. **Suggested copy** — `caption` (hook → body → closing call-to-action folded into one paragraph) + `asset_hashtags`
3. **Strategy recap** — which emotion formula this used, through which beat the product was felt, what the content_anchor did. To see what the engine actually "extracted / rewrote," call `GET /api/tasks/{task_id}/content`.
4. **Next iteration** — next time tweak content_anchor / character / product combo.

---

## content_anchor drafting framework (core subsection)

> The formula and skeleton decide which psychological path the viewer walks; `content_anchor` decides what specific content fills that path.
> When non-empty it is the **highest-priority input** for surface direction.
> Failure test: if swapping the surface for any other topic still holds, the anchor never anchored the output → invalid.
>
> **Product-image targeting (on-camera placement)**: naming a product image's exact name in the anchor narrows what the engine receives to ONLY the named image(s) — the rest of the product's images are withheld from that render. Name none → all images ship (default). Use this when the product has many images and the video should feature a specific one (e.g. "开场特写 正面图"); image names come from `GET /api/products` → `images[].name`. Matching is case-insensitive with word boundaries for ASCII names.

**Drafting template:**

```
[a specific emotion-mechanism beat of the template] × [a specific feature of the product/account] → [the viewer mind-shift you want]
```

All three variables must be specific to an actionable level — anything abstract is as good as empty.

| ✅ Focus on | ❌ Don't (lives elsewhere or zero-info) |
|---|---|
| The specific product × template join ("the scan feature × the reveal beat at segment 2") | Product generalities ("show the product's strengths") |
| The angle you want this time (which of N selling points) | Template generalities ("use the funny formula") |
| One specific face of the audience's pain point | Account positioning ("health niche" — already in persona) |
| The viewer mind-shift ("from 'I assumed it was safe' to 'a quick scan reveals hidden additives'") | Generic creative words ("authentic / real / heartfelt") |

**Where the anchor's weight goes per mode:**

| Mode | content_anchor weight |
|---|---|
| `on_camera` | Product **visual** feature × the template's on-screen action ("the package-scan gesture × the reveal beat's curiosity→surprise") |
| `off_camera` | Product **function/benefit** × the template's voiceover/subtitle ("the pain the app solves × the hook's resonance → download urge") |
| `no_product` | The account's specific angle × the template's emotion formula → the resonance you want (**the anchor matters most here** — with no product, it's the only thematic anchor) |

### How to write it (craft)

The engine already mirrors the source. Your anchor is a **delta**, not a brief.

1. **Say only what should differ from the source.** Everything you don't mention is inherited. If the only thing that changes is who is on camera, the correct anchor is empty — writing more pulls the render away from a formula that already works.

2. **Locate every change.** A change stated as a concept loses to the source; the same change stated with a place — which beat, which moment, what happens right before and after — is the one that lands.

3. **Length tracks how far you're departing, not how much you care.** A big departure needs detail; a small one needs a line. "This video matters to me" is never a reason to write more.

4. **Keep separate axes separate.** How it's shot (lighting, grain, camera feel) and what's in it (wardrobe, props, setting) are different axes. Collapse them into one sentence and one will drag the other — asking for an unpolished look often flattens the subject too.

**Building your own guard list.** When a render comes back with something you never asked for, that is the engine's default showing. Add an explicit "not X" next time. Experienced users accumulate a short list of these and paste it into every anchor — it is the cheapest thing they do.

**Place a product image on camera by name (on_camera only)**: write the product image's `name` directly in `content_anchor` text and the engine matches that name and places the image on screen. The image must be named (an unnamed image can't be referenced). Example: writing in `content_anchor` "use the ingredient-scan screen shot to reveal the hidden additives" puts the image named "ingredient-scan screen" into the matching shot. (This is plain name matching, not an @-syntax — the @-mention is only a web-UI textarea helper that inserts the name for you; agents write the name themselves.)

---

## API reference

### Service config

```
BASE_URL = https://riffkit.ai
Content-Type: application/json; charset=utf-8  (except multipart endpoints)
Auth: cookie-based session (vee_session)
```

Every path below already includes the full prefix — just append it to `${BASE_URL}` (e.g. `GET /api/auth/me` → `https://riffkit.ai/api/auth/me`).

⚠️ **Request bodies must be UTF-8.** Python `requests.post(url, json=...)`, Node `fetch`/`axios`, Go `json.Marshal` are UTF-8 by default — pure-ASCII needs nothing. **Only** on Chinese Windows `cmd` run `chcp 65001` first (PowerShell also needs `[Console]::OutputEncoding = [System.Text.Encoding]::UTF8`), or non-ASCII characters get sent as GBK and rejected with `BAD_REQUEST`. Never assemble a byte string with `data=` in any language.

### Auth

The API uses a cookie-based session (`vee_session`). **Never ask for a password in chat.** The agent obtains a session through a **one-click device-authorization flow** — the user just opens a link and clicks Approve, and the session flows back automatically. **No token is ever pasted into chat.** (Same UX as `gh auth login`.)

1. Check: `GET /api/auth/me` → 200 logged in / 401 not.
2. If not logged in (401), run the device flow:
   - **a. Start** — `POST /api/skill/device/authorize` (no body, no auth needed) → `{device_code, user_code, verification_uri, verification_uri_complete, expires_in, interval}`.
   - **b. Show the user the link + code** (do NOT ask for anything back):
     ```
     Open this and click Approve — I'll connect automatically:
     <verification_uri_complete>
     (confirm the page shows this code before approving: <user_code>)
     ```
   - **c. Poll** — `POST /api/skill/device/token` with `{"device_code": "<device_code>"}` every `interval` seconds (default 5s):
     - `{"status":"authorization_pending"}` → keep polling
     - `{"status":"approved","token":"<t>"}` → **done**; use `Cookie: vee_session=<t>` on every later request
     - `{"status":"expired"|"denied"|"invalid"|"consumed"}` → stop and start over with a fresh `authorize`
     - stop after `expires_in` (10 min) and tell the user the link expired
3. Add `Cookie: vee_session=<value>` to every subsequent request.

> The device flow is the only sign-in path — the token never gets pasted into chat. (Settings → **AI Agent mode** shows the same one-click steps.)

#### `GET /api/auth/me`

**200 → `UserOut`** / **401 → unauthenticated.**

| Field | Type | Notes |
|------|------|------|
| `id` | string | User ID |
| `email` | string | Email (= identity; no separate name) |
| `role` | string | Scope role: `owner` / `admin` / `member` |
| `is_active` | boolean | Active |
| `is_staff` | boolean | Product-level staff (default false) |
| `scope_id` | string? | Owning scope |
| `daily_credits_limit` | float | Daily credit cap (`0` = unlimited) |
| `created_at` / `last_login_at` | datetime | Created / last login |

#### `POST /api/skill/device/authorize` — start one-click sign-in

No body, no auth. **Response:**

| Field | Type | Notes |
|------|------|------|
| `device_code` | string | **Secret** — the agent polls with it; never show it to the user, never write it anywhere |
| `user_code` | string | Short code shown to the user (they confirm it matches the approval page) |
| `verification_uri` | string | Approval page (bare) |
| `verification_uri_complete` | string | Approval page with the code pre-filled — **give the user this link** |
| `expires_in` | int | Seconds until the flow expires (600) |
| `interval` | int | Seconds to wait between polls (5) |

#### `POST /api/skill/device/token` — poll for the session

**Body:** `{"device_code": "<device_code>"}`. **Response `{status, ...}`:**

| `status` | Meaning | Action |
|------|------|------|
| `authorization_pending` | User hasn't approved yet | Wait `interval` seconds, poll again |
| `approved` | Approved — response also has `token` | Use `Cookie: vee_session=<token>`; stop polling |
| `expired` / `denied` / `invalid` / `consumed` | Flow is dead | Stop; start over with a fresh `authorize` |

> The minted `token` is a normal session (identical to a browser login). Treat it like a credential: never echo it, never store it in a task/caption/product field.

---

### Video generation

#### `POST /api/riffs` — one-shot riff (preferred entry)

**Content-Type:** `multipart/form-data`

> **Non-ASCII text: write it to a UTF-8 file, don't inline it in the shell.**
> For `content_anchor` and `user_hint`, pass the value by file reference:
>
> ```
> printf '%s' "$BRIEF" > /tmp/anchor.txt      # or your language's write-file call
> curl … -F "content_anchor=<'/tmp/anchor.txt'"
> ```
>
> (`-F "name=<file"` reads the field VALUE from the file. That is not `@file`,
> which would attach it as an upload.)
>
> Why: when a brief is typed straight into a `curl -F "content_anchor=主题：…"`
> command, the bytes that reach the wire are whatever the shell's codepage
> produced. On a non-UTF-8 console — the Windows default in CJK locales — those
> are GBK/Big5/Shift-JIS bytes, and the field is stored as mojibake
> (`主题：一条…` → `Ö÷Ì⣺һÌõ…`). This happened on prod: six riffs rendered
> from garbage briefs and were billed normally. The server now rejects it with
> **400** instead, but the 400 is a backstop — you cannot see the corruption
> from inside the agent, because the text you wrote was correct and it was
> mangled a layer below you. Writing the file avoids the shell layer entirely.
>
> **If you do get that 400:** do NOT resend the same command; it will fail
> identically. Switch to the file form above. Already using it? Then the file
> itself isn't UTF-8 — rewrite it with an explicit UTF-8 encoding.

**Source (exactly one):**

| Param | Type | Notes |
|------|------|------|
| `video` | File | Upload source video (≤100MB, and ≤ the render-duration cap — default 45s; see General constraints) |
| `tiktok_url` | string | TikTok link (server downloads + extracts BGM). Must point at **one specific video** — `…/@user/video/<id>` (query params fine) or a `vm.`/`vt.`/`tiktok.com/t/` share short link. A profile-page link (`tiktok.com/@handle`, no `/video/`) is rejected with an instant 400 |
| `formula_id` | string | Analyzed template ID (yours or a public one; status must be `analyzed`, else 400) |

**Optional creative config:**

| Param | Type | Default | Notes |
|------|------|------|------|
| `character_ids` | string | `""` | JSON array string (`'["caden","chloe"]'`) or comma-separated (`caden,chloe`). **Empty = Auto mode** (no digital human, SD2 generates the person); non-empty = one task per character. **Note it's a string, not an array** (multipart limitation) |
| `product_id` | string | `""` | Empty = no product placement (`no_product` mode) |
| `product_visibility` | string | `on_camera` | `on_camera` / `off_camera`; only effective when `product_id` is non-empty (ignored when empty) |
| `language` | string | `en` | Must be a code from `GET /api/languages` (currently `en` / `es` / `pt` / `id` / `de` / `fr` / `it` / `ja` / `zh-CN`); an invalid value returns 400 |
| `video_backend` | string | `seedance` | `seedance` (Seedance 2.0, default) / `seedance25` (Seedance 2.5, premium) / `minimax`. Picks the render engine. **Resolution is engine-scoped** — see `resolution` below. An engine the deployment has no key for → 400 `video_backend_unavailable`; unknown value → 400 |
| `resolution` | string | engine base | Engine-scoped: `720p` / `1080p` for `seedance`, `720p` only for `seedance25`, `768P` / `2K` for `minimax`. **Omit it** and you get that engine's base tier; a value the picked engine doesn't sell is a 400. Billing is per second at the (engine, tier) display rate — 720p 100 credits/s · 1080p 250/s · seedance25 720p 150/s · H3 768P 40/s · H3 2K 80/s. `1080p` on a Fast-tier deployment → 400 (Fast has no 1080p). Live rates: `GET /api/billing/subscription` → `video_credits_per_second_map`; the engines a deployment offers and each one's tiers: `GET /api/settings` → `video_backends` |
| `content_anchor` | string | `""` | Creative direction (≤5000 chars); to place a product image on camera, write that image's `name` in the text (on_camera; plain name match) |
| `user_hint` | string | `""` | Hook hint (≤5000); **new source only** — ignored when `formula_id` is given |
| `video_ratios` | string | `'["9:16"]'` | JSON-array string of delivery aspect ratios. **Vertical group `9:16` / `3:4` / `1:1` / `4:5` can be multi-selected** (one master render fans out into a reframed video per ratio, each metered as its own video at the **reframe** rate — on MiniMax H3 that is **80/s**, 2× that engine's 40/s render rate, still under 720p's 100/s; see Billing); a **horizontal ratio `16:9` / `4:3` / `21:9` must be requested alone** (list length 1). Deduped + returned in canonical order. Invalid ratio / horizontal-mixed → 400 |

**Response (`RiffOut`):**

| Field | Type | Notes |
|------|------|------|
| `mode` | string | `"generate"` (`formula_id` already analyzed → generation batch submitted now) / `"analyze_then_generate"` (new source → analyze submitted first; on completion the worker chains the generation) |
| `batch_id` | string | **The riff's handle** — the analyze task and chained generation task share it; poll `GET /api/tasks/batch/{batch_id}` to track the whole run |
| `formula_id` | string | Template ID (a new source creates a placeholder-named template, auto-renamed by a hook once analysis lands) |
| `analyze_task_id` | string? | Analyze task ID (only in `analyze_then_generate`) |
| `task_ids` | string[] | Generation task IDs (immediate in `generate`; in the chained mode they appear after analysis, fetched from the batch) |

**Behavior notes:**
- **Rate limit 10 / 60s**; exceeding the daily credit cap returns 429.
- The backend runs a pre-submit balance hold check; on shortfall it returns **HTTP 402** (see "Billing & balance").
- A new source's analysis isn't charged, but is guarded by a **free-cost guard** — spamming new-upload analyses gets blocked (a genuine first riff never is).
- BGM is handled by the backend automatically (use the source BGM if present, else AI-generate it). It is **not a riff parameter** — the agent neither needs to nor can set it here.

#### `POST /api/pipeline/batch` — riff video (advanced / analyzed-template batch)

`riffs` already covers nearly everything (including multi-character batches). This endpoint remains for fine-grained "analyzed template + explicit params" control; the agent rarely needs it.

| Field | Type | Req | Default | Notes |
|------|------|------|------|------|
| `formula_id` | string | ✓ | | Template ID (status must be `analyzed`) |
| `character_ids` | string[] | | `[]` | Character ID **array** (an array here, unlike riffs' string). Empty array = Auto mode |
| `product_id` | string \| null | | `null` | `null`/omitted = `no_product` |
| `product_visibility` | string | | `on_camera` | Only `on_camera`/`off_camera`; `no_product` is derived from `product_id=null`, never passed directly |
| `content_anchor` | string | | `""` | ≤5000 chars |
| `language` | string | ✓ | | Must be a code from `GET /api/languages` |
| `video_backend` | string | | `seedance` | `seedance` (Seedance 2.0, default) / `seedance25` (Seedance 2.5, premium) / `minimax`. Picks the render engine. **Resolution is engine-scoped** — see `resolution` below. An engine the deployment has no key for → 400 `video_backend_unavailable`; unknown value → 400 |
| `resolution` | string | | engine base | Engine-scoped: `720p` / `1080p` for `seedance`, `720p` only for `seedance25`, `768P` / `2K` for `minimax`. **Omit it** and you get that engine's base tier; a value the picked engine doesn't sell is a 400. Billing is per second at the (engine, tier) rate, quoted against the 720p base: 1080p ×2.5, seedance25 720p ×1.5, 768P ×0.4, 2K ×0.8. `1080p` on a Fast-tier deployment → 400 (Fast has no 1080p). Live rates: `GET /api/billing/subscription` → `video_credits_per_second_map`; the engines a deployment offers and each one's tiers: `GET /api/settings` → `video_backends` |
| `video_ratios` | string[] | | `["9:16"]` | Delivery aspect ratios (array here, unlike riffs' string). Vertical group `9:16`/`3:4`/`1:1`/`4:5` multi-selectable (fans out one video per ratio × character; each extra ratio is metered at the reframe rate — on MiniMax H3 that is 80/s); a horizontal ratio `16:9`/`4:3`/`21:9` must be alone. Invalid / horizontal-mixed → 400 |

**Response (`PipelineBatchResponse`):** `batch_id` / `task_ids[]` / `total` (`task_ids` are the MASTER tasks; extra-ratio reframe children join the same `batch_id` after each master completes).

#### `POST /api/pipeline/backfill` — add ratios to already-delivered videos

Add extra **vertical** aspect ratios to renders you already have, without re-generating from scratch (each new ratio reframes the existing render). **Body (JSON):** `{source_asset_ids: string[], video_ratios: string[]}` (vertical ratios only — a horizontal ratio → 400). Any member of a render family works as the source: a reframed variant's `asset_id` resolves to the family's original master render automatically. **Response:** `{submitted: [{task_id, asset_id, ratio}], skipped: [{asset_id, ratio, reason}], batch_id}`. Skip reasons: `already_occupied` (ratio already delivered or in-flight for that family), `source_not_reframeable` (no reusable render — this also covers a **Seedance 2.5 master longer than 15s**: 2.5 renders ≤30s in one segment but reframes execute on the 2.0 engine whose per-call window is 15s, so long 2.5 masters can't fan out into extra ratios — and submitting multi-ratio on 2.5 with a >15s source is itself a 400 (`seedance25_multi_ratio_over_15s`), so the only route to several ratios at that length is the default engine), `landscape_source` (a `16:9`/`4:3`/`21:9` render can't be reframed — targets are portrait-only and cross-orientation reframe is unsupported; don't submit landscape sources). 402 when the balance can't cover the submitted reframes.

#### `GET /api/pipeline/backfill/occupied?asset_id=<id>` — ratios already produced

Returns `{occupied: string[]}` — the delivery ratios already delivered or in-flight for the asset's render family (grey these out in a ratio picker; they'd be skipped by the backfill).

#### `POST /api/creation/batch` — creation video (original, no source video)

The second generation mode: no source video, no template — the **creative direction IS the script's source**, so here it is REQUIRED (on riffs it optionally steers a template). The engine authors an original ad video from it (per-second billing, same rates as riffs).

**Content-Type:** `application/json`

| Param | Type | Required | Notes |
|------|------|------|------|
| `content_anchor` | string | **yes** | Creative direction, 1-5000 chars — the story/scene, captions, lines, pacing. The more specific, the more controllable |
| `character_ids` | string[] | no | Empty = Auto (AI generates the on-screen person) |
| `product_id` | string? | no | null = no product placement. `on_camera` placement requires the product to have images (400 otherwise) |
| `product_visibility` | string | no | `on_camera` (default) / `off_camera` |
| `duration_mode` | string | no | `smart` (default: AI picks the length by content, capped at 45s AND at what the balance affords) / `fixed` |
| `duration_seconds` | int | with fixed | 4-45; required when `duration_mode=fixed` |
| `language` | string | no | Default `en`; same whitelist as riffs |
| `video_backend` | string | no | `seedance` (Seedance 2.0, default) / `seedance25` (Seedance 2.5) / `minimax`; 400 if not configured on the deployment |
| `resolution` | string | no | Engine-scoped — `720p`/`1080p` (seedance), `720p` (seedance25) or `768P`/`2K` (minimax). Omit for the engine base tier; a value the picked engine doesn't sell is a 400. Same rate rules as riffs |
| `video_ratio` | string | no | Single ratio, default `9:16` (creation has no reframe fan-out at submit; use backfill later — currently riff-only) |

**Response:** `{batch_id, task_ids: string[], total}` — one task per character (or one Auto task). Task `type` is `creation`; poll the batch exactly like a riff. 402 detail shape is identical to riffs. Task output shows in Library like any riff (`AssetOut.content_anchor` carries the direction).

---

### Templates (formula library)

#### `GET /api/formulas`

**Query:** `status` (collected/analyzed/archived), `template_type` (use `pipeline`), `tags` (comma-separated), `search`, `sort` (created_desc/created_asc/used_desc), `limit` (default 50), `offset`.

**Response (`FormulaListOut`):** `items: FormulaOut[]` / `total` / `limit` / `offset` (header `X-Total-Count` = filtered total).

**FormulaOut (customer-visible fields):**

| Field | Type | Notes |
|------|------|------|
| `id` / `name` | string | Template ID / name |
| `template_type` | string? | Default `"pipeline"` (this skill only consumes this; filter out others) |
| `status` | string | `collected` / `analyzed` / `archived` |
| `emotion_arc` | string? | Emotion arc (generic funnel-stage sequence, e.g. "hook → build-up → cta") |
| `slot_count` | int? | Number of formula segments |
| `used_count` | int? | Times riffed (high = peer-validated) |
| `tags` | string[] | Tags |
| `source_url` / `source_platform` | string? | Original link / platform |
| `thumbnail_url` | string? | Thumbnail |
| `analysis_prompt_is_latest` | bool | `false` = analysis stale; `refresh-analysis` before using |
| `visibility` | string | `scope` (this scope only) / `public` (platform-curated, prefer recommending) |
| `created_at` | datetime? | Created |

> `hook_type` / `cta_type` (two legacy formula-derivative fields) are **not exposed to customers**; the per-slot formula itself IS exposed via `extraction_summary.formula_slots` (see below). The raw `analysis_card` stays customer-hidden.

#### `GET /api/formulas/{formula_id}` — extraction summary

**Response (`FormulaDetailOut`):** all FormulaOut fields + `analysis_card` (**always `null` for customers** — the full formula DSL is engine IP) + `extraction_summary` (a safe projection).

**`extraction_summary` (the agent reads this to understand the template, NOT analysis_card):**

| Field | Type | Notes |
|------|------|------|
| `duration_seconds` | float? | Source video length |
| `language` | string? | Source language |
| `speaking_mode` | string? | Speech form |
| `narrative` | string | Plain-language record of "what happens" in the source (internal markers stripped) |
| `transcript` | `[{at, text, mode}]` | Line-by-line dialogue (`at` = start second, `mode` = on_camera_dialogue/voiceover/no_dialogue) |
| `on_screen` | `[{kind, label, at}]` | On-screen entity timeline (`kind` = person/subtitle/product_image/... `label` = human label) |
| `formula_slots` | `[{at, end, formula}]` | The real Stage A attention formula, one entry per funnel slot. `formula` is a free dict — typical keys: `funnel_stage` (attention/interest/payoff/cta), `function`, `mechanism`, `viewer_state_before`/`viewer_state_after`, `meaning_contract{atten

…(truncated)