# Atai Manual Generation Agent > Run Archetype AI's managed Manual Generation (MGA) agent over the Agents API — upload a procedure video, create a bundle from the canonical `mga` blueprint, run it, poll, download an ordered, timestamped manual. Use when the user has a recording of a procedure (a repair, an assembly, a workflow) and wants the platform to turn it into step-by-step instructions traceable back to the video. Covers video suitability (the ~5-minute ceiling, the audio-track requirement), the bundle request shape (`max_frames`, `max_new_tokens`, and a `prompt` that must say what to cover and never how to format), the run/poll/results lifecycle, the output JSON schema (`step, instruction, frame_start/end, timestamp_start/end`), and scoring against reference annotations. Do NOT use for verifying a task was performed correctly (the `tva` blueprint), for one-shot multimodal questions over a clip (`atai-newton-fusion-model`), or for time-series state classification (`atai-operational-state-monitoring-agent`). - Skill: `archetypeai/atai-manual-generation-agent` (Agent Skill, multi-file: 12 files) - Install (CLI): `npx skillmds@latest add archetypeai/atai-manual-generation-agent` - Raw SKILL.md: https://api.skillmd.com/api/skills/archetypeai/atai-manual-generation-agent/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: AI & ML - Author: archetypeai (https://skillmd.com/u/archetypeai) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/archetypeai/atai-manual-generation-agent --- # MGA Agent — Managed Manual Generation via the Agents API The MGA agent turns a procedure video into an ordered manual with timestamps. You hand the platform an `.mp4`; it samples frames, transcribes the audio, fuses both in one pass, and returns steps you can trace back to the recording: ``` video ─► sample frames ─┐ ├─► newton-fusion f1-0 ─► parse ─► steps + timestamps whisper ASR ────┘ ``` MGA V1 is **zero-shot**. The `mga` blueprint pins its own models (`newton-fusion:1.0` and `whisper:large-v3`), so unlike `osm`/`red` there is **no classifier to fit and no `artifacts` map to pass**. A run is upload → bundle → run → download. > **Availability.** The canonical `mga` blueprint resolves by key on the > production deployment (`https://api.u1.archetypeai.app`) — set > `ATAI_API_ENDPOINT` to it and the full upload → bundle → run → score cycle > works as documented here. The same 173 s video reproduces an **identical > manual** (18/18 instruction texts and timestamps) run to run, at the same > ~15 min runtime. A verified run: job time **879 s** — 13 s queued, whisper > 31.9 s download + 3.2 s load, newton-fusion **4min55s download + 1min39s > load**, then 444 s to process a 173 s video (~2.6× realtime). If the > blueprint key doesn't resolve, contact support@archetypeai.dev. ## When to Apply **Use when** the user wants a written procedure extracted from a recording — a maintenance manual from a repair video, an SOP from a screen capture, work instructions from an assembly clip — and wants each step tied to a time range in the source. **Do NOT use when:** | Need | Use instead | |---|---| | Was this task done correctly, against a known procedure? | the `tva` (Task Verification) blueprint | | One question about a clip, stateless | `atai-newton-fusion-model` (`/query`) | | Classify sensor windows into operational states | `atai-operational-state-monitoring-agent` | | The video is longer than ~5 minutes | chunk it first — see Step 1 | ## Endpoints The Agents API is mounted **without a version prefix**; the files API is under `/v0.5`. ``` POST {endpoint}/v0.5/files upload the video POST {endpoint}/agents/bundles create a bundle ← PLURAL POST {endpoint}/agents/bundles/{bundle_id}/run start a run ← returns 202 GET {endpoint}/agents/instances/{agent_id}/logs the real log stream GET {endpoint}/agents/instances/{agent_id}/results output refs POST {endpoint}/agents/instances/{agent_id}/cancel stop a run ``` `POST /agents/bundle` (singular) returns **404**. Some older clients still use it. ## ⚠️ Two values decide whether you get a manual — read before anything else Both are settable today. Both fail silently when they are wrong. | Send | Why | |---|---| | `max_new_tokens: 16384` (the blueprint default) | Shared with the model's reasoning block. Run out inside it and you get an empty manual, reported as success — check the manual is non-empty. No budget in 2048–65536 changed the output on 2026-08-20, so send the default unless you have a reason not to. | | a `prompt` saying what to **cover** | Omit it and the blueprint's own instruction applies, halving the manual. Never specify an output *format* — the parser owns the template and returns zero steps. | And **check that they are honoured**, rather than assuming. A value is real only if it is declared in the blueprint's `values` *and* referenced as `${values.}` by some node or connector. Nothing in the response tells you: an unwired value returns **HTTP 201** and is echoed straight back at you in `values`. ```python doc = GET(f"{endpoint}/agents/blueprints/{blueprint}")["document"] wired = json.dumps({"nodes": doc["nodes"], "connectors": doc["connectors"]}) inert = [k for k in my_values if k not in doc["values"] or f"${{values.{k}}}" not in wired] ``` `references/run_mga_agent.py` runs that before every run and prints both halves, so silence never has to be read as success: ``` blueprint blp_6va7xxnrrn9fr8ybhgz3v3zvw7 (key=mga, active=True) honoured: ['max_frames', 'max_new_tokens', 'prompt'] ``` Run the check every time. The `mga` blueprint is republished often — three versions in one week — and each republication can change which values are wired. When one of these two stops being honoured, the run still succeeds and the manual quietly gets worse, so the preflight is the only thing standing between you and a plausible bad result. ### `max_new_tokens` and the reasoning block The model **reasons before it answers**, and pays for the reasoning out of this same budget. If generation ends inside the reasoning block it never emits an answer, and every signal says success: `job.completed`, no ERROR row, every HTTP call 2xx, and a 38-byte output containing `{"id": "…", "results": []}`. **Check that the manual is non-empty before trusting a run** — that check is the one thing here that never goes stale. **What the budget actually changed, measured 2026-08-20 on Prod.** One 173 s video (`tire_i2JWkDyg26A`, the source of the reference outputs in `sample_data/`), five budgets, everything else held constant: | `max_new_tokens` | 2048 | 4096 | 16384 | 32768 | 65536 | |---|---|---|---|---|---| | steps | **18** | **18** | **18** | **18** | **18** | **Nothing about the budget mattered.** Every run produced the same 18-step manual — **18/18 instructions and 18/18 timestamps identical** to the 16384 run at every budget, across a 32× range. No empty manual, no reasoning-block WARN. Reproduce any cell with `--max-new-tokens `. Two caveats on that row. The five outputs are the same length (3214 bytes) but have **five different md5s**, so generation is deterministic in *content*, not in bytes — don't diff by hash. And a 173 s video with an 18-step manual is one point in a space: reasoning cost scales with how much there is to describe, so a longer or busier video may behave differently. Send the blueprint default unless you have a reason not to; the evidence for preferring any particular value is gone. #### `results: []` — plan for it, do not predict it It is a real outcome and you should handle it, but **there is no threshold to memorise.** Whether a run lands in it depends on how much reasoning the model does before answering — an interaction of the video, the prompt, and the budget, not the budget alone. It was observed at 2048 and 4096 in Aug 2026 on this very video; on 2026-08-20 the same two budgets produced full manuals. Assume it can happen on any combination you have not tried, including one that worked before. Detect it explicitly: the output must contain steps, and `/logs` is where the cause appears. The platform logs a WARN for exactly this case, and it is the only thing that makes it diagnosable: ``` WARN parser.running ManualGenerationResultsParserNode: generation ended inside the model's reasoning block, so it never produced an answer; dropping 0 rehearsed row(s). Raise `max_new_tokens` if the reasoning was truncated. ``` `dropping N` is the useful number: `N > 0` means an answer was formed and discarded, so more budget was not the missing ingredient. Note the WARN's own advice — raise the budget — is the platform's generic suggestion and is not always right; the `atai-task-verification-agent` sibling has observed the opposite direction, where a larger budget produced the empty output and a smaller one worked.
Earlier measurement (2026-08-13) — dated, and did not reproduce This skill previously measured a hard floor on the same 173 s video and prescribed the blueprint default: | `max_new_tokens` | 2048 | 4096 | **16384** | 32768 | 65536 | |---|---|---|---|---|---| | steps | 0 | 0 | **18** | 18 | 18 | `sample_data/mga-output-current-4096-EMPTY.json` is a captured artifact of that failure at 4096 — kept because nothing else shows what an empty manual looks like, not because it reproduces. At 4096 today, the same video returned the full 18-step manual. The `atai-task-verification-agent` sibling documents the **opposite** rule — that 5760 works and 8192/16384 return nothing — and that did not reproduce either. Two blueprints, opposite prescriptions, neither observable now; the shared reasoning behaviour is the likely variable rather than anything specific to either blueprint. **Non-reproduction is not retirement.** The failure was observed, and the platform still emits the WARN above for exactly this case. Only the numbers are unreliable.
### What a caller-supplied `prompt` is worth Same clip, same `max_frames`, only the instruction differing — the blueprint's hardcoded "10 steps or less" against a coverage instruction: | | hardcoded | coverage prompt | |---|---|---| | steps | 10 | **18** | | actions bundled per step | 3.2 avg, 7 max | **1.7 avg, 4 max** | | ref steps at IoU ≥ 0.5 | 4/11 | **7/11** | | ref steps at IoU ≥ 0.3 | **10/11** | 9/11 | | spoken cautions as their own step | 0 | 1 | **Judge a manual by whether it can be followed**, and note that the last two rows disagree with that judgement. At the *loose* threshold the 10-step version scores **higher**, because MGA tiles the timeline contiguously so fewer steps means longer ones that overlap a reference interval more easily. Coarser segmentation flatters loose-threshold recall while being worse to work from: an operator cannot tick off "apply the parking brake" separately from "switch off the ignition", and each step carries one frame thumbnail however many actions it contains. Where the remaining error sits is worth checking per step rather than in aggregate. On the current run the four scores below 0.4 have **two different causes, pulling in opposite directions**: - **Under-segmentation** — two reference steps collapsing onto one prediction: remove-the-nuts with remove-the-wheel (steps 5–6, IoU 0.27 and 0.26) and tighten with put-things-back (step 10, 0.34). - **Over-segmentation** — the opening reference step (14.8–28.9 s, "stop the car") spread across three predictions: stop / warning triangle / parking brake, so no single prediction covers more than a third of it (step 1, 0.35). Every other step scored 0.52–0.69. Both are prompt-shaped rather than a model ceiling, but they want **opposite** instructions — asking for the collapsed actions as separate steps risks splitting the opening further, and asking for coarser opening steps risks re-collapsing the middle. Untested, and worth testing as one change rather than two. ## Step 1 — Choose a video The platform validates none of this for you, and getting it wrong costs ~15 minutes per attempt. | Requirement | Why | |---|---| | **Under ~5 minutes** | Longer videos fail outright. A 563 s clip ran 5 min 13 s then died: `pipeline execution failed: Newton generation event stream closed without a terminal event`, exit 1. | | **Has an audio track** | `AudioReaderNode` → whisper → `prompt.audio` is half the pipeline. Spoken-only content (safety cautions) shows up in the output, so a silent video loses real information. | | **Reference steps, to score it** | Otherwise there is nothing to evaluate against. | The ~5-minute ceiling is **not** a context limit: at `max_frames: 16` the video occupies 392 visual tokens, ~0.15% of F1-0's 256K window. Source resolution barely matters — every frame is resized to `size × size` (default 224), so 360p is no worse than 1080p, only smaller and faster to upload. ### Longer videos: chunk them Chunking is also the only way to get more than ~10 steps out of a long procedure, since the step ceiling is per-job (see above). What has worked in production: - **90 s chunks with 15 s overlap.** 90 s keeps each job well inside the failure band; the overlap is there because a step that straddles a cut is otherwise described from half its evidence in both chunks. - **Cut on keyframes.** `ffmpeg -ss -i in.mp4 -c copy` rewinds to the previous keyframe and silently shifts the segment — measured **−7.57 s** of drift on a real file. Read the keyframe timestamps first (`ffprobe -select_streams v -skip_frame nokey -show_entries frame=pts_time`) and snap each cut to one, or re-encode. - **Step from each chunk's actual start, not from `i × stride`.** Snapping to keyframes moves the boundaries; a fixed grid then eats the overlap (15 s → 10 s on our first attempt). - **One run per chunk, or one run with `source` as an array.** The array form loads the model once instead of per chunk, which is the dominant cost — but results come back **out of order** and every timestamp is **chunk-local**. Tag each input and add the chunk's start offset before merging, or the manual is silently scrambled. - **Dedupe the overlap conservatively.** Two steps may be merged only if they came from *different* chunks *and* their time spans actually overlap. A similarity-only rule deletes real repeated actions — three genuine "Click on the OK button" steps collapsed into one before we added the time test. A `pipeline execution failed: Newton generation event stream closed without a terminal event` at 138–175 s of runtime, on inputs that are well under the ceiling, is a **platform-side** failure and not something to fix by shortening further — retry the chunk. ## Step 2 — Upload the video ```python POST {endpoint}/v0.5/files # multipart/form-data, field name "file" → {"is_valid": true, "file_id": "clip.mp4", "file_uid": "fil_…"} ``` The **declared** `Content-Type` is enforced against a MIME allowlist, not the bytes — send `video/mp4`. Use `file_id` (the filename) in the run payload, not `file_uid`. ## Step 3 — Create a bundle **No `artifacts` map** — the blueprint pins its own models. This is the main shape difference from `osm`/`red`. **Send a `prompt`.** Omit it and the blueprint's own instruction applies, which on current versions has read *"…with 10 steps or less"* — halving the manual for a reason nothing in the response reveals. ```json POST {endpoint}/agents/bundles { "blueprint": "mga", "name": "mga tire_i2JWkDyg26A mnt16384", "values": { "max_frames": 64, "max_new_tokens": 16384, "prompt": "Generate a concise, ordered list of every distinct step performed in this video, covering the procedure from the first action to the last. Include brief steps and steps that are repeated. Use up to 20 steps. Keep each step to at most 15 words. Use both what is shown and what is said." } } → 201 {"id": "bnd_…", "status": "ready", "is_canonical": false} ``` > **Name the bundle after its inputs.** A bundle's `name` is set at creation and > **cannot be changed** — `PATCH` and `PUT` on `/agents/bundles/{id}` both return > **405** — and it is the only thing distinguishing your runs from each other in > the console. Reuse one constant name across several runs and every row looks > identical, leaving you to recover which bundle was which from your own local > records. `run_mga_agent.py` defaults `--name` to `mga