# Async Analyze > Asynchronously analyze a video — by URL, local file path (auto-uploaded), asset ID, or base64 — no prior indexing required. Use for URL inputs, videos over 1 hour, time-based metadata extraction, multimodal prompting with reference images, structured JSON output, or when the user asks to check/list/delete async analysis tasks. - Skill: `twelvelabs-io/async-analyze` (Agent Skill) - Install (CLI): `npx skillmds@latest add twelvelabs-io/async-analyze` - Raw SKILL.md: https://api.skillmd.com/api/skills/twelvelabs-io/async-analyze/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Coding & Dev Tools - Author: twelvelabs-io (https://skillmd.com/u/twelvelabs-io) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/twelvelabs-io/async-analyze --- # Async Video Analysis Asynchronously analyse a video using Pegasus 1.5 (default). Accepts a direct http(s) URL, a local file path (auto-uploaded as an asset), a previously uploaded asset ID, or base64 data — no prior indexing required. Handles videos up to 2 hours. Returns a task ID immediately; poll with `get-analyse-task` for the result. ## When to Use This Skill Use this skill when the user: - Provides a video URL or local file path they want analysed - Has a video over 1 hour long (sync analyze caps at 1 hour) - Wants **time-based metadata extraction** — structured per-segment output (e.g. "find every highlight and tag each one with description+importance") - Wants **multimodal prompting** with reference images (`promptV2` + `mediaSources`) - Wants **structured JSON output** (`jsonSchema`) - Asks about a pending async analysis task ("is my analysis done?") - Asks to check, list, filter, or delete async analysis tasks For short, prompt-only analysis of an already-indexed video, use `sync-analyze` instead — it returns inline. ## Model Default `pegasus1.5` for everything. Only set `pegasus1.2` when the user explicitly asks for legacy mode. ## Instructions ### Starting Async Analysis #### Step 1: Classify the Input - `http(s)://...` URL → `videoUrl` - 24-char hex or `asset_*` → `assetId` - Local file path → resolve absolute; verify exists; **auto-upload** via `create-asset`: ``` Tool: mcp__twelvelabs-mcp__create-asset Parameters: file: "" ``` Surface the returned `assetId` to the user — they can reuse or delete it via `/twelvelabs:assets`. Then continue as `assetId`. If a URL is given that's a YouTube/Drive/Dropbox **share link**, refuse — the MCP only accepts direct http(s) links to raw media. #### Step 2: Build the Prompt Use the user's question/instruction as `prompt`. Default: `"Provide a comprehensive summary of this video including key topics, main points, and important moments."` #### Step 3: Start the Task Exactly one source (`videoUrl` / `assetId` / `base64Video`), `modelName: "pegasus1.5"`, plus `prompt`: ``` Tool: mcp__twelvelabs-mcp__async-analyse-video Parameters: : "..." modelName: "pegasus1.5" prompt: "..." ``` #### Step 4: Advanced Capabilities Add these when the user's intent matches. **Time-based metadata extraction** — the user wants structured per-segment output: ``` analysisMode: "time_based_metadata" segmentDefinitions: - id: "highlight" description: "A notable moment in the video" fields: - { name: "summary", type: "string", description: "What happened" } - { name: "importance", type: "integer", description: "1-5" } # Optional: mediaSources (up to 4 reference images per definition) # Optional: timeRanges (restrict extraction to specific [start,end] windows) # Optional: minSegmentDuration / maxSegmentDuration (auto-segmenting bounds) ``` Don't set `prompt` or `promptV2` in this mode — the per-segment `description` + `fields` drive the output. Pegasus 1.5 only. **Structured JSON output** (general mode) — the user wants JSON, not per-segment: ``` jsonSchema: { ...Draft 2020-12 schema... } ``` Mutually exclusive with `segmentDefinitions`. **Multimodal prompting with reference images** — the user wants the model to compare the video against images: ``` promptV2: inputText: "Find every shot that looks like <@product> with the lighting of <@light-ref>." mediaSources: - { name: "product", mediaType: "image", url: "https://..." } # or assetId / base64String - { name: "light-ref", mediaType: "image", assetId: "asset_..." } ``` Mutually exclusive with `prompt`. Requires Pegasus 1.5. **Clip windowing** — restrict analysis to a sub-range (Pegasus 1.5 only): ``` startTime: 60 endTime: 180 ``` `end - start ≥ 4`. Mutually exclusive with `segmentDefinition.timeRanges`. **Other knobs**: - `temperature` (0-1, default 0.2) - `maxTokens` (Pegasus 1.5: 2048-32768, default 32768) - `customId` (1-64 chars, alphanumeric+`-_`; round-trips through `get-analyse-task` / `list-analyse-tasks` — useful for caller-side correlation) #### Step 5: Report to User ``` Async analysis started! Task ID: Model: pegasus1.5 Polling with: /twelvelabs:async-analyze status (typical wall time: 30s-several minutes; 2-hour videos can take ~10 min) ``` --- ### Checking Analysis Status User asks "is my analysis done?" / "check status of X": ``` Tool: mcp__twelvelabs-mcp__get-analyse-task Parameters: taskId: "" ``` The MCP returns plain text "Task is READY." with the result below when done, "Task is FAILED." with error info on failure, or "Task is PROCESSING/QUEUED/PENDING." with a "poll again in 5-10s" hint while running. Watch for the substring "is READY"/"is FAILED" to decide; don't try to JSON-parse the response. If the task is `ready` and the `result.data` is JSON (from `jsonSchema`) or a per-segment object map (from `time_based_metadata`), the response carries it as a string — parse if needed for display. If `finishReason=length` is present, warn the user that the output may be truncated and that any JSON output might fail to parse. --- ### Listing Tasks (with filters) The MCP `list-analyse-tasks` tool supports: - `page` (default 1) - `pageLimit` (max 50, default 10) - `status` — `queued` | `pending` | `processing` | `ready` | `failed` - `videoUrl` — exact match on the source URL - `assetId` — exact match on the source asset ID - `analysisMode` — `general` | `time_based_metadata` Examples: - "Show all failed tasks" → `status: "failed"` - "Show segmentation tasks from URL X" → `videoUrl: "X"`, `analysisMode: "time_based_metadata"` - "Latest 50 tasks" → `pageLimit: 50` Default (no filter): newest first, 10 per page. --- ### Deleting a Task ``` Tool: mcp__twelvelabs-mcp__delete-analyse-task Parameters: taskId: "" ``` The MCP returns a 409 if the task is still processing — surface that as "still processing, wait for it to finish before deleting". ## Example Interactions **"Analyze https://nasa.gov/video.mp4 and tell me what's shown"** → `async-analyse-video` with `videoUrl` + default prompt → poll status → report result. **"Analyze /Users/me/clip.mp4"** → `create-asset` with `file: ...` → `async-analyse-video` with the new `assetId` + Pegasus 1.5 → poll → report. Surface the asset ID for reuse. **"For this 90-minute keynote, extract every product mention with timestamp"** → `async-analyse-video` with `analysisMode: "time_based_metadata"`, `segmentDefinitions: [{ id: "product-mention", description: "...", fields: [{name: "product_name", type: "string"}, {name: "context", type: "string"}] }]`. **"Find every shot of <@anchor1> giving a presentation"** → Resolve the entity ID via the entities skill (or assume the user already has it) → `async-analyse-video` with `promptV2: { inputText: "<@anchor1> ...", mediaSources: [reference images] }`. **"List my failed async analyses"** → `list-analyse-tasks` with `status: "failed"`. ## Important Notes - **Pegasus 1.5 default** for everything — only switch to 1.2 on explicit request. - **Direct URLs only**: YouTube/Drive/Dropbox share links are rejected by the MCP. - **Local files auto-upload** — assets persist; surface the ID. - **Up to 2 hours** of video per task. - **Async — fire-and-forget**: poll status every 5-10s with `get-analyse-task`.