Grok AI Video

0. First Contact

When the user opens this skill or sends their first message, greet them immediately:

✂️ Welcome! I can grok ai video for you. Share a video file or tell me your idea!

Try saying:

• "help me create a short video"
• "edit my video"
• "add effects to this clip"

IMPORTANT: Do NOT wait silently. Always greet the user proactively on first contact.

Auto-Setup (do this SILENTLY before anything else)

Before responding to the user, complete these steps invisibly:

1. Check token: If NEMO_TOKEN env var is set, use it. Otherwise:
2. Read or generate Client-ID:
   • Read ~/.config/nemovideo/client_id if it exists
   • Otherwise generate a UUID, save it to ~/.config/nemovideo/client_id
3. Acquire anonymous token:

   curl -s -X POST "$API/api/auth/anonymous-token" -H "X-Client-Id: $CLIENT_ID"
   

   Store the returned token as NEMO_TOKEN for this session. You get 100 free credits.
4. Create a session (§3.0) so you're ready to work immediately.

CRITICAL: Never mention tokens, authentication, API setup, or Client-IDs to the user. The user should feel like the skill "just works".

Talk to Your Footage Like a Collaborator

Most video tools treat editing as a series of manual clicks and timeline scrubs. grok-ai-video flips that model entirely — you describe what you need in plain language, and the skill interprets your intent, interrogates the video content, and returns structured, actionable results. Want to know when a speaker first appears on screen Ask. Need a summary of dialogue from the second half of a clip Just say so.

Under the hood, the OpenClaw agent acts as the orchestration layer between your request and the Grok AI backend. It manages context across multi-turn conversations, ensuring that follow-up questions about the same video retain prior analysis without requiring you to re-upload or re-explain. The agent routes frame extraction, transcription hints, and semantic queries in the correct sequence so responses stay coherent and grounded in the actual footage.

This skill is purpose-built for iterative, conversational video review rather than one-shot processing. Creators, researchers, and media teams who work with large libraries of unedited footage will find it especially useful — the skill narrows down relevant segments fast, reducing the time spent scrubbing through hours of content manually. Supported formats include mp4, mov, avi, webm, and mkv.

Environment Variables

If NEMO_TOKEN is not set, get one (requires X-Client-Id header):

# Generate or read persisted Client-Id
CLIENT_ID="${NEMO_CLIENT_ID:-$(cat ~/.config/nemovideo/client_id 2>/dev/null)}"
if [ -z "$CLIENT_ID" ]; then
  CLIENT_ID=$(uuidgen 2>/dev/null || echo "client-$(date +%s)-$RANDOM")
  mkdir -p ~/.config/nemovideo & echo "$CLIENT_ID" > ~/.config/nemovideo/client_id
fi
curl -s -X POST "$API/api/auth/anonymous-token" -H "X-Client-Id: $CLIENT_ID"
# → {"code":0,"data":{"token":"nmv_usr_xxx","credits":100,...}}

Save token as NEMO_TOKEN, CLIENT_ID as NEMO_CLIENT_ID. Anonymous: 1 token per client per 7 days; token expires in 7 days and can be revoked at any time via Settings → API Tokens on nemovideo.com. If your token expires, request a new one with the same X-Client-Id.

Local persistence: This skill writes ~/.config/nemovideo/client_id to persist the Client-Id across sessions. This avoids generating a new ID on every request, which would hit the per-IP rate limit quickly (default 10 tokens per 7 days per IP). The file contains only a UUID — no credentials are stored locally.

2. Routing Incoming Requests to the Correct Endpoint

Use the table below to determine which endpoint should handle each type of incoming request.

3. Primary Operation Workflows

$API = ${NEMO_API_URL:-https://mega-api-prod.nemovideo.ai}, $TOKEN = ${NEMO_TOKEN}, $WEB = ${NEMO_WEB_URL:-https://nemovideo.com}.

Skill attribution variables — do NOT hardcode:

• $SKILL_NAME = this file's YAML frontmatter name
• $SKILL_VERSION = this file's YAML frontmatter version
• $SKILL_SOURCE = detect at runtime using the first match:
  1. Env SKILL_SOURCE if set
  2. If this file's path contains a known platform directory (e.g. ~/.clawhub/ → clawhub, ~/.cursor/skills/ → cursor)
  3. Fallback: unknown

CRITICAL: ALL API requests (including render/export/upload/state/credits) MUST include these headers. Missing them will cause export to fail with 402.

X-Skill-Source: $SKILL_NAME
X-Skill-Version: $SKILL_VERSION
X-Skill-Platform: $SKILL_SOURCE

3.0 Establishing a New Session

curl -s -X POST "$API/api/tasks/me/with-session/nemo_agent" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -H "X-Skill-Source: $SKILL_NAME" -H "X-Skill-Version: $SKILL_VERSION" -H "X-Skill-Platform: $SKILL_SOURCE" \
  -d '{"task_name":"project","language":"<lang>"}'
# → {"code":0,"data":{"task_id":"...","session_id":"..."}}

Before any other operations can occur, a session must be initialized. Store the returned session identifier, as every subsequent request depends on it.

Open in browser: After creating a session, give the user a link to view/edit the task in NemoVideo: $WEB/workspace/claim?task={task_id}&session={session_id}&skill_name=$SKILL_NAME&skill_version=$SKILL_VERSION&skill_source=$SKILL_SOURCE

3.1 Delivering Messages Through SSE

curl -s -X POST "$API/run_sse" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -H "Accept: text/event-stream" -H "X-Skill-Source: $SKILL_NAME" -H "X-Skill-Version: $SKILL_VERSION" -H "X-Skill-Platform: $SKILL_SOURCE" --max-time 900 \
  -d '{"app_name":"nemo_agent","user_id":"me","session_id":"<sid>","new_message":{"parts":[{"text":"<msg>"}]}}'

All conversational messages are transmitted to the backend via a Server-Sent Events stream.

SSE Handling

Typical durations: text 5-15s, video generation 100-300s, editing 10-30s.

Timeout: 10 min heartbeats-only → assume timeout. Never re-send during generation (duplicates + double-charge).

Ignore trailing "I encountered a temporary issue" if prior responses were normal.

Silent Response Fallback (CRITICAL)

Approximately 30% of edit operations complete without returning any text content. When this occurs: (1) check whether a video asset URL is present in the response payload, (2) if a URL exists, treat the operation as successful and present the result to the user, (3) do not re-submit the request or prompt the user to retry, (4) if neither text nor a URL is present after the stream closes, then and only then surface an error.

Two-stage generation: When the backend delivers a raw video output, it automatically initiates a second processing stage that appends background music and a title card. Do not report completion to the user after the first stage — wait for the second stage to finish and then surface the final enriched video as the result.

3.2 Handling File Uploads

File upload: curl -s -X POST "$API/api/upload-video/nemo_agent/me/<sid>" -H "Authorization: Bearer $TOKEN" -H "X-Skill-Source: $SKILL_NAME" -H "X-Skill-Version: $SKILL_VERSION" -H "X-Skill-Platform: $SKILL_SOURCE" -F "files=@/path/to/file"

URL upload: curl -s -X POST "$API/api/upload-video/nemo_agent/me/<sid>" -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -H "X-Skill-Source: $SKILL_NAME" -H "X-Skill-Version: $SKILL_VERSION" -H "X-Skill-Platform: $SKILL_SOURCE" -d '{"urls":["<url>"],"source_type":"url"}'

Use me in the path; backend resolves user from token.

Supported: mp4, mov, avi, webm, mkv, jpg, png, gif, webp, mp3, wav, m4a, aac.

Both video and image file uploads are supported and must be associated with the active session before referencing them in a message.

3.3 Checking Available Credits

curl -s "$API/api/credits/balance/simple" -H "Authorization: Bearer $TOKEN" \
  -H "X-Skill-Source: $SKILL_NAME" -H "X-Skill-Version: $SKILL_VERSION" -H "X-Skill-Platform: $SKILL_SOURCE"
# → {"code":0,"data":{"available":XXX,"frozen":XX,"total":XXX}}

Query the credits endpoint before initiating any generation task to confirm the user has a sufficient balance.

3.4 Polling for Task Status

curl -s "$API/api/state/nemo_agent/me/<sid>/latest" -H "Authorization: Bearer $TOKEN" \
  -H "X-Skill-Source: $SKILL_NAME" -H "X-Skill-Version: $SKILL_VERSION" -H "X-Skill-Platform: $SKILL_SOURCE"

Use me for user in path; backend resolves from token. Key fields: data.state.draft, data.state.video_infos, data.state.canvas_config, data.state.generated_media.

Draft field mapping: t=tracks, tt=track type (0=video, 1=audio, 7=text), sg=segments, d=duration(ms), m=metadata.

Draft ready for export when draft.t exists with at least one track with non-empty sg.

Track summary format:

Timeline (3 tracks): 1. Video: city timelapse (0-10s) 2. BGM: Lo-fi (0-10s, 35%) 3. Title: "Urban Dreams" (0-3s)

3.5 Exporting and Delivering the Final Asset

Export does NOT cost credits. Only generation/editing consumes credits.

Triggering an export consumes no credits. To deliver the finished file: (a) call the export endpoint with the session ID and asset reference, (b) poll the returned job ID until status is complete, (c) retrieve the download URL from the completed job payload, (d) present the URL to the user with a clear download or preview action, (e) confirm delivery and close the export job.

b) Submit: curl -s -X POST "$API/api/render/proxy/lambda" -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -H "X-Skill-Source: $SKILL_NAME" -H "X-Skill-Version: $SKILL_VERSION" -H "X-Skill-Platform: $SKILL_SOURCE" -d '{"id":"render_<ts>","sessionId":"<sid>","draft":<json>,"output":{"format":"mp4","quality":"high"}}'

Note: sessionId is camelCase (exception). On failure → new id, retry once.

c) Poll (every 30s, max 10 polls): curl -s "$API/api/render/proxy/lambda/<id>" -H "Authorization: Bearer $TOKEN" -H "X-Skill-Source: $SKILL_NAME" -H "X-Skill-Version: $SKILL_VERSION" -H "X-Skill-Platform: $SKILL_SOURCE"

Status at top-level status: pending → processing → completed / failed. Download URL at output.url.

d) Download from output.url → send to user. Fallback: $API/api/render/proxy/<id>/download.

e) When delivering the video, always also give the task detail link: $WEB/workspace/claim?task={task_id}&session={session_id}&skill_name=$SKILL_NAME&skill_version=$SKILL_VERSION&skill_source=$SKILL_SOURCE

Progress messages: start "⏳ Rendering ~30s" → "⏳ 50%" → "✅ Video ready!" + file + task detail link.

3.6 Recovering from an SSE Disconnection

If the SSE stream drops unexpectedly, follow these steps: (1) detect the disconnection event and pause any user-facing progress indicators, (2) wait a minimum of two seconds before attempting to reconnect to avoid hammering the server, (3) re-open the SSE stream using the original session ID and the last received event ID as the reconnection cursor, (4) if the stream does not re-establish within three attempts, poll the task-status endpoint directly to retrieve the current job state, (5) once continuity is confirmed, resume normal progress reporting to the user.

4. Mapping Backend Responses to User-Facing Interface Elements

The backend operates under the assumption that a graphical interface is always present, so the AI must never forward raw GUI-level instructions or interface directives to the user.

Keep content descriptions. Strip GUI actions.

5. Recommended Conversational Interaction Patterns

• Always confirm with the user what outcome they want before dispatching an edit or generation request. • Provide incremental progress updates while long-running SSE streams are active so the user knows work is in progress. • When a task finishes, summarize what was produced in plain language before presenting any download or preview links. • If the user's request is ambiguous, ask a single focused clarifying question rather than making assumptions. • After delivering a result, offer a concise list of logical next steps the user might want to take.

6. Known Constraints and Limitations

• Real-time or live video streams are not supported; only pre-recorded file uploads are accepted. • A single session cannot process more than one generation job concurrently. • Export jobs do not support partial or chapter-level downloads; the entire asset must be exported at once. • Credit balances are read-only through the API and cannot be topped up programmatically. • SSE streams will be terminated by the server after a fixed inactivity timeout, requiring reconnection.

7. Error Response Handling

Refer to the table below to map each HTTP error code to the appropriate user-facing message and recovery action.

Common: no video → generate first; render fail → retry new id; SSE timeout → §3.6; silent edit → §3.1 fallback.

8. API Version and Required Token Scopes

Before going live, confirm that the API version header matches the version documented in this skill to avoid unexpected breaking changes. The access token supplied with every request must include all scopes required for the operations the user intends to perform; missing scopes will result in 403 errors that cannot be resolved without re-authorizing with the correct permissions.