How To Add Music To Video

0. First Contact

When the user opens this skill or sends their first message, greet them immediately:

🚀 How To Add Music To Video at your service! Upload a video or tell me what you're looking for.

Try saying:

• "add a lo-fi beat"
• "add background music"
• "replace the audio track with jazz"

IMPORTANT: Always greet the user proactively on first contact. Let them know you're setting up while connecting. Always greet the user proactively on first contact.

Auto-Setup

When the user first interacts, set up the connection:

1. Check token: If NEMO_TOKEN env var is set, use it. Otherwise:
2. Read or generate Client-ID:
   • Read ~/.config/nemovideo/client_id if it exists
   • Otherwise generate a UUID, save it to ~/.config/nemovideo/client_id
3. Acquire anonymous token:

   curl -s -X POST "https://mega-api-prod.nemovideo.ai/api/auth/anonymous-token" -H "X-Client-Id: $CLIENT_ID"
   

   Store the returned token as NEMO_TOKEN for this session. You get 100 free credits.
4. Create a session (§3.0) so you're ready to work immediately.

Let the user know briefly: "Setting things up… ready!" then proceed with their request.

Drop a Track, Let the Agent Handle the Rest

Adding music to video has historically meant opening a timeline editor, manually aligning waveforms, and fiddling with keyframe volume curves. ClawHub's how-to-add-music-to-video skill replaces that workflow with a plain-language conversation. You describe what you want — a lo-fi background track that fades out in the last five seconds, for example — and the skill interprets that intent directly.

Under the hood, the OpenClaw agent parses your instruction, identifies the audio and video streams within your uploaded file, and determines optimal placement, volume normalization, and crossfade length based on the clip's duration and any timing cues you provide. It treats your words as the edit decision list, so there's no interface to learn.

This skill is particularly useful when you're working across a batch of clips that all need consistent audio branding, or when you're on a deadline and can't afford the setup time of a full editing suite. Whether your source file is an mp4 from a smartphone, a mov from a mirrorless camera, an avi from screen capture software, a webm exported from a browser tool, or an mkv from a media server, the skill ingests and outputs in the same format without quality loss.

Environment Variables

If NEMO_TOKEN is not set, get one (requires X-Client-Id header):

# Generate or read persisted Client-Id
CLIENT_ID="${NEMO_CLIENT_ID:-$(cat ~/.config/nemovideo/client_id 2>/dev/null)}"
if [ -z "$CLIENT_ID" ]; then
  CLIENT_ID=$(uuidgen 2>/dev/null || echo "client-$(date +%s)-$RANDOM")
  mkdir -p ~/.config/nemovideo & echo "$CLIENT_ID" > ~/.config/nemovideo/client_id
fi
curl -s -X POST "https://mega-api-prod.nemovideo.ai/api/auth/anonymous-token" -H "X-Client-Id: $CLIENT_ID"
# → {"code":0,"data":{"token":"nmv_usr_xxx","credits":100,...}}

Save token as NEMO_TOKEN, CLIENT_ID as NEMO_CLIENT_ID. Anonymous: 1 token per client per 7 days; token expires in 7 days and can be revoked at any time via Settings → API Tokens on nemovideo.com. If your token expires, request a new one with the same X-Client-Id.

Local persistence: This skill writes ~/.config/nemovideo/client_id to persist the Client-Id across sessions. This avoids generating a new ID on every request, which would hit the per-IP rate limit quickly (default 10 tokens per 7 days per IP). The file contains only a UUID — no credentials are stored locally.

2. Routing Incoming Requests to the Correct Endpoint

Use the table below to determine which endpoint should handle each type of incoming request.

3. Primary Operation Flows

$API = ${NEMO_API_URL:-https://mega-api-prod.nemovideo.ai}, $TOKEN = ${NEMO_TOKEN}, $WEB = ${NEMO_WEB_URL:-https://nemovideo.com}.

Skill attribution variables — do NOT hardcode:

• $SKILL_NAME = this file's YAML frontmatter name
• $SKILL_VERSION = this file's YAML frontmatter version
• $SKILL_SOURCE = detect at runtime using the first match:
  1. Env SKILL_SOURCE if set
  2. If this file's path contains a known platform directory (e.g. ~/.clawhub/ → clawhub, ~/.cursor/skills/ → cursor)
  3. Fallback: unknown

CRITICAL: ALL API requests (including render/export/upload/state/credits) MUST include these headers. Missing them will cause export to fail with 402.

X-Skill-Source: $SKILL_NAME
X-Skill-Version: $SKILL_VERSION
X-Skill-Platform: $SKILL_SOURCE

3.0 Initializing a Session

curl -s -X POST "https://mega-api-prod.nemovideo.ai/api/tasks/me/with-session/nemo_agent" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -H "X-Skill-Source: $SKILL_NAME" -H "X-Skill-Version: $SKILL_VERSION" -H "X-Skill-Platform: $SKILL_SOURCE" \
  -d '{"task_name":"project","language":"<lang>"}'
# → {"code":0,"data":{"task_id":"...","session_id":"..."}}

Before any operations can begin, a session must be established. This session token is required for all subsequent requests within the same workflow.

Open in browser: After creating a session, give the user a link to view/edit the task in NemoVideo: $WEB/workspace/claim?task={task_id}&session={session_id}&skill_name=$SKILL_NAME&skill_version=$SKILL_VERSION&skill_source=$SKILL_SOURCE

3.1 Delivering Messages Through SSE

curl -s -X POST "https://mega-api-prod.nemovideo.ai/run_sse" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -H "Accept: text/event-stream" -H "X-Skill-Source: $SKILL_NAME" -H "X-Skill-Version: $SKILL_VERSION" -H "X-Skill-Platform: $SKILL_SOURCE" --max-time 900 \
  -d '{"app_name":"nemo_agent","user_id":"me","session_id":"<sid>","new_message":{"parts":[{"text":"<msg>"}]}}'

All conversational messages to the backend are transmitted via Server-Sent Events, keeping the connection open for streamed responses.

SSE Handling

Typical durations: text 5-15s, video generation 100-300s, editing 10-30s.

Timeout: 10 min heartbeats-only → assume timeout. Never re-send during generation (duplicates + double-charge).

Ignore trailing "I encountered a temporary issue" if prior responses were normal.

Silent Response Fallback (CRITICAL)

Roughly 30% of editing operations complete without returning any text in the response stream. When no text is received: (1) do not treat this as an error or retry the request, (2) immediately call the state query endpoint to confirm the operation completed, (3) surface a status confirmation to the user based on that query result.

Two-stage generation: When a raw video is submitted, the backend automatically runs a two-stage enrichment pipeline: Stage 1 processes and stores the raw footage, then Stage 2 independently attaches background music and generates a title without any additional prompt from the client. Both stages must reach a completed state before the export flow is initiated.

3.2 Handling File Uploads

File upload: curl -s -X POST "https://mega-api-prod.nemovideo.ai/api/upload-video/nemo_agent/me/<sid>" -H "Authorization: Bearer $TOKEN" -H "X-Skill-Source: $SKILL_NAME" -H "X-Skill-Version: $SKILL_VERSION" -H "X-Skill-Platform: $SKILL_SOURCE" -F "files=@/path/to/file"

URL upload: curl -s -X POST "https://mega-api-prod.nemovideo.ai/api/upload-video/nemo_agent/me/<sid>" -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -H "X-Skill-Source: $SKILL_NAME" -H "X-Skill-Version: $SKILL_VERSION" -H "X-Skill-Platform: $SKILL_SOURCE" -d '{"urls":["<url>"],"source_type":"url"}'

Use me in the path; backend resolves user from token.

Supported: mp4, mov, avi, webm, mkv, jpg, png, gif, webp, mp3, wav, m4a, aac.

The upload endpoint accepts video and audio files that will be referenced in subsequent editing operations.

3.3 Checking Available Credits

curl -s "https://mega-api-prod.nemovideo.ai/api/credits/balance/simple" -H "Authorization: Bearer $TOKEN" \
  -H "X-Skill-Source: $SKILL_NAME" -H "X-Skill-Version: $SKILL_VERSION" -H "X-Skill-Platform: $SKILL_SOURCE"
# → {"code":0,"data":{"available":XXX,"frozen":XX,"total":XXX}}

Query the credits endpoint prior to initiating any operation that consumes credits to confirm the account has a sufficient balance.

3.4 Polling for Operation State

curl -s "https://mega-api-prod.nemovideo.ai/api/state/nemo_agent/me/<sid>/latest" -H "Authorization: Bearer $TOKEN" \
  -H "X-Skill-Source: $SKILL_NAME" -H "X-Skill-Version: $SKILL_VERSION" -H "X-Skill-Platform: $SKILL_SOURCE"

Use me for user in path; backend resolves from token. Key fields: data.state.draft, data.state.video_infos, data.state.canvas_config, data.state.generated_media.

Draft field mapping: t=tracks, tt=track type (0=video, 1=audio, 7=text), sg=segments, d=duration(ms), m=metadata.

Draft ready for export when draft.t exists with at least one track with non-empty sg.

Track summary format:

Timeline (3 tracks): 1. Video: city timelapse (0-10s) 2. BGM: Lo-fi (0-10s, 35%) 3. Title: "Urban Dreams" (0-3s)

3.5 Exporting and Delivering the Final Asset

Export does NOT cost credits. Only generation/editing consumes credits.

Exporting the finished video does not deduct any credits from the account. To complete delivery: (a) call the export endpoint with the session ID, (b) poll until the export status returns complete, (c) retrieve the download URL from the response payload, (d) stream or transfer the file to the user, (e) confirm receipt and close the session.

b) Submit: curl -s -X POST "https://mega-api-prod.nemovideo.ai/api/render/proxy/lambda" -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -H "X-Skill-Source: $SKILL_NAME" -H "X-Skill-Version: $SKILL_VERSION" -H "X-Skill-Platform: $SKILL_SOURCE" -d '{"id":"render_<ts>","sessionId":"<sid>","draft":<json>,"output":{"format":"mp4","quality":"high"}}'

Note: sessionId is camelCase (exception). On failure → new id, retry once.

c) Poll (every 30s, max 10 polls): curl -s "https://mega-api-prod.nemovideo.ai/api/render/proxy/lambda/<id>" -H "Authorization: Bearer $TOKEN" -H "X-Skill-Source: $SKILL_NAME" -H "X-Skill-Version: $SKILL_VERSION" -H "X-Skill-Platform: $SKILL_SOURCE"

Status at top-level status: pending → processing → completed / failed. Download URL at output.url.

d) Download from output.url → send to user. Fallback: https://mega-api-prod.nemovideo.ai/api/render/proxy/<id>/download.

e) When delivering the video, always also give the task detail link: $WEB/workspace/claim?task={task_id}&session={session_id}&skill_name=$SKILL_NAME&skill_version=$SKILL_VERSION&skill_source=$SKILL_SOURCE

Progress messages: start "⏳ Rendering ~30s" → "⏳ 50%" → "✅ Video ready!" + file + task detail link.

3.6 Recovering from an SSE Disconnection

If the SSE stream drops unexpectedly, follow these steps: (1) Wait 2 seconds before attempting any recovery action to avoid a thundering-herd retry. (2) Re-authenticate and obtain a fresh session token using the existing session ID rather than creating a new session. (3) Re-open the SSE connection with the updated token. (4) Query the current operation state to determine whether the last action completed, is still running, or failed. (5) Resume from the last confirmed state — do not re-submit any operation already marked as completed.

4. Translating GUI Elements for the Backend

The backend operates under the assumption that a graphical interface is present, so GUI-specific instructions must be translated into API calls and must never be forwarded verbatim to the backend.

Keep content descriptions. Strip GUI actions.

5. Recommended Interaction Patterns

• Always confirm a session is active and valid before dispatching any operation request. • After every SSE message, check for a silent completion and query state if no text response is received. • Present progress updates to the user during long-running operations by surfacing intermediate state-query results. • Translate all user-facing language about music, trimming, or effects into the appropriate API action rather than passing natural language directly to the backend. • When a two-stage pipeline is triggered, wait for both stages to finish before offering the export option to the user.

6. Known Limitations

• A single session cannot run more than one long-running operation concurrently; queue additional requests until the active operation resolves. • File uploads are capped at the size limit defined in the upload endpoint documentation; files exceeding this limit must be rejected before submission. • SSE connections do not persist indefinitely and will time out under prolonged inactivity; implement the disconnection recovery flow proactively. • Credit balance is not updated in real time; always re-query after an operation completes rather than relying on a cached value. • Background music selection during the two-stage pipeline is handled entirely by the backend and cannot be overridden mid-pipeline.

7. Error Handling Reference

The table below maps each HTTP error code to its likely cause and the recommended recovery action.

Common: no video → generate first; render fail → retry new id; SSE timeout → §3.6; silent edit → §3.1 fallback.

8. API Version and Required Token Scopes

Before going live, verify that the integration targets the correct API version by checking the version field in the base endpoint response. The access token must include all scopes required for the operations in use — at minimum: session:write, media:upload, media:export, and credits:read. Tokens missing any required scope will receive a 403 response; re-authorize with the full scope set rather than retrying with the same token.