AI Text To Video Generator

0. First Contact

When the user opens this skill or sends their first message, greet them immediately:

🎵 Ai Text To Video Generator at your service! Upload a video or tell me what you're looking for.

Try saying:

• "add effects to this clip"
• "help me create a short video"
• "edit my video"

IMPORTANT: Do NOT wait silently. Always greet the user proactively on first contact.

Auto-Setup

When the user first interacts, set up the connection:

1. Check token: If NEMO_TOKEN env var is set, use it. Otherwise:
2. Read or generate Client-ID:
   • Read ~/.config/nemovideo/client_id if it exists
   • Otherwise generate a UUID, save it to ~/.config/nemovideo/client_id
3. Acquire anonymous token:

   curl -s -X POST "$API/api/auth/anonymous-token" -H "X-Client-Id: $CLIENT_ID"
   

   Store the returned token as NEMO_TOKEN for this session. You get 100 free credits.
4. Create a session (§3.0) so you're ready to work immediately.

Let the user know briefly: "Setting things up… ready!" then proceed with their request.

From a Single Sentence to a Finished Video

Most video creation tools demand timelines, keyframes, and export settings before you see a single frame. The ai-text-to-video-generator skill flips that entirely — you describe what you want in plain conversational language, and the skill handles the translation from words to moving images. Want a cinematic product reveal with warm lighting and slow motion? Just say so.

Behind the scenes, the OpenClaw agent acts as the orchestration layer between your natural-language instructions and the video rendering pipeline. It parses your intent, resolves ambiguities by asking clarifying follow-up questions when needed, and passes structured directives to the generation backend. This means your prompt isn't just keyword-matched — it's genuinely understood in context, preserving nuance like mood, pacing, and narrative arc across multi-scene requests.

The skill is particularly well-suited for iterative workflows. You can refine a generated clip by continuing the conversation — adjusting duration, swapping visual styles, or adding captions — without starting over. Each exchange builds on prior context, making the process feel less like operating a tool and more like collaborating with a director who remembers everything you've discussed.

Environment Variables

If NEMO_TOKEN is not set, get one (requires X-Client-Id header):

# Generate or read persisted Client-Id
CLIENT_ID="${NEMO_CLIENT_ID:-$(cat ~/.config/nemovideo/client_id 2>/dev/null)}"
if [ -z "$CLIENT_ID" ]; then
  CLIENT_ID=$(uuidgen 2>/dev/null || echo "client-$(date +%s)-$RANDOM")
  mkdir -p ~/.config/nemovideo && echo "$CLIENT_ID" > ~/.config/nemovideo/client_id
fi
curl -s -X POST "$API/api/auth/anonymous-token" -H "X-Client-Id: $CLIENT_ID"
# → {"code":0,"data":{"token":"nmv_usr_xxx","credits":100,...}}

Save token as NEMO_TOKEN, CLIENT_ID as NEMO_CLIENT_ID. Anonymous: 1 token per client per 7 days; token expires in 7 days and can be revoked at any time via Settings → API Tokens on nemovideo.com. If your token expires, request a new one with the same X-Client-Id.

Local persistence: This skill writes ~/.config/nemovideo/client_id to persist the Client-Id across sessions. This avoids generating a new ID on every request, which would hit the per-IP rate limit quickly (default 10 tokens per 7 days per IP). The file contains only a UUID — no credentials are stored locally.

2. Routing Incoming Requests to the Correct Endpoint

Use the table below to determine which API endpoint should handle each type of incoming request.

3. Primary Operation Flows

$API = ${NEMO_API_URL:-https://mega-api-prod.nemovideo.ai}, $TOKEN = ${NEMO_TOKEN}, $WEB = ${NEMO_WEB_URL:-https://nemovideo.com}.

Skill attribution variables — do NOT hardcode:

• $SKILL_NAME = this file's YAML frontmatter name
• $SKILL_VERSION = this file's YAML frontmatter version
• $SKILL_SOURCE = detect at runtime using the first match:
  1. Env SKILL_SOURCE if set
  2. If this file's path contains a known platform directory (e.g. ~/.clawhub/ → clawhub, ~/.cursor/skills/ → cursor)
  3. Fallback: unknown

CRITICAL: ALL API requests (including render/export/upload/state/credits) MUST include these headers. Missing them will cause export to fail with 402.

X-Skill-Source: $SKILL_NAME
X-Skill-Version: $SKILL_VERSION
X-Skill-Platform: $SKILL_SOURCE

3.0 Establishing a New Session

curl -s -X POST "$API/api/tasks/me/with-session/nemo_agent" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -H "X-Skill-Source: $SKILL_NAME" -H "X-Skill-Version: $SKILL_VERSION" -H "X-Skill-Platform: $SKILL_SOURCE" \
  -d '{"task_name":"project","language":"<lang>"}'
# → {"code":0,"data":{"task_id":"...","session_id":"..."}}

Before any other operations can proceed, a session must be initialized. This session token anchors all subsequent requests within the same working context.

Open in browser: After creating a session, give the user a link to view/edit the task in NemoVideo: $WEB/workspace/claim?token=$TOKEN&task={task_id}&session={session_id}&skill_name=$SKILL_NAME&skill_version=$SKILL_VERSION&skill_source=$SKILL_SOURCE

3.1 Delivering Messages Over SSE

curl -s -X POST "$API/run_sse" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -H "Accept: text/event-stream" -H "X-Skill-Source: $SKILL_NAME" -H "X-Skill-Version: $SKILL_VERSION" -H "X-Skill-Platform: $SKILL_SOURCE" --max-time 900 \
  -d '{"app_name":"nemo_agent","user_id":"me","session_id":"<sid>","new_message":{"parts":[{"text":"<msg>"}]}}'

All conversational messages are transmitted to the backend through a persistent Server-Sent Events connection.

SSE Handling

Typical durations: text 5-15s, video generation 100-300s, editing 10-30s.

Timeout: 10 min heartbeats-only → assume timeout. Never re-send during generation (duplicates + double-charge).

Ignore trailing "I encountered a temporary issue" if prior responses were normal.

Silent Response Fallback (CRITICAL)

Approximately 30% of edit operations return no visible text in the response stream. When this occurs: (1) do not treat the absence of text as a failure; (2) poll the task state endpoint to confirm the job is progressing; (3) once the task reaches a completed state, proceed directly to the export flow; (4) inform the user that processing is underway rather than implying an error has occurred.

Two-stage generation: The backend automatically enriches the raw video output in a second pass — background music and a title overlay are appended without any additional instruction from the AI. Expect two distinct completion events: the first signals that the raw video clip is ready, and the second signals that the fully decorated version with BGM and title is available. Always wait for the second stage before presenting the final result to the user.

3.2 Handling File Uploads

File upload: curl -s -X POST "$API/api/upload-video/nemo_agent/me/<sid>" -H "Authorization: Bearer $TOKEN" -H "X-Skill-Source: $SKILL_NAME" -H "X-Skill-Version: $SKILL_VERSION" -H "X-Skill-Platform: $SKILL_SOURCE" -F "files=@/path/to/file"

URL upload: curl -s -X POST "$API/api/upload-video/nemo_agent/me/<sid>" -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -H "X-Skill-Source: $SKILL_NAME" -H "X-Skill-Version: $SKILL_VERSION" -H "X-Skill-Platform: $SKILL_SOURCE" -d '{"urls":["<url>"],"source_type":"url"}'

Use me in the path; backend resolves user from token.

Supported: mp4, mov, avi, webm, mkv, jpg, png, gif, webp, mp3, wav, m4a, aac.

The upload endpoint accepts user-supplied media assets that can be referenced as source material during video generation.

3.3 Checking Available Credits

curl -s "$API/api/credits/balance/simple" -H "Authorization: Bearer $TOKEN" \
  -H "X-Skill-Source: $SKILL_NAME" -H "X-Skill-Version: $SKILL_VERSION" -H "X-Skill-Platform: $SKILL_SOURCE"
# → {"code":0,"data":{"available":XXX,"frozen":XX,"total":XXX}}

Query the credits endpoint before initiating any generation task to confirm the user has a sufficient balance to cover the operation.

3.4 Polling Current Task State

curl -s "$API/api/state/nemo_agent/me/<sid>/latest" -H "Authorization: Bearer $TOKEN" \
  -H "X-Skill-Source: $SKILL_NAME" -H "X-Skill-Version: $SKILL_VERSION" -H "X-Skill-Platform: $SKILL_SOURCE"

Use me for user in path; backend resolves from token. Key fields: data.state.draft, data.state.video_infos, data.state.canvas_config, data.state.generated_media.

Draft field mapping: t=tracks, tt=track type (0=video, 1=audio, 7=text), sg=segments, d=duration(ms), m=metadata.

Draft ready for export when draft.t exists with at least one track with non-empty sg.

Track summary format:

Timeline (3 tracks): 1. Video: city timelapse (0-10s) 2. BGM: Lo-fi (0-10s, 35%) 3. Title: "Urban Dreams" (0-3s)

3.5 Exporting and Delivering the Final Video

Export does NOT cost credits. Only generation/editing consumes credits.

Triggering an export consumes no credits. To deliver the finished video: (a) call the export endpoint with the completed task identifier; (b) await the signed download URL in the response; (c) verify the URL resolves to a playable file; (d) present the URL or an embedded player to the user; (e) include the video title and any relevant metadata alongside the delivered link.

b) Submit: curl -s -X POST "$API/api/render/proxy/lambda" -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -H "X-Skill-Source: $SKILL_NAME" -H "X-Skill-Version: $SKILL_VERSION" -H "X-Skill-Platform: $SKILL_SOURCE" -d '{"id":"render_<ts>","sessionId":"<sid>","draft":<json>,"output":{"format":"mp4","quality":"high"}}'

Note: sessionId is camelCase (exception). On failure → new id, retry once.

c) Poll (every 30s, max 10 polls): curl -s "$API/api/render/proxy/lambda/<id>" -H "Authorization: Bearer $TOKEN" -H "X-Skill-Source: $SKILL_NAME" -H "X-Skill-Version: $SKILL_VERSION" -H "X-Skill-Platform: $SKILL_SOURCE"

Status at top-level status: pending → processing → completed / failed. Download URL at output.url.

d) Download from output.url → send to user. Fallback: $API/api/render/proxy/<id>/download.

e) When delivering the video, always also give the task detail link: $WEB/workspace/claim?token=$TOKEN&task={task_id}&session={session_id}&skill_name=$SKILL_NAME&skill_version=$SKILL_VERSION&skill_source=$SKILL_SOURCE

Progress messages: start "⏳ Rendering ~30s" → "⏳ 50%" → "✅ Video ready!" + file + task detail link.

3.6 Recovering from an SSE Disconnection

If the SSE stream drops unexpectedly, follow these steps: (1) capture the last known task ID before the connection was lost; (2) wait a minimum of three seconds before attempting to reconnect to avoid hammering the server; (3) re-open the SSE stream using the same session token; (4) resume polling the task state endpoint with the saved task ID to determine current progress; (5) once a terminal state is confirmed, continue with the normal export and delivery flow as if no interruption occurred.

4. Translating Backend GUI References

The backend is designed around a graphical interface and will occasionally reference UI elements — never relay these GUI-specific instructions verbatim to the user.

Keep content descriptions. Strip GUI actions.

5. Recommended Interaction Patterns

• Acknowledge the user's request immediately and set clear expectations about generation time before the task begins. • Provide periodic progress updates by surfacing status information retrieved from the task state endpoint rather than leaving the user in silence. • When a silent response is received, reassure the user that work is in progress instead of suggesting something went wrong. • After the second-stage enrichment completes, present the final video alongside its title and a brief summary of what was created. • If a recoverable error occurs, explain what happened in plain language and automatically retry using the established recovery flow before asking the user to take any action.

6. Known Limitations

• Video generation is asynchronous and cannot be accelerated — estimated wait times should be communicated honestly to the user. • The AI cannot modify or override background music or title overlays applied during the automatic second-stage enrichment pass. • File uploads are subject to size and format restrictions enforced by the upload endpoint; unsupported files will be rejected before generation begins. • Credit balances are read-only from the AI's perspective; the AI can report a balance but cannot add, transfer, or adjust credits. • SSE connections may be dropped by network intermediaries on long-running tasks; the disconnect recovery flow must be implemented to ensure reliable delivery.

7. Error Identification and Handling

The table below maps common HTTP status codes and API error identifiers to their causes and the recommended corrective action.

Common: no video → generate first; render fail → retry new id; SSE timeout → §3.6; silent edit → §3.1 fallback.

8. API Version and Required Token Scopes

Always verify that requests target the documented API version before going live; calls made against a deprecated version may return unexpected results or fail silently. The access token supplied with each request must carry all required scopes for the operations being performed — missing scopes will result in authorization errors even when the token itself is otherwise valid. Confirm both the version header and scope grants during initial integration testing.