Audio To Subtitles

Audio To Subtitles

Use this skill to:

• turn local audio/video files or public media URLs into subtitle files through AI MediaKit;
• turn text into MiniMax TTS audio;
• optionally generate subtitles from the TTS audio in the same run.

Workflow

For media inputs:

1. If the input is a local file, upload it to Cloudflare R2 first.
2. Submit the public audio_url or video_url to AI MediaKit ASR subtitles.
3. Poll the async task until it is completed or failed.
4. Save .json, .srt, and .vtt outputs.

For local video-engine projects:

1. If the media belongs to a talking-head-remotion project under <VIDEO_WORKSPACE>, save subtitles under that project's work/captions/.
2. Use captions.srt and captions.vtt as the main cleaned files, keep raw ASR backups as captions.raw.srt and captions.raw.vtt, and keep the MediaKit payload as asr-result.json.
3. Also write captions_aligned.json parsed from the cleaned SRT so Remotion work can sync src/demoData.ts.
4. Current project default for 《两个提示词让 AI 少返工》:

<VIDEO_WORKSPACE>/your-talking-head-project/work/captions

For text inputs:

1. Generate audio through MiniMax TTS.
2. Save the audio file.
3. If --subtitles is set, upload the audio to R2 and run the same AI MediaKit subtitle workflow.

CLI

SKILL_DIR="./skills/audio-to-subtitles"

# Local audio/video: upload to R2, then transcribe
npx -y bun "$SKILL_DIR/scripts/main.ts" audio.mp3 --language zh-CN --out-dir subtitles

# Existing public URL: skip R2 upload
npx -y bun "$SKILL_DIR/scripts/main.ts" "https://example.com/audio.mp3" --out-dir subtitles

# Batch files/URLs
npx -y bun "$SKILL_DIR/scripts/main.ts" a.mp3 b.m4a "https://example.com/video.mp4" --out-dir subtitles

# Batch from manifest
npx -y bun "$SKILL_DIR/scripts/main.ts" --manifest inputs.txt --out-dir subtitles

# Speaker diarization
npx -y bun "$SKILL_DIR/scripts/main.ts" interview.mp3 --speaker --language zh-CN

# Text to audio only
npx -y bun "$SKILL_DIR/scripts/main.ts" --text "你好，欢迎收听。" --voice-id female-shaonv --out-dir audio

# Text file to audio + subtitles
npx -y bun "$SKILL_DIR/scripts/main.ts" --text-file script.md --subtitles --language zh-CN --out-dir audio

Options

Manifest

Text manifests (.txt) are one media input per line.

JSON manifests can mix media and text jobs:

[
  "local-audio.mp3",
  { "url": "https://example.com/video.mp4", "mediaKind": "video" },
  { "text": "你好，欢迎收听。", "outputName": "intro", "subtitles": true, "voiceId": "female-shaonv" },
  { "textFile": "script.md", "outputName": "script-audio", "subtitles": true }
]

Environment

The script loads environment files in this order. The nearest .env.r2 is loaded last and overrides global R2 defaults for this vault.

1. ~/.skills/.env
2. ~/.baoyu-skills/.env
3. nearest .env.r2 from current directory upward
4. nearest .skills/.env from current directory upward
5. nearest .baoyu-skills/.env from current directory upward

Required for MiniMax TTS text input:

Required for MediaKit subtitle generation:

Required for local-file uploads:

Notes

• AI MediaKit only accepts public HTTP/HTTPS media URLs. Local files must be uploaded first.
• TTS audio only needs MiniMax API credentials.
• TTS + subtitles also needs MediaKit and R2 credentials.
• Supported API languages are currently Simplified Chinese and English.
• Single media duration should not exceed the AI MediaKit limit of 3 hours.
• Do not print API keys or commit real .env.r2 values.
• Do not leave finished video subtitles only in <NOTES_VAULT>; for Remotion video work, copy or write them into the matching <VIDEO_WORKSPACE>/<project>/work/captions/ directory.