Hyperframes Media

HyperFrames Media

Three preprocessing pipelines that turn raw inputs into HyperFrames-ready assets:

1. Narration — text to spoken audio (Kokoro TTS, runs locally)
2. Transcription — audio/video to timestamped captions (Whisper)
3. Cutouts — remove backgrounds from images for transparent overlays

Use this skill when the user wants any of those, as input to a HyperFrames composition. For standalone TTS or transcription that won't end up in a video, suggest a simpler approach.

When to use

• "Add a voiceover that says ..."
• "Generate narration for this script"
• "Caption this audio / video"
• "Turn this podcast into a captioned video"
• "Remove the background from this photo so I can overlay it"
• "Make this product shot transparent"

Narration (Kokoro TTS)

Local, fast, no API key needed. Good defaults:

• Voice: Kokoro ships several. Default to a neutral American voice for product/marketing; pick a warmer voice for storytelling.
• Speed: 1.0x for most cases. Slow to 0.9x if pronunciation matters (technical demos); speed to 1.1x for energetic shorts.
• Output: WAV at 24kHz, then convert to MP3 if size matters.

Drop the audio file into the HyperFrames project's assets/audio/ and reference it from a layer with data-audio="./assets/audio/voiceover.mp3". The composition's data-duration should match (or exceed) the audio length.

Transcription (Whisper)

For captions/subtitles, use Whisper to produce a timestamped transcript.

• Model size: base is fast and good enough for most accents. Use small or medium only if base produces errors.
• Output: Ask for SRT or JSON with per-word timestamps. Word-level timestamps unlock animated/karaoke-style captions.
• Format for HyperFrames: convert the timestamps to milliseconds and emit caption layers with data-start / data-end per line (or per word for kinetic captions).

If the user has source audio + a script, prefer alignment (Whisper with --initial_prompt to bias toward the known script) over raw transcription. Result is cleaner.

Background removal (cutouts)

For transparent overlays — speaker headshots floating over a colored background, product cutouts, mascot characters, etc.

• Library: rembg (uses U2-Net or similar) is the standard. Output a PNG with alpha.
• For video: process every nth frame, then chain with ffmpeg to reassemble. Slower but works.
• Quality check: zoom in on hair edges and semi-transparent areas (glass, hair) — those are where rembg struggles. Suggest the user re-shoot against a green screen if quality matters more than speed.

Drop the alpha PNG into assets/images/ and place it as a layer:

<img src="./assets/images/host.png"
     class="absolute right-12 bottom-12 w-64"
     data-start="0" data-end="6000" />

Tying it together

A common flow for a 30-second product demo:

1. User writes a 30s script
2. hyperframes-media → Kokoro generates the voiceover (~28s of audio)
3. Whisper transcribes the voiceover with word-level timestamps
4. website-to-hyperframes captures the product page at the right moments
5. HyperFrames composition layers: site capture (background), captions (timed to voiceover), cutout of the founder in the corner (start → 3s and end - 3s → end)
6. npx hyperframes render produces a captioned, narrated demo

Notes

• All three pipelines run locally. No API keys, no data leaving the machine — useful for client work under NDA.
• Keep raw inputs in assets/raw/ and processed outputs in assets/. Makes the project regeneratable.
• If a tool isn't installed yet, run the install command and proceed — don't ask the user to leave the conversation to set things up.