inclusionai 1. Problem scenario
When you build videos with code-driven renderers (e.g. Remotion, AE scripts, complex FFmpeg filter graphs), you often need picture-in-picture: one main composition with another video embedded inside it.
Typical symptom: In the exported file, motion on the main layer (translation, scale, etc.) looks smooth, but the embedded clip stutters badly, drops frames, or even freezes for long stretches.
2. Root cause: sparse keyframes
Modern codecs (H.264/H.265) save space by storing full pictures only at scene cuts or every few seconds (keyframes / I-frames). Frames in between (P-frames / B-frames) only store differences from neighbors.
Engines like Remotion export frame by frame. To render frame N, the embedded clip must seek to the matching timestamp.
If the embedded file has almost no keyframes (e.g. one I-frame at the start of a 10 s clip), the decoder often has to decode from frame 0 forward to reach frame N. That leads to:
- Very slow seeks: Decoding takes so long that the renderer times out and grabs a frame before the decode finishes.
- Repeated frames: The decoder cannot keep up, so several consecutive captures show the same old image—stutter in the final output.
3. Fix: all-intra encoding (every frame a keyframe)
Idea: Re-encode the embedded asset so every frame is a keyframe. Then any seek returns a full picture immediately, with no long chains of dependent frames.
Steps
Step 1: Re-encode with FFmpeg
Run:
ffmpeg -i input.mp4 -c:v libx264 -g 1 -pix_fmt yuv420p output_keyframes.mp4
Parameters:
| Flag | Meaning |
|---|---|
-i input.mp4 |
Source clip you embed. |
-c:v libx264 |
H.264 for broad compatibility with web and renderers. |
-g 1 |
Critical: GOP size 1 → one keyframe per frame. |
-pix_fmt yuv420p |
Common 8-bit 4:2:0 layout for players and pipelines. |
output_keyframes.mp4 |
Output used as the fixed asset. |
(All-intra files are often several times larger than the original. That is usually fine for an intermediate asset used only during rendering.)
Step 2: Point your project at the new file
Replace paths so the composition uses output_keyframes.mp4 instead of the old input.mp4.
Step 3: Re-render
Export again; embedded playback should track smoothly with the main timeline.
4. Rules of thumb
Rule: Any asset that must be seeked frame-accurately from code should be pre-converted to all-intra (-g 1) before use.
This applies broadly in programmatic video: PiP, reverse playback, scrubber-driven playback, etc. If embedded video looks choppy, check keyframe spacing first and re-encode if needed.