kimtth Goal: let the agent actually watch a video instead of guessing from its title or a partial transcript.
Use for:
- analyzing someone else's content (hooks, ad creative, podcast intros) — what was shown and said
- diagnosing a bug from a screen recording — find the frame where it breaks
- summarizing a long video faster than watching at 2x
Inputs:
- a URL anything yt-dlp supports (YouTube, Loom, TikTok, X, Vimeo, ...) or a local path (.mp4, .mov, .mkv, .webm)
- a question to answer
Workflow:
- Download the URL with yt-dlp into a temp dir; probe local files in place.
- Extract frames with ffmpeg at a duration-aware rate. Hard caps: 2 fps, 100 frames.
- <=30s ~30 frames | 30-60s ~40 | 1-3min ~60 | 3-10min ~80 | >10min 100 (sparse).
- Get a timestamped transcript: native captions first (free), Whisper fallback when none exist.
- Read each frame as an image alongside the transcript; both carry t=MM:SS markers.
- Answer grounded in what is on screen and in the audio — not the title or description.
- Clean up the temp working dir unless follow-ups are expected.
Rules:
- Best accuracy under 10 minutes; for longer videos re-run focused on a window with --start/--end.
- Frames drive token cost — narrow the window before raising the frame cap or resolution.
- Bump frame width (e.g. 1024px) only when on-screen text must be read (slides, terminals, code).
- Public URLs and local files only; never log into private platforms.