# Analyzing Video

> Analyzes video files by extracting frames, detecting scene changes, and optionally transcribing audio. Produces structured markdown analysis with timeline, visual descriptions, and key moments. Use when the user provides a video file (.mp4, .mov, .avi, .mkv, .webm) and wants analysis, asks to "analyze a video", "describe this video", "video timeline", "what happens in this video", or mentions video content understanding. Supports custom frame rates ("at 5fps", "1 frame every 3 seconds"), time ranges ("from 0:10 to 0:30", "first 15 seconds"), visual-only mode ("skip audio"), and user-provided transcripts.

- Skill: `bsisduck/analyzing-video` (Agent Skill, multi-file: 4 files)
- Install (CLI): `npx skillmds@latest add bsisduck/analyzing-video`
- Raw SKILL.md: https://api.skillmd.com/api/skills/bsisduck/analyzing-video/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Docs & Writing
- Author: bsisduck (https://skillmd.com/u/bsisduck)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/bsisduck/analyzing-video

---


# Video Analyzer

Extracts frames at adaptive rates, creates montage grids, detects scene changes, and dispatches parallel subagents for visual and audio analysis.

## Gotchas

- Social media downloads (Instagram, TikTok) embed a MJPEG thumbnail as video stream 0. The scripts auto-detect the real H.264/H.265 stream and skip `attached_pic` streams.
- macOS `grep` lacks `-P` (Perl regex). All scripts use `sed` and basic `grep` for compatibility.
- FFmpeg treats `.` in output filenames as image sequence patterns. Key frame extraction uses `-update 1` to write single images.
- Portrait videos (height > width) use 3x5 grid layout instead of 4x4 for better cell visibility.
- Scene detection threshold is `0.3`. For static content (presentations, documents), lower to `0.1`. For fast action (games, sports), raise to `0.4`.
- Always verify extraction produced output before dispatching analysis agents.

## Prerequisites

Requires `ffmpeg`, `ffprobe`, `python3`, `bc`. For transcription: `whisper` CLI.

```bash
brew install ffmpeg && pip install openai-whisper
```

## Transcription Modes

| Mode | Trigger | Behavior |
|------|---------|----------|
| **auto** | Default | Run Whisper |
| **user-provided** | User supplies transcript file/text | Use provided transcript |
| **skip** | "visual only", "skip audio", "no transcription" | No audio processing |

## Customization

Detect these overrides from the user's request:

| Override | User Says | Effect |
|----------|-----------|--------|
| **Custom FPS** | "at 5fps", "3 frames per second", "1 frame every 2 seconds" | Override tier-based fps rate |
| **Time range** | "from 0:10 to 0:30", "first 15 seconds", "last minute" | Analyze only the specified segment |
| **Both** | "analyze 0:10-0:30 at 5fps" | Custom fps on a segment |

Convert user phrasing to script parameters:
- "5fps" / "5 frames per second" → fps_override=`5`
- "1 frame every 3 seconds" → fps_override=`0.333`
- "1 frame every 10 seconds" → fps_override=`0.1`
- "from 0:10 to 0:30" → start_time=`10` end_time=`30`
- "first 15 seconds" → start_time=`0` end_time=`15`
- "last 30 seconds" → calculate: start_time=`duration-30`

Pass overrides to scripts:
```bash
bash ${CLAUDE_SKILL_DIR}/scripts/video-info.sh "<video_path>" [fps_override] [start_time] [end_time]
bash ${CLAUDE_SKILL_DIR}/scripts/extract-frames.sh "<video_path>" "<work_dir>/frames" <fps> <max> <stream> <grid> [start_time] [end_time]
```

When custom fps is set, tier becomes `custom` and the 1000-frame safety cap applies.

## Workflow

Copy this checklist and track progress:

```
- [ ] Step 1: Get video info (tier, stream, orientation)
- [ ] Step 2: Extract frames + grids + scene key frames
- [ ] Step 3: Verify extraction output (frames exist, grids exist)
- [ ] Step 4: Dispatch parallel analysis agents
- [ ] Step 5: Synthesize results into analysis document
- [ ] Step 6: Clean up temp directory
```

### Step 1: Get Video Info

```bash
bash ${CLAUDE_SKILL_DIR}/scripts/video-info.sh "$ARGUMENTS"
```

Returns JSON with: `video_stream_index`, `tier`, `fps_rate`, `max_frames`, `orientation`, `grid_layout`, `has_audio`, `work_dir`, `whisper_model`.

| Tier | Duration | Frame Rate | Whisper Model |
|------|----------|-----------|---------------|
| Short | <1 min | 2/sec | medium |
| Medium | 1-3 min | 1/sec | medium |
| Long | 3-10 min | 1/10sec | base |
| Extended | 10+ min | 1/20sec (max 60) | base |

### Step 2: Extract Frames (and Audio)

```bash
bash ${CLAUDE_SKILL_DIR}/scripts/extract-frames.sh "$ARGUMENTS" "<work_dir>/frames" <fps_rate> <max_frames> <stream_index> <grid_layout>
```

For transcription (auto mode only):
```bash
bash ${CLAUDE_SKILL_DIR}/scripts/extract-audio.sh "$ARGUMENTS" "<work_dir>/audio" <whisper_model>
```

Dispatch both in parallel when applicable.

### Step 3: Verify Extraction

Before dispatching agents, verify output:
- Check `<work_dir>/frames/grid_*.jpg` exist (at least 1 grid)
- Check `<work_dir>/frames/metadata.json` for frame count and grid count
- If scene changes detected, check `<work_dir>/frames/key_frames/` has images
- If extraction failed, report error and stop

### Step 4: Parallel Subagent Analysis

Dispatch in parallel:

- **Grid agents** (1 per 2-3 grids): Read montage grid images, describe visual content per time range
- **Key frame agent** (if scene changes detected): Read high-res `key_frames/` images for detailed scene-change analysis
- **Audio agent** (skip if mode is `skip`): Read transcription and audio metadata

### Step 5: Synthesize and Output

Merge all agent results into `<video_name>_analysis.md` next to the source video:

```markdown
# Video Analysis: [filename]

## Overview
| Property | Value |
|----------|-------|
| Duration | HH:MM:SS |
| Resolution | WxH |
| Orientation | portrait/landscape |
| Analysis Tier | short/medium/long/extended |
| Frames Analyzed | N |
| Scene Changes | N |
| Transcription | auto/user-provided/skipped |

## Executive Summary
[2-3 paragraph summary]

## Timeline
| Time | Visual | Audio/Speech |
|------|--------|-------------|
| 00:00 | [description] | [what is said/heard] |

## Detailed Scene Analysis
### Scene 1: [Title] (00:00 - 00:15)
**Visual**: [description]
**Audio**: [what is heard]
**Context**: [significance]

## Conclusions
[Overall analysis]
```

### Step 6: Cleanup

```bash
rm -rf "<work_dir>"
```

## Scripts

| Script | Purpose | Args |
|--------|---------|------|
| `scripts/video-info.sh` | Metadata, stream detection, tier | `<video_path>` |
| `scripts/extract-frames.sh` | Frames, grids, scene detection, key frames | `<video_path> <out_dir> <fps> [max] [stream] [grid]` |
| `scripts/extract-audio.sh` | Audio, silence detection, transcription | `<video_path> <out_dir> [model]` |

## References

- [references/ffmpeg-commands.md](references/ffmpeg-commands.md) — FFmpeg gotchas and non-obvious commands
- [references/transcription-guide.md](references/transcription-guide.md) — Whisper model selection and troubleshooting
- [references/analysis-strategies.md](references/analysis-strategies.md) — Subagent dispatch patterns and output template