Video Highlights

Video Highlights Clipper

Take a video file, transcribe it (if not already transcribed), score and rank segments to identify the most compelling highlights, and export them as individual clip files using ffmpeg.

PREREQUISITES

This skill requires:

• ffmpeg — for audio extraction and video clipping (brew install ffmpeg on macOS, sudo apt install ffmpeg on Linux)
• Python 3 + openai-whisper — for transcription (pip install openai-whisper)

WORKFLOW

Step 1: Locate the Video and Transcript

1. Confirm the video file path from the user's argument or conversation.
2. Check if a transcript already exists alongside the video (e.g., Name_transcript.txt or Name_transcript.json in the same directory).
3. If no transcript exists, generate one:
   • Extract audio: ffmpeg -i <video> -ar 16000 -ac 1 -vn <audio.wav>
   • Transcribe using Whisper:

     import whisper, json
     model = whisper.load_model("base")
     result = model.transcribe("<audio.wav>", word_timestamps=True, language="en")
     

   • Save both .txt (with timestamps) and .json (with segment data) transcripts in the same directory as the video.

Step 2: Read and Analyze the Full Transcript

Read the entire transcript — if it's large, read in chunks but track candidates across the full duration. Do NOT bias toward the beginning or end.

For each potential highlight segment, group consecutive transcript lines into coherent "moments" — a moment is a self-contained thought, story, or argument (typically 3-20 consecutive transcript lines).

Step 3: Score Each Candidate Moment

Rate every candidate moment on these 6 dimensions (each 1-5):

Composite Score = sum of all 6 dimensions (max 30).

Scoring rules:

• A moment does NOT need to score high on every dimension. A 5 in Quotability + 1s elsewhere (total 10) can still be a great clip if it's a killer one-liner.
• Weight Hook Strength and Quotability slightly higher for short-form platforms (Reels, TikTok, Shorts).
• Weight Insight Density and Story Arc higher for long-form platforms (YouTube, LinkedIn).
• Moments that score 3+ on at least 3 dimensions AND have a composite score >= 15 are strong candidates.

Step 4: Rank and Select Top Highlights

1. Rank all candidates by composite score (descending).
2. Select the top 5-10 (or as many as the user requests).
3. Apply diversity filter: avoid selecting multiple clips from the same 2-minute window. Spread selections across the full video timeline.
4. Apply clip length targets:
   • Default: 15-90 seconds per clip
   • Reels / TikTok / Shorts: 15-60 seconds
   • LinkedIn: 30-90 seconds
   • YouTube: 1-5 minutes
5. Determine precise start and end timestamps — begin at the start of the first sentence of the moment, end at the natural conclusion. Add 2-3 seconds of padding on each side.

Step 5: Present Highlights to User for Approval

For each selected clip, present:

## Clip N: "<Title>" (Score: XX/30)
- Timestamps: [HH:MM:SS -> HH:MM:SS] (~XXs)
- Scores: Q:X | I:X | E:X | S:X | A:X | H:X
- Why: <1-line reason this is a highlight>
- Transcript:
  > "<exact transcript text>"

Also present:

• Timeline distribution — a simple visual showing where clips fall across the video duration, to confirm good spread
• Honorable mentions — 2-3 moments that scored well but didn't make the cut, in case the user wants to swap

Wait for user confirmation before cutting. The user may:

• Approve all
• Remove specific clips
• Add honorable mentions
• Adjust timestamps
• Request different clip lengths

Step 6: Cut the Clips

Once confirmed, use ffmpeg to cut each clip:

ffmpeg -i <video> -ss <start_time> -to <end_time> -c:v libx264 -c:a aac -avoid_negative_ts make_zero -y <output_clip>

Output structure: Save clips in a clips/ subfolder next to the video:

<video_directory>/clips/<VideoName>_clip01_<title_slug>.mp4
<video_directory>/clips/<VideoName>_clip02_<title_slug>.mp4

• Use re-encoding (-c:v libx264 -c:a aac) for precise, frame-accurate cuts.
• Sanitize title slugs: lowercase, underscores, no special characters, max 40 chars.

Step 7: Generate Clips Manifest

After cutting, create a clips_manifest.md file in the clips/ folder:

# Video Highlights — <VideoName>
Source: <original video path>
Date: <today's date>
Total clips: N

| # | File | Title | Time | Duration | Score | Best Dimension |
|---|------|-------|------|----------|-------|----------------|
| 1 | clip01_xxx.mp4 | "Title" | 02:03-02:14 | 11s | 24/30 | Quotability |
| ... |

Step 8: Report Results

Report to the user:

• Total clips created with file paths
• Duration of each clip
• Combined highlight reel duration
• Any issues encountered (e.g., audio sync, encoding warnings)

RULES

• Never invent or fabricate transcript content. Only use what is actually in the transcript.
• If the video is very long (60+ min), read the transcript in chunks but track all candidates — do not discard earlier candidates when reading later chunks.
• If the user asks for a specific number of clips or total duration, adjust the selection threshold accordingly.
• Always confirm clip selections with the user before cutting — never auto-cut without approval.
• If a highlight moment crosses a natural pause or topic change, prefer trimming to the tighter version rather than including filler.
• Prefer clips that start strong — the first 3 seconds of each clip should be immediately engaging (no "um", "so", throat-clearing).
• If two adjacent highlights are within 10 seconds of each other, consider merging them into one longer clip rather than creating two clips with overlapping padding.