Verifying Stream Ground Truth
Overview
Visual ground truth (on-screen game banners, browser tabs, infocards) and real-time LiveChat listener reactions always supersede spoken ASR fluffs, streamer slip-ups, and LLM pre-training assumptions.
THE IRON RULE OF STREAM VERIFICATION:
Never guess from spoken slurs or pre-training memory when on-screen frames can be inspected.
When to Use
flowchart TD
A[Disputed Number / Proper Noun in Transcript?] -->|Yes| B{LiveChat Canary Burst?}
A -->|No| C[Normal ASR Topic Flow]
B -->|Viewer Teases/Corrects Number| D[High Suspicion: Streamer Misspoke]
B -->|Normal Banter| E{Astronomical or East Asian Unit?}
E -->|Numbers like 万, 億, พันล้าน| D
E -->|Standard Numbers| C
D --> F[Run slice_stream_frame.py for Targeted Window]
F --> G[Inspect Frame with Vision: Read Banner / UI Text]
G --> H[Convert Units & Emit Ground-Truth Timestamp]
Mandatory Triggers:
- Disputed or Astronomical Numbers: Streamer quotes impossible figures (e.g. "3,500 ล้านดาวน์โหลด", "30,000 กว่าล้าน").
- LiveChat Correction Canary: Viewers actively tease, challenge, or correct the streamer in real time ("35ล้านพอปู่ เยอะกว่าประชากรโลกละ").
- East Asian Unit Dissonance: Dialogue references Japanese/Korean game campaigns where
万 (10,000) or 億 (100,000,000) was translated incorrectly.
- Ambiguous Deictic Pronouns: Streamer says "ดูอันนี้ดิ", "ตู้ไหนดี", "ตัวนี้เก่งมาก" without naming the entity within ±30s.
- Explicit User Verification: User prompts
/btw use vision, --vision, or "ลองใช้ vision เช็คดู".
- Generative AI Outputs vs Hyperbolic Roasting: Streamer generates images/videos with AI or evaluates creative tools, and uses hyperbolic political/satirical roasts (e.g. "นี่มันพรรคคอมมิวนิสต์ชนะเลือกตั้งเหรอ", "อย่างกับรัฐบาลจีน", "หลอนเป็นเปรต").
- Iron Rule: NEVER report the streamer's hyperbolic roast as literal image content (e.g., do NOT write "TH AI เจนภาพ... กลายเป็นคณะรัฐบาลคอมมิวนิสต์").
- Action: Extract the targeted frame via
slice_stream_frame.py or inspect storyboard frames, ground what is literally depicted (e.g. "สาวไซเบอร์ผมขาว ยืนคู่บุคคลในชุดสูท"), and clearly frame the streamer's reaction as a roast under [Reaction].
When NOT to Use:
- Static podcast/radio segments where the streamer merely chats over a static background image or webcam.
- Verbatim news article reading where the text matches known RSS/news transcripts.
Quick Reference: 3-Step Verification Protocol
| Step |
Action |
Tool / Command |
| 1. Spot Canary |
Check LiveChat comments around timestamp for audience corrections. |
Grep LiveChat / view transcript overlay |
| 2. Targeted Slice |
Download ONLY a 1–2 minute slice (instant ~2-3s download). |
python3 scripts/slice_stream_frame.py "<URL>" "HH:MM:SS" |
| 3. Vision Grounding |
Inspect on-screen banner, calculate canonical number/name. |
view_file on frame image or agy --print |
Targeted Frame Extraction
Do NOT download multi-hour streams or wait for whole storyboard unpacks when verifying a single event.
Automated Helper (Recommended)
Use the included helper script:
# Slices a 60-second window around 00:14:45 and outputs exact frame
python3 <skill-dir>/scripts/slice_stream_frame.py \
"https://www.youtube.com/watch?v=VIDEO_ID" \
"00:14:45" \
-o "frames/banner_check.jpg"
Manual CLI Alternative
# 1. Download targeted 720p clip (instant ~2-3s download via Chrome cookies)
yt-dlp --cookies-from-browser chrome \
--download-sections "*00:14:15-00:15:15" \
-f "bestvideo[height<=720]+bestaudio/best[height<=720]" \
-o "target_slice.mp4" "https://www.youtube.com/watch?v=VIDEO_ID"
# 2. Extract exact frame at offset
ffmpeg -ss 00:00:30 -i target_slice.mp4.webm -frames:v 1 -q:v 2 target_frame.jpg -y
Japanese & East Asian Unit Conversion Trap
Streamers often misread Japanese kanji units on the fly when sleep-deprived:
| Japanese Kanji |
Numeric Value |
Thai Equivalent |
Streamer Misspeak Trap |
Correct Translation |
| 万 (man) |
10,000 |
1 หมื่น (หมื่น) |
Reads 3500万 as "3,500 ล้าน" or "3 หมื่นล้าน" |
3,500 × 10,000 = 35,000,000 (35 ล้าน / 35M) |
| 3000万 |
30,000,000 |
30 ล้าน |
Confused with 30,000 ล้าน |
30 ล้าน (30M) |
| 億 (oku) |
100,000,000 |
100 ล้าน |
Confused with พันล้าน |
100 ล้าน (100M) |
Rationalization Table & Red Flags
Agents under pressure find excuses to avoid visual inspection. Every excuse below is a violation:
| Rationalization / Excuse |
Reality & Counter |
| "Audio clearly says 30,000 so it must be 30M DL." |
Streamers misspeak constantly when tired. The on-screen banner is the authoritative source. |
| "I remember FGO 30M DL happened in October, so there is no DL campaign here." |
Pre-training memory is fallible and dates shift across servers. Never let memory overrule the screen. |
| "Downloading video takes too long and wastes bandwidth." |
slice_stream_frame.py uses --download-sections to fetch a tiny ~10MB slice in 2 seconds. |
| "The transcript is enough to guess the topic." |
Guessing produces hallucinations that degrade audience trust. When challenged, verify with frames. |
Red Flags - STOP and Run Vision
- Quoting download numbers > 1,000 ล้าน for a mobile game.
- Seeing LiveChat laugh (
555, ถถถ) or type numbers immediately following streamer statements.
- Removing campaign titles entirely because "the streamer sounded confused".
Common Mistakes
| Mistake |
Prevention / Fix |
| Downloading full stream video |
Always pass --download-sections "*START-END" to yt-dlp. |
| HTTP 403 Forbidden on video stream |
Always pass --cookies-from-browser chrome to authenticate yt-dlp. |
| Subagents hallucinating from fluffed speech |
Inject extracted frame OCR keywords into subagent prompts before timestamping. |
| Ignoring LiveChat correction timestamps |
Cross-reference LiveChat comments within ±30 seconds of the controversial statement. |
Real-World Impact (Case Study)
In Anibon stream mJGd2XW8Tlg:
- Streamer Spoken Words: "ฉลอง 3,500 ล้านดาวน์โหลด... ตู้ 30,000 กว่าล้านเนี่ยผมพูดผิด"
- Baseline Failure: Subagents hallucinated
30M DL. Manual review falsely deduced no DL campaign existed at all and removed the tag.
- Visual Ground Truth: Running
slice_stream_frame.py at 00:14:45 revealed the official banner: 3500万DL突破キャンペーン (35M DL) + LiveChat correcting "35ล้านพอปู่ 3หมื่นล้านเยอะกว่าประชากรโลกละ".
- Result: Corrected to
35M DL with 100% precision.
1---2name: verifying-stream-ground-truth3description: Use when livestream transcripts contain ambiguous proper nouns, disputed numbers, streamer misspeaks, or when users request visual verification of on-screen gameplay, banners, or infocards.4---56# Verifying Stream Ground Truth78## Overview910Visual ground truth (on-screen game banners, browser tabs, infocards) and real-time LiveChat listener reactions always supersede spoken ASR fluffs, streamer slip-ups, and LLM pre-training assumptions.1112```13THE IRON RULE OF STREAM VERIFICATION:14Never guess from spoken slurs or pre-training memory when on-screen frames can be inspected.15```1617---1819## When to Use2021```mermaid22flowchart TD23 A[Disputed Number / Proper Noun in Transcript?] -->|Yes| B{LiveChat Canary Burst?}24 A -->|No| C[Normal ASR Topic Flow]25 B -->|Viewer Teases/Corrects Number| D[High Suspicion: Streamer Misspoke]26 B -->|Normal Banter| E{Astronomical or East Asian Unit?}27 E -->|Numbers like 万, 億, พันล้าน| D28 E -->|Standard Numbers| C29 D --> F[Run slice_stream_frame.py for Targeted Window]30 F --> G[Inspect Frame with Vision: Read Banner / UI Text]31 G --> H[Convert Units & Emit Ground-Truth Timestamp]32```3334### Mandatory Triggers:351. **Disputed or Astronomical Numbers**: Streamer quotes impossible figures (e.g. *"3,500 ล้านดาวน์โหลด"*, *"30,000 กว่าล้าน"*).362. **LiveChat Correction Canary**: Viewers actively tease, challenge, or correct the streamer in real time (*"35ล้านพอปู่ เยอะกว่าประชากรโลกละ"*).373. **East Asian Unit Dissonance**: Dialogue references Japanese/Korean game campaigns where `万` (10,000) or `億` (100,000,000) was translated incorrectly.384. **Ambiguous Deictic Pronouns**: Streamer says *"ดูอันนี้ดิ"*, *"ตู้ไหนดี"*, *"ตัวนี้เก่งมาก"* without naming the entity within ±30s.395. **Explicit User Verification**: User prompts `/btw use vision`, `--vision`, or *"ลองใช้ vision เช็คดู"*.406. **Generative AI Outputs vs Hyperbolic Roasting**: Streamer generates images/videos with AI or evaluates creative tools, and uses hyperbolic political/satirical roasts (e.g. *"นี่มันพรรคคอมมิวนิสต์ชนะเลือกตั้งเหรอ"*, *"อย่างกับรัฐบาลจีน"*, *"หลอนเป็นเปรต"*).41 - **Iron Rule:** NEVER report the streamer's hyperbolic roast as literal image content (e.g., do NOT write *"TH AI เจนภาพ... กลายเป็นคณะรัฐบาลคอมมิวนิสต์"*).42 - **Action:** Extract the targeted frame via `slice_stream_frame.py` or inspect storyboard frames, ground what is literally depicted (e.g. *"สาวไซเบอร์ผมขาว ยืนคู่บุคคลในชุดสูท"*), and clearly frame the streamer's reaction as a roast under `[Reaction]`.4344### When NOT to Use:45- Static podcast/radio segments where the streamer merely chats over a static background image or webcam.46- Verbatim news article reading where the text matches known RSS/news transcripts.4748---4950## Quick Reference: 3-Step Verification Protocol5152| Step | Action | Tool / Command |53| :--- | :--- | :--- |54| **1. Spot Canary** | Check LiveChat comments around timestamp for audience corrections. | Grep LiveChat / view transcript overlay |55| **2. Targeted Slice** | Download ONLY a 1–2 minute slice (instant ~2-3s download). | `python3 scripts/slice_stream_frame.py "<URL>" "HH:MM:SS"` |56| **3. Vision Grounding** | Inspect on-screen banner, calculate canonical number/name. | `view_file` on frame image or `agy --print` |5758---5960## Targeted Frame Extraction6162Do NOT download multi-hour streams or wait for whole storyboard unpacks when verifying a single event.6364### Automated Helper (Recommended)65Use the included helper script:6667```bash68# Slices a 60-second window around 00:14:45 and outputs exact frame69python3 <skill-dir>/scripts/slice_stream_frame.py \70 "https://www.youtube.com/watch?v=VIDEO_ID" \71 "00:14:45" \72 -o "frames/banner_check.jpg"73```7475### Manual CLI Alternative76```bash77# 1. Download targeted 720p clip (instant ~2-3s download via Chrome cookies)78yt-dlp --cookies-from-browser chrome \79 --download-sections "*00:14:15-00:15:15" \80 -f "bestvideo[height<=720]+bestaudio/best[height<=720]" \81 -o "target_slice.mp4" "https://www.youtube.com/watch?v=VIDEO_ID"8283# 2. Extract exact frame at offset84ffmpeg -ss 00:00:30 -i target_slice.mp4.webm -frames:v 1 -q:v 2 target_frame.jpg -y85```8687---8889## Japanese & East Asian Unit Conversion Trap9091Streamers often misread Japanese kanji units on the fly when sleep-deprived:9293| Japanese Kanji | Numeric Value | Thai Equivalent | Streamer Misspeak Trap | Correct Translation |94| :--- | :--- | :--- | :--- | :--- |95| **万 (man)** | 10,000 | 1 หมื่น (หมื่น) | Reads `3500万` as *"3,500 ล้าน"* or *"3 หมื่นล้าน"* | `3,500 × 10,000` = **35,000,000 (35 ล้าน / 35M)** |96| **3000万** | 30,000,000 | 30 ล้าน | Confused with 30,000 ล้าน | **30 ล้าน (30M)** |97| **億 (oku)** | 100,000,000 | 100 ล้าน | Confused with พันล้าน | **100 ล้าน (100M)** |9899---100101## Rationalization Table & Red Flags102103Agents under pressure find excuses to avoid visual inspection. Every excuse below is a violation:104105| Rationalization / Excuse | Reality & Counter |106| :--- | :--- |107| *"Audio clearly says 30,000 so it must be 30M DL."* | Streamers misspeak constantly when tired. The on-screen banner is the authoritative source. |108| *"I remember FGO 30M DL happened in October, so there is no DL campaign here."* | Pre-training memory is fallible and dates shift across servers. Never let memory overrule the screen. |109| *"Downloading video takes too long and wastes bandwidth."* | `slice_stream_frame.py` uses `--download-sections` to fetch a tiny ~10MB slice in 2 seconds. |110| *"The transcript is enough to guess the topic."* | Guessing produces hallucinations that degrade audience trust. When challenged, verify with frames. |111112### Red Flags - STOP and Run Vision113- Quoting download numbers > 1,000 ล้าน for a mobile game.114- Seeing LiveChat laugh (`555`, `ถถถ`) or type numbers immediately following streamer statements.115- Removing campaign titles entirely because "the streamer sounded confused".116117---118119## Common Mistakes120121| Mistake | Prevention / Fix |122| :--- | :--- |123| **Downloading full stream video** | Always pass `--download-sections "*START-END"` to yt-dlp. |124| **HTTP 403 Forbidden on video stream** | Always pass `--cookies-from-browser chrome` to authenticate yt-dlp. |125| **Subagents hallucinating from fluffed speech** | Inject extracted frame OCR keywords into subagent prompts before timestamping. |126| **Ignoring LiveChat correction timestamps** | Cross-reference LiveChat comments within ±30 seconds of the controversial statement. |127128---129130## Real-World Impact (Case Study)131132In Anibon stream `mJGd2XW8Tlg`:133- **Streamer Spoken Words**: *"ฉลอง 3,500 ล้านดาวน์โหลด... ตู้ 30,000 กว่าล้านเนี่ยผมพูดผิด"*134- **Baseline Failure**: Subagents hallucinated `30M DL`. Manual review falsely deduced no DL campaign existed at all and removed the tag.135- **Visual Ground Truth**: Running `slice_stream_frame.py` at `00:14:45` revealed the official banner: **`3500万DL突破キャンペーン`** (35M DL) + LiveChat correcting *"35ล้านพอปู่ 3หมื่นล้านเยอะกว่าประชากรโลกละ"*.136- **Result**: Corrected to `35M DL` with 100% precision.
zenithth