# Anibon Stream Activity

> Download high-resolution YouTube storyboards (sb3/sb2), detect on-screen activity, webcam presence/AFK, and facial emotions to provide visual grounding for Anibon livestream timestamping.

- Skill: `zenithth/anibon-stream-activity` (Agent Skill, multi-file: 5 files)
- Install (CLI): `npx skillmds@latest add zenithth/anibon-stream-activity`
- Raw SKILL.md: https://api.skillmd.com/api/skills/zenithth/anibon-stream-activity/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: zenithth (https://skillmd.com/u/zenithth)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/zenithth/anibon-stream-activity

---


# Anibon Stream Activity & Visual Grounding

## Overview

Tracks what speaker "Pu Boat" (โบ๊ต / PhuBoat) is doing on screen and in his webcam throughout an Anibon Official live stream. Solves the **Speech vs. Action Divergence** problem where the speaker discusses real-world topics while casually playing a game or browsing the web.

## When to Use

- Live streams featuring Pu Boat where he plays games, browses websites/reviews, reacts to videos, or pulls gacha.
- When `anibon-timestamper` needs visual ground truth to prevent hallucinated in-game lore, resolve deictic pronouns (*"ดูอันนี้ดิ"*, *"เกมนี้"*), and ground AFK breaks.
- In multi-hour streams where speaker leaves the desk (AFK empty chair detection).

---

## Storyboard Resolution & Download

Always download the highest-resolution storyboard tier available to avoid pixelation and blur when inspecting UI text, character names, or facial expressions:

```bash
# Downloads highest resolution storyboard (sb3 -> sb2 -> sb1 -> sb0)
yt-dlp -f "sb3/sb2/sb1/sb0" "https://www.youtube.com/watch?v=VIDEO_ID" -o "<workspace>/frames/storyboard.%(ext)s"
```

---

## Pipeline & Helper Scripts

All helper scripts reside in `skills/anibon-stream-activity/scripts/`:

### 1. Unpack Storyboard Slides (`unpack_storyboard.py`)

Extracts clean native JPEG slide sheets from MHTML and provides frame cropping across arbitrary grid sizes (3x3, 5x5, 10x10):

```bash
python3 scripts/unpack_storyboard.py \
  --mhtml "<workspace>/frames/storyboard*" \
  --out-dir "<workspace>/frames/slides"
```

### 2. Extract Activity & Webcam Timeline (`extract_activity_timeline.py`)

Samples storyboard frames every 60–90 seconds + scene change shifts, runs Gemini Vision / `agy` classification, and merges into continuous time intervals:

```bash
python3 scripts/extract_activity_timeline.py \
  --slides-dir "<workspace>/frames/slides" \
  --duration <DURATION_SECONDS> \
  --step-sec 60 \
  -o "<workspace>/activity_timeline.json"
```

**Output Schema (`activity_timeline.json`):**
```json
[
  {
    "start": "00:15:00",
    "end": "00:45:00",
    "start_sec": 900,
    "end_sec": 2700,
    "app_or_game": "Monster Hunter Wilds",
    "category": "Gameplay",
    "details": "Hunting Dosshaguma in Windward Plains",
    "webcam": {
      "speaker_present": true,
      "expression": "focused",
      "layout": "corner_cam"
    }
  }
]
```

### 3. Align to Transcript Chunks (`align_activity_timeline.py`)

Maps activity intervals across 5-minute transcript chunks:

```bash
python3 scripts/align_activity_timeline.py \
  --timeline "<workspace>/activity_timeline.json" \
  --chunks "<workspace>/chunks/" \
  -o "<workspace>/activity/"
```

Emits `<workspace>/activity/activity_chunk_NN.txt` ready for subagent prompt injection.

---

## Disambiguation & Subagent Reasoning Rules

When injecting `ON_SCREEN_ACTIVITY` into `anibon-chunk-timestamper`, subagents MUST apply these rules:

1. **Divergence Rule (Speech ≠ Screen Action)**:
   - When dialogue covers general topics (life stories, food, news, anime lore) while casually playing or farming:
     - Use speech-primary tag: `[Talk]`, `[News]`, or `[Question]`.
     - Suffix the background game in parentheses: `[Talk] เม้าท์มอยเรื่อง... (ระหว่างเล่น Monster Hunter)`.
     - **NEVER** fabricate or hallucinate real-life stories as in-game lore or quest text.
2. **Convergence Rule (Speech = Screen Action)**:
   - When dialogue actively reacts to in-game combat, boss mechanics, or gacha:
     - Use action tag: `[Gameplay]`, `[Gacha]`, or `[Reaction]`.
3. **Webcam & AFK Grounding**:
   - When `speaker_present: false` (empty chair / away from desk) and audio is silent or ambient:
     - Tag as `[AFK] ปู่โบ๊ตลุกไปพักเบรก / เข้าห้องน้ำ` if > 2 minutes.
     - Explains silent gaps cleanly and prevents false topic switching.
4. **Physical Emotion Grounding**:
   - Cross-reference webcam expression (`laughing`, `shocked_facepalm`) with LiveChat `555` bursts to accurately calibrate tone in descriptions.
5. **Deictic Pronoun Grounding**:
   - Resolve *"ดูนี่ดิ"*, *"เกมนี้"*, *"ตัวนี้"* using the `app_or_game` name and visible UI text in `ON_SCREEN_ACTIVITY`.
6. **Visual Ground Truth Priority Over Spoken Misspeaks**:
   - Streamers frequently misspeak numbers, campaign titles, or dates when tired. On-screen graphics, web browser tabs, and game banners take strict precedence over spoken words in ASR.
7. **Japanese Number Unit Conversion Trap (`万` vs `ล้าน`)**:
   - Japanese games and wikis use `万` (10,000 / man) and `億` (100,000,000 / oku).
   - Streamers commonly misread `3500万` as *"3,500 ล้าน"* or *"30,000 กว่าล้าน"*.
   - Always convert mathematically: `3,500 × 10,000 = 35,000,000` = **35 ล้าน (35M DL)**.
8. **LiveChat as Real-Time Correction Canary**:
   - When spoken numbers sound absurd or contradictory, look at the LiveChat comments around that timestamp. Chat listeners are often the first to notice and correct the streamer (e.g. *"35 ล้านพอปู่ 3 หมื่นล้านเยอะกว่าประชากรโลกละ"*).