Channel To Kb Ytdlp

Build an OKF Knowledge Base from a YouTube Channel (yt-dlp)

Turn any YouTube channel into a synthesized, cross-linked knowledge base in Open Knowledge Format (OKF). OKF is Google's open standard for LLM-readable knowledge bases: plain markdown + YAML frontmatter, navigated by index and relative links, no database or embeddings required. Any AI can read the output with zero setup.

Uses yt-dlp for both channel enumeration and subtitle download. Free, no API key required.

Trade-offs: most reliable method (yt-dlp is actively maintained and handles YouTube changes fast), captures publish dates automatically. Recommended for production use.

Before you start

Read the OKF contract this bundle obeys:

1. Read SCHEMA.md in this repo (the maintainer contract: page types, frontmatter schemas, linking rules)
2. Skim the OKF SPEC for the hard rules (every page needs type in frontmatter, index.md is reserved, cross-links are relative markdown paths)

Step 1: Fetch transcripts

uv run .claude/skills/channel-to-kb-ytdlp/scripts/fetch_transcripts.py $ARGUMENTS --output-dir ./raw

This enumerates all videos, downloads English subtitles as json3, converts them to timestamped OKF raw-transcript pages. Output: raw/<slug>.md files (each with type: raw-transcript frontmatter, OKF-conformant) + raw/manifest.json.

Flags:

• --limit N to cap the number of videos (start here for testing)
• --delay 3.0 to increase the pause between requests if you hit 429 rate limits

Wait for the script to complete before proceeding.

Step 2: Build the OKF knowledge base

Read .claude/references/pipeline-guide.md for the full process. It produces an OKF-conformant bundle with the structure defined in SCHEMA.md. The stages:

1. Extract - read each raw transcript, extract concepts/entities/quotes as JSON to scripts/extractions/
2. Canonicalize - merge all extractions into a frozen taxonomy (scripts/manifest.json + scripts/taxonomy.json)
3. Write - write OKF concept/entity/source pages from the manifest (each with proper type, title, description frontmatter per SCHEMA.md)
4. Index - build index.md files for each directory (OKF's navigation layer)
5. Validate - run python lint.py to enforce OKF conformance, link integrity, and index coverage

Process in batches per the guide. For channels under ~30 videos, this fits in one session. For larger channels, save your extraction JSONs and resume across sessions.

Step 3: Validate OKF conformance

python lint.py

This enforces the OKF contract: every .md has type frontmatter (E1), every relative link resolves (E2), every page appears in its directory's index.md (E3), sources/raw parity (E4). Fix all errors. The knowledge base is ready when lint passes clean.