Channel To Kb Supadata

Build an OKF Knowledge Base from a YouTube Channel (Supadata)

Turn any YouTube channel into a synthesized, cross-linked knowledge base in Open Knowledge Format (OKF). OKF is Google's open standard for LLM-readable knowledge bases: plain markdown + YAML frontmatter, navigated by index and relative links, no database or embeddings required. Any AI can read the output with zero setup.

Uses Supadata's managed API for transcript fetching.

Trade-offs: cleanest API, no IP blocking worries, AI fallback for uncaptioned videos. Requires a Supadata API key (free tier: 100 credits, then $17+/month at supadata.ai).

Before you start

Read the OKF contract this bundle obeys:

1. Read SCHEMA.md in this repo (the maintainer contract: page types, frontmatter schemas, linking rules)
2. Skim the OKF SPEC for the hard rules (every page needs type in frontmatter, index.md is reserved, cross-links are relative markdown paths)

Step 0: Set your API key

export SUPADATA_API_KEY=your_key_here

Or pass --api-key your_key to the script.

Step 1: Fetch transcripts

uv run .claude/skills/channel-to-kb-supadata/scripts/fetch_transcripts.py $ARGUMENTS --output-dir ./raw

This enumerates all videos via the Supadata channel API and fetches native English transcripts. Output: raw/<slug>.md files (each with type: raw-transcript frontmatter, OKF-conformant) + raw/manifest.json.

Flags:

• --limit N to cap the number of videos (start here for testing)
• --delay 1.0 to adjust request spacing

Credit usage: 1 credit per video (native mode). A 200-video channel costs ~200 credits.

Wait for the script to complete before proceeding.

Step 2: Build the OKF knowledge base

Read .claude/references/pipeline-guide.md for the full process. It produces an OKF-conformant bundle with the structure defined in SCHEMA.md. The stages:

1. Extract - read each raw transcript, extract concepts/entities/quotes as JSON to scripts/extractions/
2. Canonicalize - merge all extractions into a frozen taxonomy (scripts/manifest.json + scripts/taxonomy.json)
3. Write - write OKF concept/entity/source pages from the manifest (each with proper type, title, description frontmatter per SCHEMA.md)
4. Index - build index.md files for each directory (OKF's navigation layer)
5. Validate - run python lint.py to enforce OKF conformance, link integrity, and index coverage

Process in batches per the guide. For channels under ~30 videos, this fits in one session. For larger channels, save your extraction JSONs and resume across sessions.

Step 3: Validate OKF conformance

python lint.py

This enforces the OKF contract: every .md has type frontmatter (E1), every relative link resolves (E2), every page appears in its directory's index.md (E3), sources/raw parity (E4). Fix all errors. The knowledge base is ready when lint passes clean.