# Youtube Artifact Collector > Collect rich metadata AND timestamped, segment-structured transcripts for multiple YouTube videos or a whole playlist, and build lossless structured artifacts (per-video JSON + readable Markdown) plus a per-collection manifest that preserves playlist→video relationships and degrades gracefully over private/deleted/unavailable videos. Use when the user wants to collect, archive, or build a dataset/knowledge base from several videos or a playlist — e.g. "collect artifacts for this playlist", "extract metadata and transcripts for these videos", "build a structured dataset from this channel's tutorials". This is NOT a quick single-video transcript dump — for just reading one video's captions, use the `youtube-transcript` skill instead. - Skill: `alikmlsipahi/youtube-artifact-collector` (Agent Skill, multi-file: 58 files) - Install (CLI): `npx skillmds@latest add alikmlsipahi/youtube-artifact-collector` - Raw SKILL.md: https://api.skillmd.com/api/skills/alikmlsipahi/youtube-artifact-collector/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: AI & ML - Author: alikmlsipahi (https://skillmd.com/u/alikmlsipahi) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/alikmlsipahi/youtube-artifact-collector --- # youtube-artifact-collector Skill 1 of the YouTube intelligence pipeline — the **production layer**. It turns one or more video sources into high-quality, LLM-friendly artifacts and never performs analysis itself. (Downstream analysis is the job of the consumption layer, e.g. the `spec-distiller` skill.) ## When to use this skill Use it when the request is about **producing structured artifacts from video sources**, especially across **multiple videos or a playlist**: - "Collect metadata and transcripts for this playlist." - "Build a dataset from these tutorial videos." - "Archive these videos as structured JSON I can feed to an LLM." **Do not** use it for a quick one-off "just give me the transcript of this video" — that is the narrower `youtube-transcript` skill. This skill deliberately collects metadata **and** transcripts and writes a relational artifact set. ## What it produces For every video, a lossless canonical **`.json`** plus a readable **`.md`** view — the `.md` carries a metadata header (channel, publish date, duration, transcript language and whether it is manual, automatic, or local ASR, categories, tags, and the description in a collapsed block) followed by the timestamped transcript, **sectioned under the video's own chapters** as `###` headings where the uploader defined any. Where `` is the video's title slugified — collection members are additionally numbered by playlist position (`01-.json`). For a collection (playlist or multi-URL run), a **`_manifest.json`** that records the ordered membership, per-member status, and a summary — failed/unavailable videos are **listed with their status and reason, never silently dropped**. Key properties of the artifact model: - `schema_version` on every artifact — the only contract the consumption layer depends on. Currently `1.2`; artifacts written before `whisper_model` existed carry `1.1`, and ones written before `transcript.status` existed carry `1.0` — both remain readable. Nothing branches on the version string in either skill; it is a record-keeping field, not a compatibility gate. - **`transcript.status` says *why* there is no transcript**, not just that there isn't one: `ok`, `not_requested` (a `--metadata-only` run), `no_captions` (an established absence — including a local ASR run that produced zero usable segments), or `fetch_failed` (captions were on offer but could not be fetched). `available` keeps its old meaning and is unaffected — the two can never disagree. Before this field, a metadata-only run and a genuinely captionless video produced byte-identical artifacts. The `.md` header reports the verdict in its first few lines, so a folder of these files can be scanned for "which of these actually has captions" without opening any of them. - Transcripts are **segment-based**: each segment carries a stable zero-based `index`, `start`, `duration`, `end`, and verbatim `text`. The `index` is a load-bearing stable address (future visual/derived artifacts reference it). Segments never carry word-level timestamps, on either source — see **Transcript source** below for where ASR's word data actually lives. - The **selected** transcript track is recorded (language, and `manual` / `auto` / **`asr`**) alongside the **full track inventory** (`available_tracks`). `asr` means this project's own local model produced it — recording it as `auto` would misattribute it to YouTube's own auto-captions. - The `collection{}` block links each video to its playlist/collection, and the manifest preserves the playlist→ordered-member relationship. - Transcript text is reproduced **byte-for-byte** — never edited, summarized, or reflowed. ## How to invoke The skill is a single PEP-723 `uv` script — dependencies resolve from the script header; there is no separate install step. ```bash uv run skills/youtube-artifact-collector/scripts/extract_artifacts.py \ … [flags] ``` `…` accepts one or more YouTube **video URLs**, bare **11-character video IDs**, or a **playlist URL**, mixed freely. ### Flags | Flag | Default | Meaning | | --- | --- | --- | | `--playlist` | off | Treat a `watch?v=…&list=…` URL as the **whole playlist**. By default such a URL is collected as the **single video** only. | | `--langs tr,en` | `tr,en` | Transcript language preference list, in order. Within a language a **manual** track is preferred over an **auto** one; if no preferred language matches, falls back to the **first available** track. | | `--out-dir NAME` | derived | Override the collection directory name (default is `-`). | | `--root DIR` | `data` | Output root directory, resolved relative to the current working directory. | | `--no-save` / `--print` | off | Print the artifacts to stdout instead of writing files to disk. | | `--format json\|md\|both` | `both` | Which per-video artifact files to write. | | `--metadata-only` | off | Skip transcript fetching entirely; collect metadata only. `--transcript-source` is ignored when this is set — ASR never runs, and `transcript.status` stays `not_requested`. | | `--skip-existing` | off | Skip videos whose artifact JSON is both on disk **and** carries a settled transcript for the source in use. Resolved by reading `transcript.status` back off the existing artifact (no network): `ok` skips on either source; `no_captions` skips only on the source that established it — a `captions`-sourced `no_captions` stays skipped, but the same artifact is retried under `--transcript-source asr`, because rescuing exactly those videos is what the ASR path is for; every other status (`not_requested`, `fetch_failed`, or an artifact predating the field) is retried regardless of source. A collection written with `--format md` cannot be skip-resolved and will be re-fetched, because a rendered Markdown view carries no video id. | | `--transcript-source captions\|asr` | `captions` | Where the transcript comes from. `captions` is YouTube's own track via `youtube-transcript-api`, unchanged. `asr` downloads the video's audio and transcribes it locally with `mlx-whisper` — see **Transcript source** below. There is no `auto`/fallback mode: the two are measured separately on purpose, so a failure never hides which side broke. | | `--whisper-model NAME` | `mlx-community/whisper-large-v3-turbo` | mlx-whisper model id. Only read when `--transcript-source asr`. Part of the ASR cache's filename — switching models never overwrites a previous model's cached output. | | `--audio-timeout N` | `600.0` | Seconds to allow the ASR audio download before killing it. Separate from `--timeout`, which was sized for yt-dlp's metadata call (a video's audio can be tens of megabytes and take much longer). | | `--sleep-requests N` | `2` | Jittered sleep before each network-hitting video (except the first) — a random delay in `[N, 2*N)` seconds, not a fixed `N`, to avoid a bot-like fixed-interval pattern. Applies before failed requests too (so repeated failures don't hot-loop); `--skip-existing` hits never touch the network and stay free. **`0` disables it** — an explicit opt-out, not the default. | | `--retries N` | `5` | How many times to re-try a network call that failed **transiently** (rate limit, bot check, 5xx, timeout). Permanent failures — private, unavailable, no captions — are never retried: re-asking would not change the answer and would aim more traffic at the service the retry budget exists to stay welcome with. On the `asr` source this governs only the audio *download* (a yt-dlp call, classified the same way); transcription itself is never retried — a missing `ffmpeg` or an unreachable model does not clear by waiting. | | `--retry-base N` | `5.0` | The first retry's delay in seconds; doubles each retry and is jittered like `--sleep-requests`. With the defaults the rungs are 5 → 10 → 20 → 40 → 80s, so a rate-limited video costs ~2.5–5 minutes before it is given up on. | | `--retry-cap N` | `300.0` | Ceiling for a single retry's delay. Does not bind at the default `--retries 5`; it is the rail that keeps the doubling bounded if you raise it. | | `--max-pacing N` | `60.0` | Ceiling that `--sleep-requests` escalates to. Each video lost to a rate limit **permanently** doubles the pacing for the rest of the run (2 → 4 → … → N): a run that keeps knocking at the same rate after being throttled is how a soft, recoverable throttle becomes a hard IP block. The run still finishes; it stops insisting. | | `--timeout N` | `120.0` | Seconds to allow each yt-dlp call before killing it. Previously unbounded — a hung process hung the whole run with no escape. A timeout classifies as transient, so it is retried. | ### Examples ```bash # Single video → data/_singles/what-is-claude-code.json + .md uv run …/extract_artifacts.py fl1DSmwQKKY # A watch?v=…&list=… URL, collected as just the video (default) uv run …/extract_artifacts.py "https://www.youtube.com/watch?v=fl1DSmwQKKY&list=PLxxxx" # The same URL, but collect the entire playlist uv run …/extract_artifacts.py "https://www.youtube.com/watch?v=fl1DSmwQKKY&list=PLxxxx" --playlist # Several videos at once, preferring English transcripts uv run …/extract_artifacts.py vid1 vid2 vid3 --langs en,tr # Inspect one video without writing files uv run …/extract_artifacts.py fl1DSmwQKKY --print # Transcribe locally instead of relying on YouTube's captions endpoint uv run …/extract_artifacts.py fl1DSmwQKKY --transcript-source asr --langs en ``` ## Transcript source **[v2.10]** Two independent transcript backends, chosen with `--transcript-source captions|asr` (default `captions`). **There is no `auto`/fallback mode** — the two are measured and run separately on purpose; a silent fallback would hide which side actually broke. `captions` is unchanged: `youtube-transcript-api` against YouTube's `timedtext` endpoint. `asr` bypasses that endpoint entirely — it downloads the video's audio via `yt-dlp -f bestaudio` (no transcoding: the Opus/WebM `yt-dlp` picks is read by `mlx-whisper` directly) and transcribes it locally with [`mlx-whisper`](https://github.com/ml-explore/mlx-examples/tree/main/whisper) (Apple Silicon only — the PEP-723 header marks the dependency `darwin`/`arm64`-only, so it installs on other platforms as a no-op and `--transcript-source asr` fails fast with a clear message instead of a bare `ImportError`). This exists because the captions endpoint can reject a network outright (bot-check / IP block) while `yt-dlp`'s own metadata, playlist enumeration and audio download keep working over the same connection — `asr` is the fork that survives that block, since it never touches `timedtext`. **No auto-detection.** The model is always given the first `--langs` value explicitly; forcing the wrong language onto audio produces Whisper's own failure mode — a repetitive, hallucinated transcript rather than an error — so `--langs` matters more on `asr` than it does on `captions`. **Cached, never re-downloaded or re-transcribed unnecessarily:** ``` data/_media// ├── audio. # yt-dlp -f bestaudio, no transcode └── whisper--.json # raw mlx-whisper output, word timestamps included ``` Both the cache filename and the re-run behavior are keyed on **model and language together** — not model alone. mlx-whisper is never given auto-detection, so a video transcribed once under one `--langs` and re-run under a different one must not silently reuse the first run's (potentially wrong-language, hallucinated) output; a different model likewise gets its own file so switching models never discards a previous model's cached transcript. If both the audio and the matching raw transcript are already cached, re-running the same (video, model, language) costs seconds, not another download-and-transcribe pass. **Word-level timestamps stay in the raw cache file and never reach the artifact.** `mlx-whisper` always runs with `word_timestamps=True`, but only segment-level `start`/`duration`/`end`/`text` are written to `transcript.segments` — the same shape `captions` produces. Word-level data would inflate the transcript section roughly tenfold for no benefit to `spec-distiller`'s extraction, which reads at segment granularity. **ASR failures never write a partial or empty artifact.** `ffmpeg` missing, the model failing to download, a full disk, undecodable audio — none of these establish anything about the video, so nothing is written; the member is recorded as `asr_failed` with a reason, and the run continues to the next video. This is the same invariant that governs a blocked `captions` fetch, applied to ASR's own failure modes, which are unrelated to YouTube's and so are never matched against `captions`' retry signals — an ASR failure is never retried into believing it was a YouTube rate limit. A local ASR run that completes but finds **zero segments** (silence, or a genuinely wrong-language forced transcription) is written as a complete `no_captions` artifact, not `ok` — otherwise `--skip-existing` would skip it forever; `no_captions` stays retryable on the `asr` source specifically so a later run (a corrected `--langs`, a better model) can still rescue it. ## What the run shows you **[v2.9, extended in v2.10]** **All of it goes to stderr.** stdout carries the artifact stream under `--print`/`--no-save` and nothing else, so `--print --format json | jq` is safe. There is no flag for any of this — the display adapts on its own. On a terminal you get the collection's name and size, then one live line — `x/y`, a percentage, a bar, what the run is waiting on, the elapsed time and the current title — with each finished video scrolling above it in colour: ``` Playlist: Kayıt Modülü Rehberi 24 videos · 19 already on disk · → data/kayit-modulu-rehberi-PLk-DU0q ok 00:12 03/24 Tek tek öğrenci yükleme · 412 segments skipped 00:00 04/24 Toplu aktarım [07/24] 25% [████──────────────] rate-limited, retry 3/5 - 41s left Sınıf tanımlama Done in 14:32 - 24 videos: 18 ok · 2 no captions · 1 skipped · 2 rate-limited · 1 unrecognized ``` Seven outcomes, and the **word** is the differentiator — colour is decoration, because plain mode, `NO_COLOR` and a colourblind reader all have to be able to read it. `ok` and `ok, no captions` are deliberately distinct: a video that simply has no captions is not the same result as one that has them, and collapsing the two on screen would undo what `transcript.status` records on disk. A `--metadata-only` run reads as plain `ok` — that absence was requested. `asr failed` is the seventh — distinct from `unrecognized` (a signal nobody has matched yet) because it names a specific, recognized ASR-side cause (`ffmpeg` missing, model unreachable, disk full, …) that is simply not YouTube's kind of failure. On `--transcript-source asr`, the phase name also carries `downloading audio`, `audio cached`, `transcribing`, and `whisper output cached` — so a run that looks idle is either genuinely waiting on the network or genuinely transcribing, never ambiguous about which. Transcribing can take minutes per video; the elapsed clock keeps ticking through it (once a second, on a live terminal) rather than freezing until the call returns, the same way a countdown already ticks through a rate-limit wait. Every wait is counted down, once a second, so the 2.5–5 minutes a rate-limited video costs at the defaults never looks like a hang. **One honest limit:** a stalled `yt-dlp` subprocess (up to `--timeout`, default 120s, or `--audio-timeout` for an ASR audio download) does not tick — the line names the phase it is waiting on rather than pretending to measure it. The claim is *"the run always says what it is waiting on"*, not *"the terminal never sits still"*. Off a terminal — CI, a pipe, `nohup … > log 2>&1`, or `--print` — it degrades to one plain line per video with no ANSI, no `\r` and no bar, and nothing is clipped, since a log wants the whole title and the whole path. `NO_COLOR` (presence, not value) drops the colour; a stderr that cannot encode UTF-8 gets the bar in ASCII and titles transliterated rather than mangled. ## Output layout ``` data/ ├── -/ # one folder per collection │ ├── _manifest.json # ordered membership + status + summary │ ├── 01-.json # lossless canonical artifact │ ├── 01-.md # readable view │ ├── 02-.json │ └── … ├── _singles/ # standalone (true single) videos │ ├── .json │ └── .md └── _media/ # ASR's own cache — never an artifact itself └── / ├── audio. # yt-dlp -f bestaudio, no transcode └── whisper--.json # raw mlx-whisper output ``` `_media/` is keyed by video id, not by collection — it is shared across every collection or single run that ever fetches that video's audio via `asr`, and is never touched by `--transcript-source captions`. ## Artifact naming A basename is the video's title, slugified — the same `slugify` that builds the collection folder name, so Turkish characters transliterate to ASCII and the result is filesystem-safe. - **Collection members** are prefixed with their playlist position (`01-`, `02-`, …), zero-padded to the member count so lexical order matches playlist order. Standalone videos get no prefix. - **A boilerplate prefix shared by every member's title is dropped.** A series whose titles all read `Örnek Yazılım | Kayıt Modülü Nasıl Kullanılır? …` yields `01-sube-ekleme.json`, not a 43-character prefix repeated 19 times. The prefix is only dropped when at least three titles agree on it and no member would be left with an empty name. - **The video id is the fallback**, used whenever a title yields no usable slug — a private/deleted member with no title, an emoji-only title, or a script that transliterates away entirely. It is also appended to disambiguate two standalone videos that share a title. `_manifest.json` records each member's actual filenames under `files{json,md}`, so consumers resolve artifacts through the manifest rather than reconstructing names. ## Graceful degradation Each video is fetched in isolation. A private, deleted, or otherwise unavailable video does **not** abort the run: its metadata fetch returns nothing, the run records the member as `metadata_failed` with a reason, and continues. Playlists also report `hidden_unavailable_count` — the number of members YouTube hides from the listing — parsed from yt-dlp's warning. A **rate-limited** video is a different animal and is recorded differently, as `rate_limited`. The distinction is the point: `metadata_failed` will never succeed, while `rate_limited` is the same video on a better day. Every rate-limited video is also reported on stderr, because only a collection has a manifest to record it in — a single video would otherwise fail silently. A third case is recorded as `unrecognized`: a failure the collector could not identify at all. It decides whether to retry by matching YouTube's own error wording, and that wording changes — so rather than guessing, an unrecognized failure says so, **prints the text it did not recognize**, and refuses to draw a conclusion from it. If you see one, the text on stderr is the thing to add to the signal list. This is the difference between "we know this video has no captions" and "we have no idea what just happened", and the second is never written down as the first. A fourth, `asr_failed`, exists only on `--transcript-source asr`: `ffmpeg` missing, the Whisper model failing to download, a full disk, or audio mlx-whisper could not decode. These causes are unrelated to YouTube's and are classified with their own separate signal list — never matched against the `unrecognized`/`rate_limited` signals above, so an ASR failure is never mistaken for a YouTube rate limit and retried into wasting minutes on a wait that will not help. Like every other unsuccessful case here, no artifact is written. **A `.json` on disk means a complete artifact.** When YouTube blocks a transcript we could otherwise have fetched, the artifact is **not written at all** — the video is reported and left for the next run. Writing it would be worse than useless: an artifact with an empty transcript is indistinguishable from a video that genuinely has no captions, `--skip-existing` would skip it forever, and the block would quietly become permanent. A video that really has no captions *is* complete and is written as normal. Use `--metadata-only` to collect metadata alone on purpose. Among the artifacts that *are* written, `transcript.status` keeps the remaining cases apart, and the `.md` view says which one it is rather than printing one sentence for all of them: a metadata-only run, a video with no captions, and one whose captions were offered but could not be fetched are three different facts. An artifact predating the field is rendered honestly as undetermined instead of being assigned a reason it never recorded. ## Tooling - **yt-dlp** — metadata and playlist enumeration, and (on `--transcript-source asr`) audio download, all run via subprocess for stable JSON and per-video isolation. - **youtube-transcript-api** — transcript tracks and segments on the `captions` source. - **mlx-whisper** — local transcription on the `asr` source. Apple Silicon only; a PEP-723 platform marker (`darwin`/`arm64`) keeps it from installing anywhere else, and imported lazily so a `captions`-only run never pays its cost or needs it installed. Two helpers (`extract_video_id`, `format_timestamp`) are copied verbatim from the older `youtube-transcript` skill; there is no runtime dependency on it. ## Scope Phase 1 collects **metadata** and **transcripts**, the latter from either of two independent backends (`captions`, `asr`). Visual/OCR/entity and other derived artifact types are out of scope, transcript correction/normalization is deferred until raw ASR output has been evaluated, and there is no `auto` fallback mode between the two transcript sources — see **Transcript source** above. The `schema_version`'d artifact model is designed to host later additions as sibling keys without reworking existing types.