Sync All Channels

Extract videos from all enabled YouTube channels in channels.yaml. Use for batch updating video lists before downloading transcripts overnight, when the user mentions "sync channels", "update video lists", "refresh channels", or before running download-all-transcripts.

majiayu000 Updated Sep 9, 2026 567 repo stars

Coding & Dev Tools

Install

Press C to copy

$ npx skillmds@latest add majiayu000/sync-all-channels

Installs into ~/.claude/skills (or .claude/skills inside a project). Restart Claude Code to pick it up.

＋ Add to plugin

README badge

[![SkillMD](https://skillmd.com/badge/majiayu000/sync-all-channels.svg)](https://skillmd.com/skills/majiayu000/sync-all-channels)

Trust & safety

2 of 3 audits passed

Last audited Sep 24, 2026 · commit 5b0d21a

Permissions this skill uses

Makes network calls
The instructions fetch remote URLs or call external services.

Works with: Claude Code Claude.ai OpenAI Codex +

File contents

Sync All Channels

Why? Manually running extract-videos on each channel is tedious and error-prone. This skill automates batch extraction with rate limiting to respect YouTube's API.

Quick Start

# 1. Ensure yq is installed
brew install yq

# 2. Run the sync
./scripts/sync_all_channels.sh

[!TIP] Run this before download-all-transcripts to ensure video lists are current.

Workflow

1. Verify Prerequisites

Check that yq is installed:

which yq || echo "Install with: brew install yq"

2. Review Channel Configuration

Open data/channels.yaml and verify:

All desired channels have enabled: true
Video counts are appropriate (20-100 typical)
URLs point to channel video pages

channels:
  - folder: OpenAI
    url: https://www.youtube.com/@OpenAI/videos
    count: 50
    enabled: true

[!CAUTION] Setting count above 200 significantly increases sync time and may trigger rate limiting.

3. Execute the Sync

./scripts/sync_all_channels.sh

The script will:

Parse data/channels.yaml
Skip channels with enabled: false
Extract videos for each enabled channel
Wait 10 seconds between channels (rate limiting)
Report completion and next steps

4. Verify Results

After completion, check:

Each channel folder exists in data/
Each folder contains videos.csv with entries
No persistent errors in output

# Quick verification
ls -la data/*/videos.csv | head -10

Configuration

Adding a New Channel

Add entry to data/channels.yaml:

- folder: new-channel-name
  url: https://www.youtube.com/@ChannelName/videos
  count: 50
  enabled: true

Run sync - folder is created automatically

[!WARNING] Folder names should use lowercase with hyphens (e.g., my-channel). Avoid spaces or special characters.

Disabling a Channel

Set enabled: false - the channel will be skipped during sync but data is preserved.

Recommended Video Counts

Channel Type	Recommended Count
Infrequent posters (1-2/month)	20
Regular posters (weekly)	50
High-volume (daily)	100
Conference channels	100-200

Troubleshooting

Problem	Cause	Solution
`yq: command not found`	yq not installed	Run `brew install yq`
`Config file not found`	Running from wrong directory	Run from repo root: `./scripts/sync_all_channels.sh`
`Failed to sync [channel]`	Network issue or invalid URL	Check URL is accessible in browser, retry
Sync takes forever	Too many channels or high counts	Reduce counts or disable some channels
Empty videos.csv	Channel has no public videos or URL is wrong	Verify URL ends in `/videos`
Rate limit errors	Too many requests	Wait 1 hour and retry; reduce counts

Common Mistakes

Wrong URL format - Use https://www.youtube.com/@ChannelName/videos not /channel/ or /c/ URLs
Running from wrong directory - Must run from repo root, not from scripts/
Forgetting to enable - New channels default to enabled: false in templates
Excessive counts - Starting with count: 500 causes long waits; start with 50
Missing yq - The script fails immediately without yq; install before first run

Next Steps

After syncing channels:

# Download transcripts for all synced videos
./scripts/download_all_transcripts.sh

Quality Checklist

Before running sync:

yq is installed (which yq)
Running from repository root
data/channels.yaml exists and is valid YAML
At least one channel has enabled: true

After sync completes:

No persistent errors in output (warnings OK)
Each enabled channel has data/[folder]/videos.csv
CSV files contain video entries

Rate Limiting Details

10 second delay between channels (built into script)
Sequential processing - one channel at a time
Graceful failure - if one channel fails, others continue

[!TIP] For overnight batch operations, run sync first, then download-all-transcripts. This sequence ensures video lists are current before downloading.

---name: sync-all-channelsdescription: Extract videos from all enabled YouTube channels in channels.yaml. Use for batch updating video lists before downloading transcripts overnight, when the user mentions "sync channels", "update video lists", "refresh channels", or before running download-all-transcripts.--- # Sync All Channels **Why?** Manually running extract-videos on each channel is tedious and error-prone. This skill automates batch extraction with rate limiting to respect YouTube's API. ## Quick Start ```bash # 1. Ensure yq is installed brew install yq # 2. Run the sync ./scripts/sync_all_channels.sh ``` > [!TIP] > Run this before `download-all-transcripts` to ensure video lists are current. --- ## Workflow ### 1. Verify Prerequisites Check that `yq` is installed: ```bash which yq || echo "Install with: brew install yq" ``` ### 2. Review Channel Configuration Open `data/channels.yaml` and verify: - All desired channels have `enabled: true` - Video counts are appropriate (20-100 typical) - URLs point to channel video pages ```yaml channels: - folder: OpenAI url: https://www.youtube.com/@OpenAI/videos count: 50 enabled: true ``` > [!CAUTION] > Setting `count` above 200 significantly increases sync time and may trigger rate limiting. ### 3. Execute the Sync ```bash ./scripts/sync_all_channels.sh ``` The script will: 1. Parse `data/channels.yaml` 2. Skip channels with `enabled: false` 3. Extract videos for each enabled channel 4. Wait 10 seconds between channels (rate limiting) 5. Report completion and next steps ### 4. Verify Results After completion, check: - Each channel folder exists in `data/` - Each folder contains `videos.csv` with entries - No persistent errors in output ```bash # Quick verification ls -la data/*/videos.csv | head -10 ``` --- ## Configuration ### Adding a New Channel 1. Add entry to `data/channels.yaml`: ```yaml - folder: new-channel-name url: https://www.youtube.com/@ChannelName/videos count: 50 enabled: true ``` 2. Run sync - folder is created automatically > [!WARNING] > Folder names should use lowercase with hyphens (e.g., `my-channel`). Avoid spaces or special characters. ### Disabling a Channel Set `enabled: false` - the channel will be skipped during sync but data is preserved. ### Recommended Video Counts | Channel Type | Recommended Count | |-------------|-------------------| | Infrequent posters (1-2/month) | 20 | | Regular posters (weekly) | 50 | | High-volume (daily) | 100 | | Conference channels | 100-200 | --- ## Troubleshooting | Problem | Cause | Solution | |---------|-------|----------| | `yq: command not found` | yq not installed | Run `brew install yq` | | `Config file not found` | Running from wrong directory | Run from repo root: `./scripts/sync_all_channels.sh` | | `Failed to sync [channel]` | Network issue or invalid URL | Check URL is accessible in browser, retry | | Sync takes forever | Too many channels or high counts | Reduce counts or disable some channels | | Empty videos.csv | Channel has no public videos or URL is wrong | Verify URL ends in `/videos` | | Rate limit errors | Too many requests | Wait 1 hour and retry; reduce counts | --- ## Common Mistakes 1. **Wrong URL format** - Use `https://www.youtube.com/@ChannelName/videos` not `/channel/` or `/c/` URLs 2. **Running from wrong directory** - Must run from repo root, not from `scripts/` 3. **Forgetting to enable** - New channels default to `enabled: false` in templates 4. **Excessive counts** - Starting with `count: 500` causes long waits; start with 50 5. **Missing yq** - The script fails immediately without yq; install before first run --- ## Next Steps After syncing channels: ```bash # Download transcripts for all synced videos ./scripts/download_all_transcripts.sh ``` --- ## Quality Checklist Before running sync: - [ ] `yq` is installed (`which yq`) - [ ] Running from repository root - [ ] `data/channels.yaml` exists and is valid YAML - [ ] At least one channel has `enabled: true` After sync completes: - [ ] No persistent errors in output (warnings OK) - [ ] Each enabled channel has `data/[folder]/videos.csv` - [ ] CSV files contain video entries --- ## Rate Limiting Details - **10 second delay** between channels (built into script) - **Sequential processing** - one channel at a time - **Graceful failure** - if one channel fails, others continue > [!TIP] > For overnight batch operations, run sync first, then download-all-transcripts. This sequence ensures video lists are current before downloading.

majiayu000/claude-skill-registry-data/tree/main/data/sync-all-channels commit 5b0d21a796

Frequently asked questions

npx skillmds@latest add majiayu000/sync-all-channels