Docs Sources

Documentation Sources Skill

Understand documentation platforms and how to efficiently fetch content from each. Supports 20+ platforms and frameworks.

Variables

Instructions

MANDATORY - Follow the Workflow steps below when adding or analyzing documentation sources.

• Always check for AI-native signals first (llms.txt)
• Prefer raw content over rendered HTML
• Use browser automation only when necessary

Red Flags - STOP and Reconsider

If you're about to:

• Web crawl a site without checking for llms.txt first
• Use browser automation without trying curl first
• Add a source without detecting the documentation framework
• Skip GitHub raw access when repo is available

STOP -> Read the appropriate cookbook file -> Follow detection order -> Then proceed

Quick Decision Tree

What documentation are you adding?
│
├─ Has /llms.txt? ──────────────────────► llmstxt-strategy.md
│
├─ GitHub repo available?
│   └─ Has docs.yml/mint.json/mkdocs.yml? ► github-strategy.md
│
├─ Has /openapi.json or /asyncapi.yaml? ─► openapi-strategy.md
│
├─ Has /sitemap.xml? ───────────────────► sitemap-strategy.md
│
├─ Curl returns <1KB? (JS-rendered) ────► browser-strategy.md
│
└─ Nothing else works? ─────────────────► sitemap-strategy.md (crawl mode)

Workflow

1. Identify the documentation URL
2. CHECKPOINT: Check for AI-native signals (llms.txt, ai.txt)
3. Check if GitHub repo is available
4. Detect documentation framework from config files
5. Select appropriate strategy from cookbook
6. CHECKPOINT: Test fetch with curl before using browser
7. Configure registry entry with correct strategy
8. Validate fetch works correctly

Strategy Priority Order

Cookbook

llms.txt (AI-Native)

• IF: Site has /llms.txt or /llms-full.txt
• THEN: Read cookbook/llmstxt-strategy.md
• CHECK: curl -sI {url}/llms.txt | head -1

GitHub Raw

• IF: GitHub repo is linked from docs site
• THEN: Read cookbook/github-strategy.md
• SUPPORTS: Fern, Docusaurus, MkDocs, Mintlify, Sphinx, Nextra, Starlight

OpenAPI / AsyncAPI / GraphQL

• IF: Site has API spec (/openapi.json, /asyncapi.yaml, /graphql)
• THEN: Read cookbook/openapi-strategy.md

Sitemap / Web Crawl

• IF: Site has /sitemap.xml or need to crawl
• THEN: Read cookbook/sitemap-strategy.md

Browser Automation

• IF: Curl fails (JS-rendered, blocked, <1KB response)
• THEN: Read cookbook/browser-strategy.md
• ALSO: See browser-discovery skill for IDE browser tools

Quick Detection Commands

# Check for llms.txt
curl -sI "https://docs.viperjuice.dev/llms.txt" | head -1

# Check for sitemap
curl -sI "https://docs.viperjuice.dev/sitemap.xml" | head -1

# Check for OpenAPI
curl -sI "https://api.viperjuice.dev/openapi.json" | head -1

# Test curl response size (detect JS-rendered)
curl -s "https://docs.viperjuice.dev" | wc -c
# If < 1000 bytes, likely JS-rendered -> use browser

Framework Detection

See reference/framework-detection.md for detailed patterns.

Registry Configuration

See reference/registry-examples.md for configuration templates.

Basic structure:

{
  "source-id": {
    "name": "Display Name",
    "strategy": "strategy_name",
    "paths": {
      "homepage": "https://..."
    }
  }
}

Auto-Discovery from Manifests

When used with the library-detection skill, automatically suggest documentation sources based on detected dependencies.

Workflow

1. Receive stack info from library-detection skill
2. Map each framework/library to known documentation sources
3. Check if source already exists in ai-docs/libraries/_registry.json
4. Suggest /ai-dev-kit:docs-add commands for missing sources

Library-to-Docs Mapping

Auto-Discovery Command

When /ai-dev-kit:quickstart-codebase or manual request:

# Get detected frameworks
FRAMEWORKS=$(library-detection --output json | jq -r '.frameworks[].name')

# For each framework, check if docs exist
for fw in $FRAMEWORKS; do
  if ! grep -q "\"$fw\"" ai-docs/libraries/_registry.json; then
    echo "Missing docs for: $fw"
    echo "Suggest: /ai-dev-kit:docs-add <url> $fw"
  fi
done

Output

Returns list of:

{
  "already_tracked": ["react", "typescript"],
  "missing": [
    {"library": "fastapi", "url": "https://fastapi.tiangolo.com", "command": "/ai-dev-kit:docs-add https://fastapi.tiangolo.com fastapi"}
  ],
  "unknown": ["custom-internal-lib"]
}

Output

After adding a source, verify:

1. Pages are discovered correctly
2. Content is fetched without errors
3. Registry entry is valid JSON

Converted and distributed by TomeVault — claim your Tome and manage your conversions.