aselimc Read Software Docs
Fetch, parse, and navigate software documentation websites to answer user questions or build understanding of APIs, SDKs, and frameworks.
Script
scripts/fetch_docs.py — run with the project venv:
.venv/bin/python3 <skill-dir>/scripts/fetch_docs.py <mode> <url> [options]
Modes:
| Mode | Purpose | Key Options |
|---|---|---|
page |
Fetch a single page as clean markdown | --output FILE |
links |
List all internal doc links on a page | --filter REGEX |
sitemap |
Crawl from a root URL, build page tree | --depth N (default 1), --filter REGEX, --output FILE |
Dependencies (beautifulsoup4, html2text, requests) must be installed in the .venv.
Workflow
1. Orient: discover the doc structure
Start with links mode on the user's URL (or the doc site index) to see available pages:
.venv/bin/python3 scripts/fetch_docs.py links <url>
Use --filter to narrow results when the link list is large:
.venv/bin/python3 scripts/fetch_docs.py links <url> --filter "api|reference|python"
For broader discovery, use sitemap with shallow depth:
.venv/bin/python3 scripts/fetch_docs.py sitemap <url> --depth 1
2. Read: fetch specific pages
Fetch pages identified in step 1:
.venv/bin/python3 scripts/fetch_docs.py page <page-url>
For large pages, save to a file and read selectively:
.venv/bin/python3 scripts/fetch_docs.py page <page-url> --output .cache/knowledge/doc_page.md
3. Drill down: follow links as needed
If the initial page references sub-pages or related APIs, fetch those too. Repeat steps 1-2 to navigate deeper into the documentation tree.
4. Synthesize
After reading the relevant pages, produce the deliverable the user requested — this could be:
- A summary of an API or module
- A how-to guide assembled from multiple doc pages
- Answers to specific questions about the software
- Code examples based on the documented API
Save research output to .knowledge/ for easy user access (same pattern as read-arxiv-paper).
Tips
- WebFetch fallback: If
fetch_docs.pystruggles with a page (e.g., JavaScript-rendered content), fall back to theWebFetchtool which may handle it differently. - API docs: For Python/C++ API reference pages, use
--filterwith module names to quickly find the right page (e.g.,--filter "omni.isaac.core|isaacsim"). - Version pinning: Many doc sites include version numbers in URLs. Confirm the user wants the version in the URL before deeply reading docs for a different version.
- Caching: Save fetched pages to
.cache/knowledge/to avoid re-fetching during a session. Use descriptive filenames like.cache/knowledge/isaacsim_core_api.md.