Web to Markdown
Fetch a URL and save its content as a clean .md file in the project's docs/ directory.
Quick Start
Run the bundled script:
python3 <skill-path>/scripts/fetch_markdown.py <url> [--output <path>]
The script automatically:
- Tries
Accept: text/markdownheader (Cloudflare Markdown for Agents) - Falls back to HTML fetch + conversion if markdown not available
- Saves to
docs/<url-derived-name>.mdby default
Options
| Flag | Description | Default |
|---|---|---|
--output, -o |
Custom output path | docs/<name>.md |
--api-key |
Cloudflare API key | $CLOUDFLARE_API_KEY env |
--zone-id |
Cloudflare Zone ID | $CLOUDFLARE_ZONE_ID env |
Usage Patterns
Single page
python3 <skill-path>/scripts/fetch_markdown.py "https://react.dev/reference/react/useState"
# → docs/reference_react_useState.md
Custom output path
python3 <skill-path>/scripts/fetch_markdown.py "https://docs.example.com/api" --output docs/api/example-api.md
Multiple pages (run sequentially)
python3 <skill-path>/scripts/fetch_markdown.py "https://docs.example.com/getting-started" --output docs/getting-started.md
python3 <skill-path>/scripts/fetch_markdown.py "https://docs.example.com/api-reference" --output docs/api-reference.md
With Cloudflare API key
export CLOUDFLARE_API_KEY="your-key"
python3 <skill-path>/scripts/fetch_markdown.py "https://example.com/docs"
Conversion Strategy
- Cloudflare sites: If the site has Markdown for Agents enabled, the server returns clean markdown directly — no client-side parsing needed. This produces the highest quality output.
- All other sites: HTML is fetched and converted locally. The script extracts
<main>,<article>, or<body>content, strips nav/footer/scripts, and converts to markdown. Quality is good for documentation sites but may include some noise for complex layouts.
Output Format
Each saved file includes source metadata:
<!-- Source: https://example.com/docs/page -->
<!-- Fetched: 2026-02-17 14:30 -->
# Page Title
...content...
Notes
- No external Python dependencies required (uses only stdlib)
- Cloudflare API key is optional — the
Accept: text/markdownheader works without authentication on many Cloudflare-proxied sites that have enabled the feature - For JavaScript-heavy SPAs that render client-side, the HTML fallback may capture minimal content. Consider using a browser-based approach for those sites.
Resources
scripts/
fetch_markdown.py— Main fetch and conversion script. Zero dependencies, runs with Python 3.10+.