How do I install the Web Fetch skill?

Run npx skillmds@latest add aiskillstore/web-fetch in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is Web Fetch safe to use?

This skill has not completed SkillMD's automated safety review yet. Capability flags: executes scripts, makes network calls. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Web Fetch?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Web Fetch free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

Who published Web Fetch?

aiskillstore (@aiskillstore) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Web Fetch by aiskillstore — Agent Skill

Name: Web Fetch
Rating: 4.2 (5 reviews)
Author: aiskillstore

File contents

Web Content Fetching

Fetch web content using curl | html2markdown with CSS selectors for clean, complete markdown output.

Quick Usage (Known Sites)

Use site-specific selectors for best results:

# Anthropic docs
curl -s "<url>" | html2markdown --include-selector "#content-container"

# MDN Web Docs
curl -s "<url>" | html2markdown --include-selector "article"

# GitHub docs
curl -s "<url>" | html2markdown --include-selector "article" --exclude-selector "nav,.sidebar"

# Generic article pages
curl -s "<url>" | html2markdown --include-selector "article,main,[role=main]" --exclude-selector "nav,header,footer"

Site Patterns

Site	Include Selector	Exclude Selector
platform.claude.com	`#content-container`	-
docs.anthropic.com	`#content-container`	-
developer.mozilla.org	`article`	-
github.com (docs)	`article`	`nav,.sidebar`
Generic	`article,main`	`nav,header,footer,script,style`

Universal Fallback (Unknown Sites)

For sites without known patterns, use the Bun script which auto-detects content:

bun ~/.claude/skills/web-fetch/fetch.ts "<url>"

Setup (one-time)

cd ~/.claude/skills/web-fetch && bun install

Finding the Right Selector

When a site isn't in the patterns list:

# Check what content containers exist
curl -s "<url>" | grep -o '<article[^>]*>\|<main[^>]*>\|id="[^"]*content[^"]*"' | head -10

# Test a selector
curl -s "<url>" | html2markdown --include-selector "<selector>" | head -30

# Check line count
curl -s "<url>" | html2markdown --include-selector "<selector>" | wc -l

Options Reference

--include-selector "CSS"  # Only include matching elements
--exclude-selector "CSS"  # Remove matching elements
--domain "https://..."    # Convert relative links to absolute

Comparison

Method	Anthropic Docs	Code Blocks	Complexity
Full page	602 lines	Yes	Noisy
`--include-selector "#content-container"`	385 lines	Yes	Clean
Bun script (universal)	383 lines	Yes	Clean

Troubleshooting

Wrong content selected: The site may have multiple articles. Inspect the HTML:

curl -s "<url>" | grep -o '<article[^>]*>'

Empty output: The selector doesn't match. Try broader selectors like main or body.

Missing code blocks: Check if the site uses non-standard code formatting.

Client-rendered content: If HTML only has "Loading..." placeholders, the content is JS-rendered. Neither curl nor the Bun script can extract it; use browser-based tools.

---name: web-fetchdescription: Fetches web content with intelligent content extraction, converting HTML to clean markdown. Use for documentation, articles, and reference pages http/https URLs.---# Web Content FetchingFetch web content using `curl | html2markdown` with CSS selectors for clean, complete markdown output.## Quick Usage (Known Sites)Use site-specific selectors for best results:```bash# Anthropic docscurl -s "<url>" | html2markdown --include-selector "#content-container"# MDN Web Docscurl -s "<url>" | html2markdown --include-selector "article"# GitHub docscurl -s "<url>" | html2markdown --include-selector "article" --exclude-selector "nav,.sidebar"# Generic article pagescurl -s "<url>" | html2markdown --include-selector "article,main,[role=main]" --exclude-selector "nav,header,footer"```## Site Patterns| Site | Include Selector | Exclude Selector ||------|------------------|------------------|| platform.claude.com | `#content-container` | - || docs.anthropic.com | `#content-container` | - || developer.mozilla.org | `article` | - || github.com (docs) | `article` | `nav,.sidebar` || Generic | `article,main` | `nav,header,footer,script,style` |## Universal Fallback (Unknown Sites)For sites without known patterns, use the Bun script which auto-detects content:```bashbun ~/.claude/skills/web-fetch/fetch.ts "<url>"```### Setup (one-time)```bashcd ~/.claude/skills/web-fetch && bun install```## Finding the Right SelectorWhen a site isn't in the patterns list:```bash# Check what content containers existcurl -s "<url>" | grep -o '<article[^>]*>\|<main[^>]*>\|id="[^"]*content[^"]*"' | head -10# Test a selectorcurl -s "<url>" | html2markdown --include-selector "<selector>" | head -30# Check line countcurl -s "<url>" | html2markdown --include-selector "<selector>" | wc -l```## Options Reference```bash--include-selector "CSS" # Only include matching elements--exclude-selector "CSS" # Remove matching elements--domain "https://..." # Convert relative links to absolute```## Comparison| Method | Anthropic Docs | Code Blocks | Complexity ||--------|----------------|-------------|------------|| Full page | 602 lines | Yes | Noisy || `--include-selector "#content-container"` | 385 lines | Yes | Clean || Bun script (universal) | 383 lines | Yes | Clean |## Troubleshooting**Wrong content selected**: The site may have multiple articles. Inspect the HTML:```bashcurl -s "<url>" | grep -o '<article[^>]*>'```**Empty output**: The selector doesn't match. Try broader selectors like `main` or `body`.**Missing code blocks**: Check if the site uses non-standard code formatting.**Client-rendered content**: If HTML only has "Loading..." placeholders, the content is JS-rendered. Neither curl nor the Bun script can extract it; use browser-based tools.

Frequently asked questions

Web Fetch