Crawlberg

Crawl, scrape, and convert websites to Markdown using the local crawlberg CLI and its MCP server. Use when the user wants to fetch a page, follow links across a domain, enumerate URLs, or drive a real browser. Covers installation, the subcommands (scrape, crawl, map, interact, batch-scrape, batch-crawl, download, citations, version, mcp, serve), output formats (JSON + Markdown), browser fallback, and when to prefer the MCP server over shelling out.

xberg-io Updated 9.6k repo stars
Xberg

File contents

xberg-io/crawlberg commit abd0097e69

Frequently asked questions

npx get-skill add xberg-io/crawlberg-2