How do I install the Spider skill?

Run npx skillmds@latest add spider-rs/spider in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is Spider safe to use?

This skill has not completed SkillMD's automated safety review yet. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Spider?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Spider free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

Who published Spider?

spider-rs (@spider-rs) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Spider by spider-rs — Agent Skill

Name: Spider
Author: spider-rs

Spider

Overview and tool-selection guide for Spider Cloud — crawl, scrape, search the web, extract structured data with AI, and drive remote browsers with anti-bot bypass. Use whenever "Spider" or "Spider Cloud" is mentioned, or when the task is to get web content (scrape a page, crawl a site), search the web, extract structured data from pages, or automate a real browser, and you want to pick the right Spider tool for the job.

spider-rs 89efc35 2.8 KB Updated Sep 17, 2026

File contents

Spider Cloud

Spider is a high-throughput web data platform for AI agents: crawl sites, scrape pages, search the web, extract structured data with AI, and control remote browsers — all through the hosted Spider MCP server with built-in anti-bot bypass, proxy rotation, and stealth escalation.

Use this skill as the starting point for any Spider task: it maps the request to the right tool. The individual capabilities each have a deeper skill — spider-crawl-scrape for fetching/searching content and spider-browser for interactive browser sessions.

Setup

The Spider MCP server is hosted at https://mcp.spider.cloud/mcp and uses OAuth: on first connection the client prompts for browser authorization — no API key to paste (a SPIDER_API_KEY Bearer token from https://spider.cloud/api-keys works as an alternative). Core tools run on pay-per-use credits; the AI tools require an AI subscription.

Picking the right tool

Need web content?

One page → spider_scrape
Multiple pages / follow links → spider_crawl
Behind bot protection → spider_unblocker
Already have HTML locally → spider_transform

Need to find something?

Web search → spider_search (set fetch_page_content: true to get full content)
Links on a known page → spider_links

Need AI-driven extraction? (requires AI subscription, each takes a plain-English prompt)

Structured data from a page → spider_ai_scrape
Crawl guided by intent → spider_ai_crawl
Search with ranking/filtering → spider_ai_search
Link discovery by description → spider_ai_links
Multi-step browser task in English → spider_ai_browser

Need interactive browser control? → the spider_browser_* tools (see the spider-browser skill)

Need account info?

Credit balance → spider_get_credits (check before large operations)

Common patterns

Search then scrape — spider_search to find URLs, then spider_scrape each, or set fetch_page_content: true to do it in one call.
Scrape then transform — spider_scrape with return_format: "raw" for HTML, then spider_transform to reformat without extra requests.
Bypass bot protection — try spider_scrape first; if blocked, escalate to spider_unblocker; for interactive sites use the browser tools with higher stealth.
Budget guard — spider_get_credits before a large crawl.

---name: spiderdescription: Overview and tool-selection guide for Spider Cloud — crawl, scrape, search the web, extract structured data with AI, and drive remote browsers with anti-bot bypass. Use whenever "Spider" or "Spider Cloud" is mentioned, or when the task is to get web content (scrape a page, crawl a site), search the web, extract structured data from pages, or automate a real browser, and you want to pick the right Spider tool for the job.---# Spider CloudSpider is a high-throughput web data platform for AI agents: crawl sites, scrapepages, search the web, extract structured data with AI, and control remotebrowsers — all through the hosted Spider MCP server with built-in anti-botbypass, proxy rotation, and stealth escalation.Use this skill as the starting point for any Spider task: it maps the request tothe right tool. The individual capabilities each have a deeper skill —`spider-crawl-scrape` for fetching/searching content and `spider-browser` forinteractive browser sessions.## SetupThe Spider MCP server is hosted at `https://mcp.spider.cloud/mcp` and uses OAuth:on first connection the client prompts for browser authorization — no API key topaste (a `SPIDER_API_KEY` Bearer token from https://spider.cloud/api-keys works asan alternative). Core tools run on pay-per-use credits; the AI tools require an[AI subscription](https://spider.cloud/ai/pricing).## Picking the right tool**Need web content?**- One page → `spider_scrape`- Multiple pages / follow links → `spider_crawl`- Behind bot protection → `spider_unblocker`- Already have HTML locally → `spider_transform`**Need to find something?**- Web search → `spider_search` (set `fetch_page_content: true` to get full content)- Links on a known page → `spider_links`**Need AI-driven extraction?** (requires AI subscription, each takes a plain-English `prompt`)- Structured data from a page → `spider_ai_scrape`- Crawl guided by intent → `spider_ai_crawl`- Search with ranking/filtering → `spider_ai_search`- Link discovery by description → `spider_ai_links`- Multi-step browser task in English → `spider_ai_browser`**Need interactive browser control?** → the `spider_browser_*` tools (see the `spider-browser` skill)**Need account info?**- Credit balance → `spider_get_credits` (check before large operations)## Common patterns- **Search then scrape** — `spider_search` to find URLs, then `spider_scrape` each, or set `fetch_page_content: true` to do it in one call.- **Scrape then transform** — `spider_scrape` with `return_format: "raw"` for HTML, then `spider_transform` to reformat without extra requests.- **Bypass bot protection** — try `spider_scrape` first; if blocked, escalate to `spider_unblocker`; for interactive sites use the browser tools with higher `stealth`.- **Budget guard** — `spider_get_credits` before a large crawl.

Frequently asked questions