How do I install the Scrapling skill?

Run npx skillmds@latest add dvcrn/scrapling-4 in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is Scrapling safe to use?

This skill has not completed SkillMD's automated safety review yet. Independent scanners report: SkillSpector: PASS, Skill Scanner: PASS. Capability flags: executes scripts, makes network calls. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Scrapling?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Scrapling free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

Who published Scrapling?

dvcrn (@dvcrn) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Scrapling by dvcrn — Agent Skill

Name: Scrapling
Rating: 4.2 (5 reviews)
Author: dvcrn

Scrapling

Web scraping using Scrapling — a Python framework with anti-bot bypass (Cloudflare Turnstile, fingerprint spoofing), adaptive element tracking, stealth headless browser, and full CSS/XPath extraction. Use when web_fetch fails (Cloudflare, JS-rendered pages), or when extracting structured data from websites (prices, articles, lists). Supports HTTP, stealth, and full browser modes. Source: github.com/D4Vinci/Scrapling (PyPI: scrapling). Only use on sites you have permission to scrape.

dvcrn Updated Sep 9, 2026 32 repo stars

File contents

Scrapling Skill

Source: https://github.com/D4Vinci/Scrapling (open source, MIT-like license) PyPI: scrapling — install before first use (see below)

⚠️ Only scrape sites you have permission to access. Respect robots.txt and Terms of Service. Do not use stealth modes to bypass paywalls or access restricted content without authorization.

Installation (one-time, confirm with user before running)

pip install scrapling[all]
patchright install chromium  # required for stealth/dynamic modes

scrapling[all] installs patchright (a stealth fork of Playwright, bundled as a PyPI package — not a typo), curl_cffi, MCP server deps, and IPython shell.
patchright install chromium downloads Chromium (~100 MB) via patchright's own installer (same mechanism as playwright install chromium).
Confirm with user before running — installs ~200 MB of dependencies and browser binaries.

Script

scripts/scrape.py — CLI wrapper for all three fetcher modes.

# Basic fetch (text output)
python3 ~/skills/scrapling/scripts/scrape.py <url> -q

# CSS selector extraction
python3 ~/skills/scrapling/scripts/scrape.py <url> --selector ".class" -q

# Stealth mode (Cloudflare bypass) — only on sites you're authorized to access
python3 ~/skills/scrapling/scripts/scrape.py <url> --mode stealth -q

# JSON output
python3 ~/skills/scrapling/scripts/scrape.py <url> --selector "h2" --json -q

Fetcher Modes

http (default) — Fast HTTP with browser TLS fingerprint spoofing. Most sites.
stealth — Headless Chrome with anti-detect. For Cloudflare/anti-bot.
dynamic — Full Playwright browser. For heavy JS SPAs.

When to Use Each Mode

web_fetch returns 403/429/Cloudflare challenge → use --mode stealth
Page content requires JS execution → use --mode dynamic
Regular site, just need text/data → use --mode http (default)

Python Inline Usage

For custom logic beyond the CLI, write inline Python. See references/patterns.md for:

Adaptive scraping (auto_save / adaptive — saves element fingerprints locally)
Session/cookie handling
Async usage
XPath, find_similar, attribute extraction

Notes

MCP server (scrapling mcp): starts a local network service for AI-native scraping. Only start if explicitly needed and trusted — it exposes a local HTTP server.
auto_save=True: persists element fingerprints to disk for adaptive re-scraping. Creates local state in working directory.
Stealth/dynamic modes use Chromium headless — no xvfb-run needed.
For large-scale crawls, use the Spider API (see Scrapling docs).

---name: scrapling-4description: Web scraping using Scrapling — a Python framework with anti-bot bypass (Cloudflare Turnstile, fingerprint spoofing), adaptive element tracking, stealth headless browser, and full CSS/XPath extraction. Use when web_fetch fails (Cloudflare, JS-rendered pages), or when extracting structured data from websites (prices, articles, lists). Supports HTTP, stealth, and full browser modes. Source: github.com/D4Vinci/Scrapling (PyPI: scrapling). Only use on sites you have permission to scrape.--- # Scrapling Skill **Source:** https://github.com/D4Vinci/Scrapling (open source, MIT-like license) **PyPI:** `scrapling` — install before first use (see below) > ⚠️ Only scrape sites you have permission to access. Respect `robots.txt` and Terms of Service. Do not use stealth modes to bypass paywalls or access restricted content without authorization. ## Installation (one-time, confirm with user before running) ```bash pip install scrapling[all] patchright install chromium # required for stealth/dynamic modes ``` - `scrapling[all]` installs `patchright` (a stealth fork of Playwright, bundled as a PyPI package — not a typo), `curl_cffi`, MCP server deps, and IPython shell. - `patchright install chromium` downloads Chromium (~100 MB) via patchright's own installer (same mechanism as `playwright install chromium`). - Confirm with user before running — installs ~200 MB of dependencies and browser binaries. ## Script `scripts/scrape.py` — CLI wrapper for all three fetcher modes. ```bash # Basic fetch (text output) python3 ~/skills/scrapling/scripts/scrape.py <url> -q # CSS selector extraction python3 ~/skills/scrapling/scripts/scrape.py <url> --selector ".class" -q # Stealth mode (Cloudflare bypass) — only on sites you're authorized to access python3 ~/skills/scrapling/scripts/scrape.py <url> --mode stealth -q # JSON output python3 ~/skills/scrapling/scripts/scrape.py <url> --selector "h2" --json -q ``` ## Fetcher Modes - **http** (default) — Fast HTTP with browser TLS fingerprint spoofing. Most sites. - **stealth** — Headless Chrome with anti-detect. For Cloudflare/anti-bot. - **dynamic** — Full Playwright browser. For heavy JS SPAs. ## When to Use Each Mode - `web_fetch` returns 403/429/Cloudflare challenge → use `--mode stealth` - Page content requires JS execution → use `--mode dynamic` - Regular site, just need text/data → use `--mode http` (default) ## Python Inline Usage For custom logic beyond the CLI, write inline Python. See `references/patterns.md` for: - Adaptive scraping (`auto_save` / `adaptive` — saves element fingerprints locally) - Session/cookie handling - Async usage - XPath, find_similar, attribute extraction ## Notes - **MCP server** (`scrapling mcp`): starts a local network service for AI-native scraping. Only start if explicitly needed and trusted — it exposes a local HTTP server. - **`auto_save=True`**: persists element fingerprints to disk for adaptive re-scraping. Creates local state in working directory. - Stealth/dynamic modes use Chromium headless — no `xvfb-run` needed. - For large-scale crawls, use the Spider API (see Scrapling docs).

Frequently asked questions