A high-performance engineering tool for deep web scraping. It uses a containerized Docker + Crawlee (Playwright) environment to penetrate protections on complex websites like YouTube and X/Twitter, providing "interception-level" raw data.
Requirements
Docker: Must be installed and running on the host machine.
Image: Build the environment with the tag clawd-crawlee.
Simply copy the skills/deep-scraper directory into your skills/ folder. Ensure the Dockerfile remains within the skill directory for self-contained deployment.
Standard Interface (CLI)
docker run -t --rm -v $(pwd)/skills/deep-scraper/assets:/usr/src/app/assets clawd-crawlee node assets/main_handler.js [TARGET_URL]
Output Specification (JSON)
The scraping results are printed to stdout as a JSON string:
status: SUCCESS | PARTIAL | ERROR
type: TRANSCRIPT | DESCRIPTION | GENERIC
videoId: (For YouTube) The validated Video ID.
data: The core text content or transcript.
Core Rules
ID Validation: All YouTube tasks MUST verify the Video ID to prevent cache contamination.
Privacy: Strictly forbidden from scraping password-protected or non-public personal information.
Alpha-Focused: Automatically strips ads and noise, delivering pure data optimized for LLM processing.
1---2name: deep-scraper3description: Skill: deep-scraper4---5# Skill: deep-scraper67## Overview8A high-performance engineering tool for deep web scraping. It uses a containerized Docker + Crawlee (Playwright) environment to penetrate protections on complex websites like YouTube and X/Twitter, providing "interception-level" raw data.910## Requirements111. **Docker**: Must be installed and running on the host machine.122. **Image**: Build the environment with the tag `clawd-crawlee`.13 * Build command: `docker build -t clawd-crawlee skills/deep-scraper/`1415## Integration Guide16Simply copy the `skills/deep-scraper` directory into your `skills/` folder. Ensure the Dockerfile remains within the skill directory for self-contained deployment.1718## Standard Interface (CLI)19```bash20docker run -t --rm -v $(pwd)/skills/deep-scraper/assets:/usr/src/app/assets clawd-crawlee node assets/main_handler.js [TARGET_URL]21```2223## Output Specification (JSON)24The scraping results are printed to stdout as a JSON string:25- `status`: SUCCESS | PARTIAL | ERROR26- `type`: TRANSCRIPT | DESCRIPTION | GENERIC27- `videoId`: (For YouTube) The validated Video ID.28- `data`: The core text content or transcript.2930## Core Rules311. **ID Validation**: All YouTube tasks MUST verify the Video ID to prevent cache contamination.322. **Privacy**: Strictly forbidden from scraping password-protected or non-public personal information.333. **Alpha-Focused**: Automatically strips ads and noise, delivering pure data optimized for LLM processing.
Run npx skillmds@latest add demerzels-lab/deep-scraper in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.
Skill: deep-scraper It is listed under Coding & Dev Tools on SkillMD.
This skill has not completed SkillMD's automated safety review yet. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.
This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.
Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.
Demerzels-lab (@demerzels-lab) published this skill. Their other Agent Skills are listed on their SkillMD profile.
Demerzels-lab