# Karvey Scrape

> karvey-scrape

- Skill: `mauricioquezadahaintech/karvey-scrape` (Agent Skill)
- Install (CLI): `npx skillmds@latest add mauricioquezadahaintech/karvey-scrape`
- Raw SKILL.md: https://api.skillmd.com/api/skills/mauricioquezadahaintech/karvey-scrape/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: MauricioQuezadaHaintech (https://skillmd.com/u/mauricioquezadahaintech)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/mauricioquezadahaintech/karvey-scrape

---


# karvey-scrape

**CROSS-CUTTING SKILL of the Karvey™ Method.** It is a **support layer, NOT a phase**. It does not change `spec.json:phase` or advance the method's flow. It can be invoked from any phase when you need to extract data from a web page and, optionally, leave that extraction codified as a reusable artifact.

Inspired by gstack's `/scrape` + `/skillify`: it combines a **web extractor** (quick prototype) with a **skill codifier** (reusable script + test + fixture).

## Purpose

- **Prototype**: on the first call, extract the requested data from the target page. When the content is dynamic or requires interaction (JS, login, pagination, scroll), use the available browse runtime; for static HTML, `curl`/`fetch` + parsing is enough.
- **Codify (`--codify` / skillify)**: synthesize from the prototype a **reusable script + test + fixture**, run the test, and **ask for confirmation before committing**.
- **Result**: repeatable, fast extractions for recurring intents, without having to rediscover selectors or logic each time.

## Steps

### 1. Prototype (first call)

1. Receive the `[url]` and the description of the data to extract.
2. Determine the nature of the page:
   - **Static** → download HTML and parse (CSS/XPath selectors, scoped regex, embedded JSON).
   - **Dynamic / interactive** → use the browse runtime (navigate, wait for selectors, scroll, pagination, login if applicable).
3. Extract the requested data and present it to the user in a structured format (JSON/table).
4. Record the relevant details of the extraction: URL, selectors/queries, navigation steps, output format. This is the basis for codification.

### 2. Codify (`--codify` / skillify)

When `--codify` is passed (or the user asks to "codify skill" / "skillify"):

1. **Synthesize the reusable script** from the prototype: parameterize URL/inputs, encapsulate selectors and navigation steps, define the output contract.
2. **Generate a test** that validates the extraction against a **fixture** (snapshot of the HTML/response captured in the prototype), so the test is deterministic and does not depend on the network.
3. **Run the test** and adjust until it passes.
4. **Ask for explicit confirmation before committing** the script + test + fixture.

## Rules

- **Respect sites' terms of use and `robots.txt`.** Do not use this skill for abusive scraping, evasion of protections, or unauthorized mass collection.
- **Does not advance the phase.** It's a support layer; never modify `spec.json:phase` from here.
- Keep fixtures scoped and tests deterministic (no network calls in the test).
- Do not commit anything without the user's confirmation.

---
*Part of the Karvey™ Method — © HainTech, by Mauricio Quezada Ibáñez · Apache 2.0 · see `karvey/LICENSE` and `karvey/TRADEMARK.md`.*