Scrape Website

Scrape a website

Trigger

Extracting content from a single web page — an article, product page, documentation, or any URL.

Workflow

1. Verify anakin-cli is authenticated by running anakin status.
2. Scrape the URL in the requested format:
   • Markdown (default): anakin scrape "<url>" -o output.md
   • Structured JSON: anakin scrape "<url>" --format json -o output.json
   • Full raw response: anakin scrape "<url>" --format raw -o output.json
3. For JavaScript-heavy or single-page app sites, add the --browser flag.
4. For geo-targeted content, add --country <code>.
5. If the scrape times out, increase with --timeout <seconds>.
6. Read the output file and present results to the user.

Commands

# Clean readable text (default)
anakin scrape "<url>" -o output.md

# Structured data
anakin scrape "<url>" --format json -o output.json

# Full API response with HTML and metadata
anakin scrape "<url>" --format raw -o output.json

# JavaScript-heavy or single-page app sites
anakin scrape "<url>" --browser -o output.md

# Geo-targeted scraping
anakin scrape "<url>" --country gb -o output.md

# Custom timeout for slow pages
anakin scrape "<url>" --timeout 300 -o output.md

Guardrails

• Always quote URLs to prevent shell interpretation of ?, &, # characters.
• Default to markdown format unless the user asks for structured data or raw output.
• Use --browser only when a standard scrape returns empty or incomplete content.
• On 429 errors, wait before retrying rather than looping immediately.

Output

• Scraped content in the requested format
• File path where results were saved