Results for “scraping”

Automates Chrome/Chromium via CDP with accessibility-tree snapshots and element refs, covering web pages, Electron apps, Slack, and cloud browsers.

Automate real browsers from the terminal for navigation, form filling, screenshots, data extraction, and UI-flow debugging using a CLI wrapper around Playwright.

Automates browser interactions — navigating pages, filling forms, clicking buttons, taking screenshots, extracting data, and running assertions — using a native Rust CLI.

Guides users through creating, testing, and deploying browser automation functions with the Stagehand CLI, including interactive site exploration, coding, and deployment to Browserbase.

Navigate a Taobao or Tmall shop's catalog page to extract paginated product listings including itemId, title, and image URL.

Extracts clean Markdown content from any website URL using the BrowserAct Web Search Scraper API, with automatic retry and error handling.

Reduces a webpage to a structural skeleton by tokenizing content in the browser and applying LLM reasoning to collapse repeated patterns.

Crawls websites and saves content from multiple pages as local markdown files using the Tavily CLI, with options for depth, breadth, path filtering, and semantic focus.

Controls a Chrome browser through the AIPex extension and MCP bridge, enabling navigation, clicking, form filling, screenshots, tab management, and content downloads.

Test local web applications by writing native Python Playwright scripts, with helpers for server lifecycle management and a reconnaissance-then-action pattern for dynamic UIs.

Exports cookies from a local Chrome browser and saves them into a Browserbase persistent context, enabling authenticated browsing sessions via the browse CLI.

Fetch customer reviews for a Taobao or Tmall product by itemId, returning reviewer name, date, purchased variant, review text, and photo URLs.

Scrapes Instagram posts by hashtag, returning media items with captions, like/comment counts, media URLs and user info from the hashtag explore feed.

Extracts clean markdown content from web pages using the Defuddle CLI, removing clutter and navigation to save tokens. Prefer over WebFetch for reading or analyzing standard web pages.

Discovers and lists all URLs on a website without extracting content, using the Tavily CLI. Faster than crawling, with options for depth, filtering, and natural language instructions.

Automate web browser interactions using natural language via CLI commands. Supports local and remote Browserbase sessions with CAPTCHA solving, residential proxies, and session persistence.

Extracts full detail data from a single Goofish (闲鱼) second-hand item page, including title, price, seller info, description, images, tags, and want-count.

Scrapes posts from an Instagram user's profile feed including captions, media URLs, like/comment counts, timestamps, and location tags using cursor-based pagination.

Fetch Xiaohongshu (RedNote) note details and comments by note ID, returning title, description, author info, engagement stats, tags, and paginated comment list.

Extracts complete metadata from a TikTok video page by reading SSR-embedded data, including author profile, engagement stats, music info, hashtags, and slideshow images.

Guided multi-page redesign of an existing website through a four-phase pipeline — extract, direct, prototype, and migrate. Tracks progress incrementally per page so redesigns are resumable.

Deep-crawl any website from start URLs, returning per-page LLM-ready text, markdown, or HTML with metadata and in-scope outbound links.

Control a headless browser to navigate pages, click elements, fill forms, take screenshots, record video, and execute JavaScript using Playwright and inference.sh.

Fetches any URL via Chrome CDP and converts HTML to clean markdown, supporting auto-capture and wait-for-user modes for login-required pages.

Retrieves HTML or JSON from static pages via the Browserbase Fetch API, returning status codes, headers, and content without a full browser session. Supports redirects, proxies, and insecure SSL options.

Extract customer reviews from e-commerce product pages, returning reviewer name, rating, date, title, body, verified purchase status, and helpful votes. Supports Amazon, WooCommerce, Shopify, and generic sites with pagination.

Searches for existing content pages containing a specific block in AEM Edge Delivery Services, reporting URLs with occurrences and variants to help identify test content during development.

Controls a live browser session on scraped pages to click, fill forms, navigate flows, and extract data using natural language prompts or code, including support for authenticated sessions via profiles.

Operate Playwright for browser automation end to end: author and debug E2E test suites (robust locators, network interception and mocking, parallel workers, accessibility snapshot checks), wire them into CI, and drive headless browsing and scraping with an extract -> validate -> save loop. Use when writing, running, fixing, or scraping with Playwright, when a Playwright CI failure or JSON report needs triage, or when the bundled pwrun script should analyze a run. Do not use for QA strategy or test framework selection (route to qa-methodology), for frontend component or architecture design (route to frontend-engineering), or for Cloudflare/DDoS-GUARD challenge bypass (use flaresolverr).

Extract paginated product listings from e-commerce category pages, search results, or keyword searches with filters like price, brand, rating, and stock status. Supports Amazon, eBay, Walmart, Shopify, WooCommerce, and Google Shopping.

Fetch any URL and convert it to clean markdown using a Chrome CDP-based CLI with site-specific adapters for X/Twitter, YouTube, Hacker News, and generic pages.

Import a single webpage from any URL into canonical EDS block format — structured HTML that authors edit in DA. Scrapes the page, analyzes structure, maps to existing blocks, and generates HTML for immediate local preview.

Captures a spatial hierarchy of rendered DOM elements from any webpage via Playwright CLI, returning an LLM-friendly text tree, structured JSON tree, and a node map with CSS selectors and overlay metadata.

Scrapes posts from a Facebook group given a group URL, sort order, and desired count, returning structured post metadata including post_id, permalink, author, timestamp, body text, media, reactions, comments, and shares.

Drive Moli (`moli`), Lexmount's open-source headless browser for AI agents, built around on-demand rendering: real JavaScript, DOM, and CSS by default, with layout and pixels computed only when explicitly requested via `--layout`. Use when the user wants to fetch/extract a live JavaScript-rendered page as Markdown/HTML/JSON/semantic-tree, capture a screenshot or PDF, run a small bounded crawl, start a CDP/WebDriver automation server for Playwright/Puppeteer, replace a Chromium/ChromeDriver dependency, or diagnose readiness/network/frame issues on a rendered page. Triggers on: "moli fetch", "moli serve", "headless browser for agents", "on-demand rendering browser", "CDP server without Chrome", "structure-first web scraping", "Lexmount browser", "moli-webfetch", "moli-cdp-server".