Results for “web-scraping”

Automate Diffbot data extraction and analysis through Composio's Diffbot toolkit via Rube MCP.

Crypto wallet, token swaps, cross-chain bridges, and access to paid external services (search, image gen, web scraping, AI, and more) via x402 payments.

Discovers and lists all URLs on a website without extracting content, using the Tavily CLI. Faster than crawling, with options for depth, filtering, and natural language instructions.

Extract official websites and social media profiles (LinkedIn, Facebook, X/Twitter, Instagram, YouTube, TikTok) from a company name or website URL using automated browser scripts.

Automate web browser interactions using natural language via CLI commands. Supports local and remote Browserbase sessions with CAPTCHA solving, residential proxies, and session persistence.

Manage Browserbase platform resources, Functions, and API workflows through the `browse` CLI.

Converts documentation websites, GitHub repositories, and PDF files into AI-ready skills for multiple LLM platforms.

Trigger: chrome-devtools, browser automation, headless browser, web scraping, inspect element, chrome debugger, inspect page, network request, console logs. Scope: Interact with and automate headless Chrome via Chrome DevTools Protocol (CDP) for testing, debugging, and scraping. Boundary: Do not use for macOS native desktop GUI automation (use peekaboo instead).

Control browsers via a CLI tool that runs sandboxed JavaScript scripts for automation, testing, and data extraction.

Create and deploy browser automation functions with the Stagehand CLI, from interactive site exploration to production deployment.

Automates Browserbase Tool operations through Composio's Rube MCP integration, including tool discovery, connection management, and execution.

Search, browse, and analyze content on Xiaohongshu (Little Red Book) including feeds, note details, comments, and user profiles.

Scrape web pages using Scrapling with anti-bot bypass (like Cloudflare Turnstile), stealth headless browsing, spiders framework, adaptive scraping, and JavaScript rendering. Use when asked to scrape, crawl, or extract data from websites; web_fetch fails; the site has anti-bot protections; write Python code to scrape/crawl; or write spiders.

Automates Parsehub data extraction tasks through Composio's Parsehub toolkit via Rube MCP, with tool discovery and connection management.

Automate Zyte API operations through Composio's Rube MCP integration, including tool discovery, connection management, and execution.

Automates Scrapegraph AI operations through Composio's toolkit via Rube MCP, with tool discovery and connection management.

Automates browser tasks including navigation, data extraction, screenshots, form filling, and session management via a CLI tool.

Monitors Xianyu (goofish.com) for newly listed items matching keywords, price ranges, and filters, with automated scanning and Discord notifications.

Reposts external blog articles into StaticFlow with style-aware translation, source normalization, image ingestion, and verified publishing.

Extracts business contact details from Google Maps search results and place detail pages, then visits each business website to collect emails, phone numbers, and social media profiles.

Fetches comments from an Instagram post including comment text, username, timestamp, like count, and reply count using cursor-based pagination.

Extracts structured data from YouTube search results, including videos, shorts, channels, and playlists, using the BrowserAct API.

Parses betting odds HTML tables with Java and Jsoup, extracting company names, odds values, and change timestamps using specific CSS selectors.

Extracts clean markdown or text from one or more URLs using the Tavily CLI, with support for JavaScript-rendered pages and query-focused chunking.

Turns any website's data extraction or operation needs into reusable Agent-callable Skill packages by exploring API endpoints or DOM methods, then generating SKILL.md and Python scripts.

Automate real browsers from the terminal for navigation, form filling, screenshots, data extraction, and UI-flow debugging using a CLI wrapper around Playwright.

Extracts YouTube video transcripts and metadata (title, publisher, likes) via the BrowserAct API without CAPTCHA or IP restrictions.

Automates browser interactions — navigating pages, filling forms, clicking buttons, taking screenshots, extracting data, and running assertions — using a native Rust CLI.

Guides users through creating, testing, and deploying browser automation functions with the Stagehand CLI, including interactive site exploration, coding, and deployment to Browserbase.

Import a single webpage from any URL into canonical EDS block format — structured HTML that authors edit in DA. Scrapes the page, analyzes structure, maps to existing blocks, and generates HTML for immediate local preview.

Captures a spatial hierarchy of rendered DOM elements from any webpage via Playwright CLI, returning an LLM-friendly text tree, structured JSON tree, and a node map with CSS selectors and overlay metadata.

Builds reliable browser automation skills through iterative experimentation, running an inner agent to browse sites and improving navigation instructions until tasks pass consistently.

Navigate a Taobao or Tmall shop's catalog page to extract paginated product listings including itemId, title, and image URL.

Extracts structured business data from Google Maps, including names, categories, contact info, ratings, and addresses, using the BrowserAct API.

Extracts structured article details and full content from Zhihu search results via the BrowserAct API, with keyword and date filtering.

Acquires resources from URLs, PDFs, and local files, categorizes them, and organizes content and media into a structured workspace with a lowercase assets folder.