Results for “ai-crawlers”

Rastreie sites inteiros usando a API /crawl do Cloudflare Browser Rendering. Inicia jobs de rastreamento assíncronos, monitora a conclusão e salva os resultados como arquivos markdown. Útil para ingerir sites de documentação, bases de conhecimento ou qualquer conteúdo web no contexto do seu projeto. Requer variáveis de ambiente CLOUDFLARE_ACCOUNT_ID e CLOUDFLARE_API_TOKEN.

Crawl entire websites using Cloudflare Browser Rendering /crawl API. Initiates async crawl jobs, polls for completion, and saves results as markdown files. Useful for ingesting documentation sites, knowledge bases, or any web content into your project context. Requires CLOUDFLARE_ACCOUNT_ID and CLOUDFLARE_API_TOKEN environment variables.

Crawls websites and saves content from multiple pages as local markdown files using the Tavily CLI, with options for depth, breadth, path filtering, and semantic focus.

Search, scrape, and interact with the web via the Firecrawl CLI. Use this skill whenever the user wants to search the web, find articles, research a topic, look something up online, scrape a webpage, grab content from a URL, get data from a website, crawl documentation, download a site, or interact with pages that need clicks or logins. Also use when they say "fetch this page", "pull the content from", "get the page at https://", or reference external websites. This provides real-time web search with full page content and interact capabilities — beyond what Claude can do natively with built-in tools. Do NOT trigger for local file operations, git commands, deployments, or code editing tasks.