Web Crawler

Bounded, privacy-aware website and sitemap crawler with AgentConfig-managed egress, TLS, output confinement, and optional Arbora document ingestion. Use when fetching one page, crawling a site or sitemap, or ingesting approved web content into Arbora.

Knuckles-Team Updated 5 repo stars

File contents

knuckles-team/universal-skills commit 7389accd32

Frequently asked questions

npx get-skill add knuckles-team/web-crawler-2