Algo SEO Crawl

Implement a web crawler pipeline covering URL discovery, fetching, parsing, and storage. Use this skill when the user needs to build a site crawler, audit website structure, or collect web data systematically — even if they say 'scrape a website', 'crawl all pages', or 'site audit spider'.

asgard-ai-platform Updated 244 repo stars

File contents

asgard-ai-platform/skills commit 6d4b36a4f8

Frequently asked questions

npx get-skill add asgard-ai-platform/algo-seo-crawl