How do I install the SEO Crawl skill?

Run npx skillmds@latest add bestdeejay-design/seo-crawl in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is SEO Crawl safe to use?

This skill has not completed SkillMD's automated safety review yet. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with SEO Crawl?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is SEO Crawl free to use?

Yes. Installing skills from SkillMD is free. This skill is licensed under MIT.

Who published SEO Crawl?

bestdeejay-design (@bestdeejay-design) published this skill. Their other Agent Skills are listed on their SkillMD profile.

SEO Crawl by bestdeejay-design — Agent Skill

Name: SEO Crawl
Author: bestdeejay-design

SEO Crawl

Crawlability and site structure for SEO: robots.txt, noindex, canonical, redirect chains, sitemap.xml; internal linking, click depth, orphan pages, URL hierarchy, breadcrumbs; competitor comparison (gaps, overlapping keywords, content gaps). Commands documented in commands/. Triggers: 'seo crawl', 'crawlability', 'robots txt', 'site structure', 'URL structure', 'competitor seo', 'индексация', 'структура сайта', 'конкуренты seo', 'краулинг'.

bestdeejay-design fdda226 6 files · 12.5 KB Updated Sep 17, 2026

File contents

SEO Crawl — crawlability, structure & competitors

Use this skill for the crawl/visibility layer of SEO: whether search engines can reach the site, how it is internally linked, and how it compares to competitors.

Commands (documented in `commands/`)

Command	What it does
`seo-crawl`	Crawlability: robots.txt, noindex, canonical, redirect chains, sitemap.xml
`seo-structure`	Internal linking, click depth, orphan pages, URL hierarchy, breadcrumbs
`seo-compare`	Competitor comparison: gaps, overlapping keywords, content gaps (URL mode)

Modes: URL (external sites) and File (local projects); seo-compare is URL-only.

When to use

User asks for "seo crawl", "crawlability", "robots txt", "site structure", "URL structure", "competitor seo", "индексация", "структура сайта".
Indexation, linking structure or competitor gaps are the task.

Do NOT use

For the full technical audit / CWV / reports / fixes — that is seo-audit.
For schema.org / meta tags — that is seo-schema.
For content / keywords / headings / images — that is seo-content.

Frequently asked questions