Robots Sitemap Validator

Check local robots.txt and sitemap.xml files for crawl-blocking mistakes. Use when someone asks to validate a sitemap, inspect robots.txt, or confirm whether local crawl files are safe to publish.

JustHandled Labs 1.1 KB Updated

File contents

Robots Sitemap Validator

Detect common local robots.txt and sitemap.xml mistakes that can hurt crawling. This detector does not fetch, submit, or change live sites.

Workflow

  1. Run scripts/scan_robots_sitemap.py with a folder, files, or --stdin.
  2. Review findings by rule id, severity, file, and line.
  3. Treat missing sibling sitemap files as review flags when robots.txt references them.

Guardrails

  • Read local files or stdin only.
  • Do not fetch live URLs or submit sitemaps.
  • Do not modify robots.txt or sitemap files.

Verification

Run tests/test_robots_sitemap.py. Risky fixture must produce findings and clean fixture must produce zero findings.

Commercial Terms

Price: $0 / 0 credits. Free commercial use for personal or internal team workflows. Do not resell or redistribute the package as-is.

justhandledlabs/skills/blob/main/plugins/robots-sitemap-validator/skills/robots-sitemap-validator/SKILL.md

Frequently asked questions

npx skillmds add heather/robots-sitemap-validator