Site Capture & Audit
What this does
Captures a website to local files, then turns that capture into a prioritized,
90-day quick-win audit. The capture step saves raw HTML, a cleaned
text/markdown version of each page, and the site's sitemap.xml + robots.txt.
The audit step grades the site against a P0/P1/P2 rubric covering technical
SEO (indexability, canonicals, redirects), on-page (titles, meta, headings,
schema, internal links), content depth, Core Web Vitals, and conversion — and
emits a ranked fix list with effort/impact.
When to use it
- Onboarding a new client/site and you need a fast, evidence-based baseline.
- Auditing technical + on-page SEO before a redesign, migration, or content push.
- Producing a defensible "first 90 days" quick-win list for a stakeholder.
How to use it
- Capture the site with the bundled script:
python scripts/capture.py https://example.com --max 25 --out ./capture
It fetches the homepage, discovers more URLs via sitemap.xml / robots.txt,
saves raw HTML to <out>/raw-html/, cleaned markdown to <out>/clean-md/,
and <out>/sitemaps/ (sitemaps + robots.txt). See the script --help.
- Read the cleaned pages in
<out>/clean-md/ and skim the raw HTML <head>
blocks for titles, meta, canonical, robots directives, and JSON-LD.
- Work through
references/audit-checklist.md
in priority order — P0 indexability first, then P1 on-page, then P2 depth +
CWV + conversion. Record evidence (the actual title, the missing canonical,
the redirect chain) for each finding.
- For each finding assign: severity (P0/P1/P2), impact (high/med/low),
effort (S/M/L), and the concrete fix.
- Write the audit as a ranked table grouped by priority, with a short
executive summary and a 90-day sequencing (do P0 in weeks 1–2, etc.).
- For schema gaps found in the audit, hand off to a
schema-org-sprint skill;
for content gaps, hand off to a content-brief skill.
Inputs
- A base URL (the site to capture and audit).
- Optional: a max page count (
--max) and output dir (--out).
- Optional: access to Google Search Console / analytics to ground impact (the
audit works from the capture alone, but real query/traffic data sharpens it).
Output
- A capture folder:
raw-html/, clean-md/, sitemaps/.
- A prioritized 90-day quick-win audit (markdown): executive summary, ranked
P0/P1/P2 findings with evidence + fix + effort/impact, and a week-by-week plan.
Notes & constraints
- The capture script uses only the Python standard library (
urllib) plus
optional BeautifulSoup; if BeautifulSoup is absent it falls back to a
regex tag-strip, so it runs on a stock Python 3. It sets a descriptive
User-Agent and adds a small delay between requests to be polite.
- It only fetches when you invoke it with a URL. Respect
robots.txt,
rate limits, and the site owner's terms; capture sites you're authorized to.
- The audit is an expert review against the rubric, not a substitute for live
Core Web Vitals field data or full crawl tooling — call those out where they'd
change a conclusion.
- Use forward-slash paths.
1---2name: site-capture-audit3description: Captures a website (raw HTML), cleans pages to markdown, pulls sitemaps and robots.txt, then produces a prioritized 90-day quick-win SEO audit. Use when onboarding a new site, baselining technical SEO and on-page health, or building a ranked P0/P1/P2 fix list across indexability, on-page, content, and conversion.4license: MIT5---67# Site Capture & Audit89## What this does1011Captures a website to local files, then turns that capture into a prioritized,1290-day quick-win audit. The capture step saves raw HTML, a cleaned13text/markdown version of each page, and the site's `sitemap.xml` + `robots.txt`.14The audit step grades the site against a P0/P1/P2 rubric covering technical15SEO (indexability, canonicals, redirects), on-page (titles, meta, headings,16schema, internal links), content depth, Core Web Vitals, and conversion — and17emits a ranked fix list with effort/impact.1819## When to use it2021- Onboarding a new client/site and you need a fast, evidence-based baseline.22- Auditing technical + on-page SEO before a redesign, migration, or content push.23- Producing a defensible "first 90 days" quick-win list for a stakeholder.2425## How to use it26271. Capture the site with the bundled script:28 `python scripts/capture.py https://example.com --max 25 --out ./capture`29 It fetches the homepage, discovers more URLs via `sitemap.xml` / `robots.txt`,30 saves raw HTML to `<out>/raw-html/`, cleaned markdown to `<out>/clean-md/`,31 and `<out>/sitemaps/` (sitemaps + robots.txt). See the script `--help`.322. Read the cleaned pages in `<out>/clean-md/` and skim the raw HTML `<head>`33 blocks for titles, meta, canonical, robots directives, and JSON-LD.343. Work through [`references/audit-checklist.md`](references/audit-checklist.md)35 in priority order — P0 indexability first, then P1 on-page, then P2 depth +36 CWV + conversion. Record evidence (the actual title, the missing canonical,37 the redirect chain) for each finding.384. For each finding assign: **severity** (P0/P1/P2), **impact** (high/med/low),39 **effort** (S/M/L), and the concrete **fix**.405. Write the audit as a ranked table grouped by priority, with a short41 executive summary and a 90-day sequencing (do P0 in weeks 1–2, etc.).426. For schema gaps found in the audit, hand off to a `schema-org-sprint` skill;43 for content gaps, hand off to a content-brief skill.4445## Inputs4647- A base URL (the site to capture and audit).48- Optional: a max page count (`--max`) and output dir (`--out`).49- Optional: access to Google Search Console / analytics to ground impact (the50 audit works from the capture alone, but real query/traffic data sharpens it).5152## Output5354- A capture folder: `raw-html/`, `clean-md/`, `sitemaps/`.55- A prioritized 90-day quick-win audit (markdown): executive summary, ranked56 P0/P1/P2 findings with evidence + fix + effort/impact, and a week-by-week plan.5758## Notes & constraints5960- The capture script uses only the Python standard library (`urllib`) plus61 **optional** BeautifulSoup; if BeautifulSoup is absent it falls back to a62 regex tag-strip, so it runs on a stock Python 3. It sets a descriptive63 User-Agent and adds a small delay between requests to be polite.64- It only fetches when **you** invoke it with a URL. Respect `robots.txt`,65 rate limits, and the site owner's terms; capture sites you're authorized to.66- The audit is an expert review against the rubric, not a substitute for live67 Core Web Vitals field data or full crawl tooling — call those out where they'd68 change a conclusion.69- Use forward-slash paths.
sujanbhuiyan