GEO Technical SEO Audit
Purpose
Technical SEO forms the foundation of both traditional search visibility and AI search citation. A technically broken site cannot be crawled, indexed, or cited by any platform. This skill audits 8 categories of technical health with specific attention to GEO requirements — most critically, server-side rendering (AI crawlers do not execute JavaScript) and AI crawler access (many sites inadvertently block AI crawlers in robots.txt).
How to Use This Skill
- Collect the target URL (homepage + 2-3 key inner pages)
- Fetch each page using curl/WebFetch to get raw HTML and HTTP headers
- Run through each of the 8 audit categories below
- Score each category using the rubric
- Generate GEO-TECHNICAL-AUDIT.md with results
Category 1: Crawlability (15 points)
1.1 robots.txt Validity
- Fetch
https://[domain]/robots.txt
- Check for syntactic validity: proper
User-agent, Allow, Disallow directives
- Check for common errors: missing User-agent, wildcards blocking important paths, Disallow: / blocking entire site
- Verify XML sitemap is referenced:
Sitemap: https://[domain]/sitemap.xml
1.2 AI Crawler Access (CRITICAL for GEO)
Check robots.txt for directives targeting these AI crawlers:
| Crawler |
User-Agent |
Platform |
| GPTBot |
GPTBot |
ChatGPT / OpenAI |
| Google-Extended |
Google-Extended |
Gemini / Google AI training |
| Googlebot |
Googlebot |
Google Search + AI Overviews |
| Bingbot |
bingbot |
Bing Copilot + ChatGPT (via Bing) |
| PerplexityBot |
PerplexityBot |
Perplexity AI |
| ClaudeBot |
ClaudeBot |
Anthropic Claude |
| Amazonbot |
Amazonbot |
Alexa / Amazon AI |
| CCBot |
CCBot |
Common Crawl (used by many AI models) |
| FacebookBot |
FacebookExternalHit |
Meta AI |
| Bytespider |
Bytespider |
TikTok / ByteDance AI |
| Applebot-Extended |
Applebot-Extended |
Apple Intelligence |
Scoring for AI crawler access:
- All major AI crawlers allowed: 5 points
- Some blocked but Googlebot + Bingbot allowed: 3 points
- GPTBot or PerplexityBot blocked: 1 point (significant GEO impact)
- Googlebot blocked: 0 points (fatal)
Important nuance: Blocking Google-Extended does NOT block Googlebot. Google-Extended only controls AI training data usage, not search indexing. However, blocking Google-Extended may reduce presence in AI Overviews. Recommend allowing Google-Extended unless there is a specific data licensing concern.
1.3 XML Sitemaps
- Fetch sitemap (check robots.txt for location, or try
/sitemap.xml, /sitemap_index.xml)
- Validate XML syntax
- Check for
<lastmod> dates (should be present and accurate)
- Count URLs — compare to expected number of indexable pages
- Check for sitemap index if large site (50,000+ URLs per sitemap max)
- Verify all sitemap URLs return 200 status codes (sample check)
1.4 Crawl Depth
- Homepage = depth 0. Check that all important pages are reachable within 3 clicks (depth 3)
- Pages at depth 4+ receive significantly less crawl budget and are less likely to be cited by AI
- Check internal linking: are key content pages linked from the homepage or main navigation?
1.5 Noindex Management
- Check for
<meta name="robots" content="noindex"> on pages that SHOULD be indexed
- Check for
X-Robots-Tag: noindex HTTP headers
- Common mistakes: noindex on paginated pages, category pages, or key landing pages
Category Scoring:
| Check |
Points |
| robots.txt valid and complete |
3 |
| AI crawlers allowed |
5 |
| XML sitemap present and valid |
3 |
| Crawl depth within 3 clicks |
2 |
| No erroneous noindex directives |
2 |
Category 2: Indexability (12 points)
2.1 Canonical Tags
- Every indexable page must have a
<link rel="canonical" href="..."> tag
- Canonical must point to itself (self-referencing) for the authoritative version
- Check for conflicting canonicals (canonical in HTML vs. HTTP header)
- Check for canonical chains (A canonicals to B, B canonicals to C — should be A to C)
2.2 Duplicate Content
- Check for www vs. non-www (both should resolve, one should redirect)
- Check for HTTP vs. HTTPS (HTTP should redirect to HTTPS)
- Check for trailing slash consistency (pick one pattern and redirect the other)
- Check for parameter-based duplicates (
?sort=price creating duplicate pages)
2.3 Pagination
- If paginated content exists, check for
rel="next" / rel="prev" (note: Google ignores these as of 2019, but Bing still uses them)
- Preferred: use
rel="canonical" on paginated pages pointing to a view-all page or the first page
- Ensure paginated pages are not noindexed if they contain unique content
2.4 Hreflang (international sites)
- Check for
<link rel="alternate" hreflang="xx"> tags
- Validate: reciprocal hreflang (if page A points to page B, B must point back to A)
- Validate: x-default fallback exists
- Check for language/region code validity (ISO 639-1 / ISO 3166-1)
2.5 Index Bloat
- Estimate number of indexed pages (check sitemap count, use
site:domain.com estimate)
- Compare indexed pages to actual valuable content pages
- Flag if indexed pages significantly exceed content pages (index bloat from thin/duplicate/parameter pages)
Category Scoring:
| Check |
Points |
| Canonical tags correct on all pages |
3 |
| No duplicate content issues |
3 |
| Pagination handled correctly |
2 |
| Hreflang correct (if applicable) |
2 |
| No index bloat |
2 |
Category 3: Security (10 points)
3.1 HTTPS Enforcement
- Site must load over HTTPS
- HTTP must redirect to HTTPS (301 redirect)
- No mixed content warnings (HTTP resources on HTTPS pages)
- SSL/TLS certificate must be valid and not expired
3.2 Security Headers
Check HTTP response headers for:
| Header |
Required Value |
Purpose |
Strict-Transport-Security |
max-age=31536000; includeSubDomains |
Forces HTTPS |
Content-Security-Policy |
Appropriate policy |
Prevents XSS |
X-Content-Type-Options |
nosniff |
Prevents MIME sniffing |
X-Frame-Options |
DENY or SAMEORIGIN |
Prevents clickjacking |
Referrer-Policy |
strict-origin-when-cross-origin or stricter |
Controls referrer data |
Permissions-Policy |
Appropriate restrictions |
Controls browser features |
Category Scoring:
| Check |
Points |
| HTTPS enforced with valid cert |
4 |
| HSTS header present |
2 |
| X-Content-Type-Options |
1 |
| X-Frame-Options |
1 |
| Referrer-Policy |
1 |
| Content-Security-Policy |
1 |
Category 4: URL Structure (8 points)
4.1 Clean URLs
- URLs should be human-readable:
/blog/seo-guide not /blog?id=12345
- No session IDs in URLs
- Lowercase only (no mixed case)
- Hyphens for word separation (not underscores)
- No special characters or encoded spaces
4.2 Logical Hierarchy
- URL path should reflect site architecture:
/category/subcategory/page
- Flat where appropriate — avoid unnecessarily deep nesting
- Consistent pattern across the site
4.3 Redirect Chains
- Check for redirect chains (A redirects to B redirects to C)
- Maximum 1 hop recommended (A redirects to C directly)
- Check for redirect loops
- All redirects should be 301 (permanent), not 302 (temporary), unless intentionally temporary
4.4 Parameter Handling
- URL parameters should not create duplicate indexable pages
- Use canonical tags or
robots.txt Disallow for parameter variations
- Configure parameter handling in Google Search Console and Bing Webmaster Tools
Category Scoring:
| Check |
Points |
| Clean, readable URLs |
2 |
| Logical hierarchy |
2 |
| No redirect chains (max 1 hop) |
2 |
| Parameter handling configured |
2 |
Category 5: Mobile Optimization (10 points)
Critical Context
As of July 2024, Google crawls ALL sites exclusively with mobile Googlebot. There is no desktop crawling. If your site does not work on mobile, it does not work for Google. Period.
5.1 Responsive Design
- Check for
<meta name="viewport" content="width=device-width, initial-scale=1">
- Content must not require horizontal scrolling on mobile
- No fixed-width layouts wider than viewport
5.2 Tap Targets
- Interactive elements (buttons, links) must be at least 48x48 CSS pixels
- Minimum 8px spacing between tap targets
- Check that navigation is usable on mobile
5.3 Font Sizes
- Base font size should be at least 16px
- No text requiring zoom to read
- Sufficient contrast ratio (WCAG AA: 4.5:1 for normal text, 3:1 for large text)
5.4 Mobile Content Parity
- All content visible on desktop must also be visible on mobile
- No hidden content behind "read more" toggles that Googlebot cannot expand (though Google has improved at expanding these as of 2025)
- Images and media must load on mobile
Category Scoring:
| Check |
Points |
| Viewport meta tag correct |
3 |
| Responsive layout (no horizontal scroll) |
3 |
| Tap targets appropriately sized |
2 |
| Font sizes legible |
2 |
Category 6: Core Web Vitals (15 points)
2026 Metrics and Thresholds
Core Web Vitals use the 75th percentile of real user data (field data) as the benchmark. Lab data is useful for debugging but field data determines the ranking signal.
| Metric |
Good |
Needs Improvement |
Poor |
Notes |
| LCP (Largest Contentful Paint) |
< 2.5s |
2.5s - 4.0s |
> 4.0s |
Measures loading — time until largest visible element renders |
| INP (Interaction to Next Paint) |
< 200ms |
200ms - 500ms |
> 500ms |
Replaced FID in March 2024. Measures ALL interactions, not just first |
| CLS (Cumulative Layout Shift) |
< 0.1 |
0.1 - 0.25 |
> 0.25 |
Measures visual stability — unexpected layout movements |
How to Assess Without CrUX Data
When real user data is unavailable, estimate from page characteristics:
- LCP: Check largest above-fold element. Is it an image (check size/format)? Is it text (check web font loading)? Server response time (TTFB)?
- INP: Check for heavy JavaScript on page. Long tasks (>50ms) block interactivity. Check for third-party scripts.
- CLS: Check for images without explicit width/height. Check for dynamically inserted content above the fold. Check for web fonts causing layout shift (FOUT/FOIT).
Common LCP Fixes
- Optimize hero images: WebP/AVIF format, correct sizing, preload with
<link rel="preload">
- Reduce server response time (TTFB < 800ms)
- Eliminate render-blocking CSS/JS
- Preconnect to critical third-party origins
Common INP Fixes
- Break up long tasks (>50ms) into smaller chunks using
requestIdleCallback or scheduler.yield()
- Reduce third-party JavaScript
- Use
content-visibility: auto for off-screen content
- Debounce/throttle event handlers
Common CLS Fixes
- Always include
width and height attributes on images and videos
- Reserve space for ads and embeds with CSS
aspect-ratio or explicit dimensions
- Use
font-display: swap with size-adjusted fallback fonts
- Avoid inserting content above existing content after page load
Category Scoring:
| Check |
Points |
| LCP < 2.5s |
5 |
| INP < 200ms |
5 |
| CLS < 0.1 |
5 |
Category 7: Server-Side Rendering (15 points) — CRITICAL FOR GEO
Why SSR Is Mandatory for AI Visibility
AI crawlers (GPTBot, PerplexityBot, ClaudeBot, etc.) do NOT execute JavaScript. They fetch the raw HTML and parse it. If your content is rendered client-side by React, Vue, Angular, or any other JavaScript framework, AI crawlers see an empty page.
Even Googlebot, which does execute JavaScript, deprioritizes JS-rendered content due to the additional crawl budget required. Google processes JS rendering in a separate "rendering queue" that can delay indexing by days or weeks.
Detection Method
- Fetch the page with curl (no JavaScript execution):
curl -s [URL]
- Compare the raw HTML to the rendered DOM (via browser)
- If key content (headings, paragraphs, product info, article text) is MISSING from the curl output, the site relies on client-side rendering
What to Check
- Main content text: Is the article body / product description / page content in the raw HTML?
- Headings: Are H1, H2, H3 tags present in raw HTML?
- Navigation: Is the main navigation server-rendered?
- Structured data: Is JSON-LD in the raw HTML or injected by JavaScript?
- Meta tags: Are title, description, canonical, OG tags in the raw HTML?
- Internal links: Are navigation and content links in the raw HTML? (Critical for crawlability)
SSR Solutions to Recommend
| Framework |
SSR Solution |
| React |
Next.js (SSR/SSG), Remix, Gatsby (SSG) |
| Vue |
Nuxt.js (SSR/SSG) |
| Angular |
Angular Universal |
| Svelte |
SvelteKit |
| Generic |
Prerender.io (prerendering service), Rendertron |
Scoring Detail
- All key content server-rendered: 15 points
- Main content server-rendered but some elements JS-only: 10 points
- Critical content requires JS (product info, article text): 5 points
- Entire page is client-rendered (empty body in raw HTML): 0 points
Category Scoring:
| Check |
Points |
| Main content in raw HTML |
8 |
| Meta tags + structured data in raw HTML |
4 |
| Internal links in raw HTML |
3 |
Category 8: Page Speed & Server Performance (15 points)
8.1 Time to First Byte (TTFB)
- Target: < 800ms (ideally < 200ms)
- Measure with curl:
curl -o /dev/null -s -w 'TTFB: %{time_starttransfer}s\n' [URL]
- If TTFB > 800ms: check server location, caching, database queries, CDN usage
8.2 Resource Optimization
- Total page weight target: < 2MB (critical pages < 1MB)
- Check for uncompressed resources (gzip/brotli compression should be enabled)
- Check for unminified CSS and JavaScript
- Check for unused CSS/JS (can represent 50%+ of downloaded bytes on many sites)
8.3 Image Optimization
- Check image formats: WebP or AVIF preferred over JPEG/PNG
- Check for oversized images (images larger than display size)
- Check for lazy loading: images below fold should have
loading="lazy"
- Check for explicit dimensions (width/height attributes prevent CLS)
- Above-fold images should NOT be lazy loaded (harms LCP)
8.4 Code Splitting and Lazy Loading
- JavaScript should be code-split so each page only loads what it needs
- Check for large JavaScript bundles (> 200KB compressed is a warning, > 500KB is critical)
- Third-party scripts should load asynchronously (
async or defer)
- Check for render-blocking resources in
<head>
8.5 Caching
- Check
Cache-Control headers on static resources (images, CSS, JS)
- Static assets should have long cache times:
max-age=31536000 (1 year) with content-hashed filenames
- HTML pages should have shorter cache or
no-cache with validation (ETag or Last-Modified)
8.6 CDN Usage
- Check if static resources are served from a CDN (different domain or CDN-specific headers)
- For global audience, CDN is critical for consistent performance
- Check for CDN-specific headers:
CF-Ray (Cloudflare), X-Cache (AWS CloudFront), X-Served-By (Fastly)
Category Scoring:
| Check |
Points |
| TTFB < 800ms |
3 |
| Page weight < 2MB |
2 |
| Images optimized (format, size, lazy) |
3 |
| JS bundles reasonable (< 200KB compressed) |
2 |
| Compression enabled (gzip/brotli) |
2 |
| Cache headers on static resources |
2 |
| CDN in use |
1 |
Category 9: Agent-Readiness Signals (non-scoring)
These checks surface emerging AI agent compatibility signals. None contribute to the numeric score — they produce a pass or a recommendation. The underlying standards are either IETF drafts or early-adoption features; penalizing absence would be unfair.
9.1 RFC 8288 Link Headers (Service Discovery)
RFC 8288 (Web Linking) defines the HTTP Link: response header. Servers can use it to advertise related resources — API catalog, service docs, MCP server card — in a machine-readable way, without HTML parsing.
How to check: Capture all Link: response headers from the standard homepage fetch (no extra request).
What to look for:
- Parse
<url>; rel="relation-type" pairs.
- High-value rel types:
api-catalog (RFC 9609), describedby, service-doc, mcp-server-card.
When to surface a recommendation: Only for API-first sites (API docs linked in nav, /api/ or /developers/ paths, swagger/OpenAPI in sitemap). Omit this section entirely for standard business sites — absence is expected and not noteworthy.
| State |
Treatment |
Link: headers present, known rel types |
Informational — document what was found |
Link: headers present, unknown rel types |
Informational — note and explain |
| Absent, API-first site |
Recommendation — explain and suggest implementation |
| Absent, standard business site |
Omit — do not surface |
9.2 Markdown Content Negotiation
Checks if the server responds to Accept: text/markdown with Content-Type: text/markdown. Cloudflare's "Markdown for Agents" feature enables this — AI agents receive clean Markdown instead of HTML, eliminating boilerplate stripping and improving content extraction accuracy.
How to check: Send a GET to the homepage with Accept: text/markdown. This is one additional HTTP request per audit.
Evaluation:
- If response
Content-Type is text/markdown (or text/markdown; charset=utf-8): pass — note as a leading-edge capability.
- Otherwise: forward-looking recommendation, not a failure.
- If the request errors or returns non-200: skip and note the error. Do not penalize.
| State |
Treatment |
text/markdown returned |
Bonus — note as a leading-edge capability |
| Standard HTML returned |
Forward-looking recommendation |
| Request errors / non-200 |
Skip, note the error, do not penalize |
IndexNow Protocol
What It Is
IndexNow is an open protocol that allows websites to notify search engines instantly when content is created, updated, or deleted. Supported by Bing, Yandex, Seznam, and Naver. Google does NOT support IndexNow but monitors the protocol.
Why It Matters for GEO
ChatGPT uses Bing's index. Bing Copilot uses Bing's index. Faster Bing indexing means faster AI visibility on two major platforms.
Implementation Check
- Check for IndexNow key file:
https://[domain]/.well-known/indexnow-key.txt or similar
- Check if CMS has IndexNow plugin (WordPress: IndexNow plugin; many modern CMS platforms support it natively)
- If not implemented, recommend adding it with instructions
Overall Scoring
| Category |
Max Points |
Weight |
| Crawlability |
15 |
Core foundation |
| Indexability |
12 |
Core foundation |
| Security |
10 |
Trust signal |
| URL Structure |
8 |
Crawl efficiency |
| Mobile Optimization |
10 |
Google requirement |
| Core Web Vitals |
15 |
Ranking signal |
| Server-Side Rendering |
15 |
GEO critical |
| Page Speed & Server |
15 |
Performance |
| Total |
100 |
|
Non-scoring checks (Category 9) appear in the output under "Agent-Readiness Signals" and do not affect this total.
Score Interpretation
- 90-100: Excellent — technically sound for both traditional SEO and GEO
- 70-89: Good — minor issues to address but fundamentally solid
- 50-69: Needs Work — significant technical debt impacting visibility
- 30-49: Poor — major issues blocking crawling, indexing, or AI visibility
- 0-29: Critical — fundamental technical failures requiring immediate attention
Output Format
Generate GEO-TECHNICAL-AUDIT.md with:
# GEO Technical SEO Audit — [Domain]
Date: [Date]
## Technical Score: XX/100
## Score Breakdown
| Category | Score | Status |
|---|---|---|
| Crawlability | XX/15 | Pass/Warn/Fail |
| Indexability | XX/12 | Pass/Warn/Fail |
| Security | XX/10 | Pass/Warn/Fail |
| URL Structure | XX/8 | Pass/Warn/Fail |
| Mobile Optimization | XX/10 | Pass/Warn/Fail |
| Core Web Vitals | XX/15 | Pass/Warn/Fail |
| Server-Side Rendering | XX/15 | Pass/Warn/Fail |
| Page Speed & Server | XX/15 | Pass/Warn/Fail |
Status: Pass = 80%+ of category points, Warn = 50-79%, Fail = <50%
## AI Crawler Access
| Crawler | User-Agent | Status | Recommendation |
|---|---|---|---|
| GPTBot | GPTBot | Allowed/Blocked | [Action] |
| Googlebot | Googlebot | Allowed/Blocked | [Action] |
[Continue for all AI crawlers]
## Critical Issues (fix immediately)
[List with specific page URLs and what is wrong]
## Warnings (fix this month)
[List with details]
## Recommendations (optimize this quarter)
[List with details]
## Agent-Readiness Signals (non-scoring)
### RFC 8288 Link Headers (Service Discovery)
**Status:** Present / Absent / Not Applicable
<!-- If present: -->
| Relation Type | URL | Meaning |
|---|---|---|
| api-catalog | /.well-known/api-catalog | Machine-readable index of available APIs |
| mcp-server-card | /.well-known/mcp.json | MCP server capability declaration |
AI agents and API clients can discover your services without parsing HTML.
<!-- If absent, API-first site only: -->
**Informational Recommendation:** This site has API/developer-oriented content but no `Link:` headers advertising discoverable services.
Example: `Link: </.well-known/api-catalog>; rel="api-catalog"`
Relevant for: sites with public APIs, OpenAPI docs, or MCP server integrations.
Reference: RFC 8288, RFC 9609.
<!-- If absent, standard business site: omit this section entirely -->
### Markdown Content Negotiation
**Status:** Supported / Not Supported
**Test:** GET [url] with `Accept: text/markdown`
**Response Content-Type:** [value]
<!-- If supported: -->
This site serves clean Markdown to AI agents on request. AI crawlers that support content negotiation receive formatted text without HTML boilerplate.
<!-- If not supported: -->
**Forward-Looking Recommendation:** Cloudflare Workers/Pages sites can enable Markdown content negotiation with a one-line configuration change. When an AI agent sends `Accept: text/markdown`, the server responds with clean Markdown instead of HTML.
- Currently Cloudflare-specific
- Relevant for: sites already on Cloudflare infrastructure
- Other CDNs and frameworks expected to adopt this pattern as AI agent traffic grows
## Detailed Findings
[Per-category breakdown with evidence]
1---
2name: geo-technical
3description: Technical SEO audit with GEO-specific checks — crawlability, indexability, security, performance, SSR, and AI crawler access
4version: 1.0.0
5author: geo-seo-claude
6tags: [geo, technical-seo, core-web-vitals, ssr, crawlability, security, performance]
7allowed-tools: Read, Grep, Glob, Bash, WebFetch, Write
8metadata:
9 source: https://skillmd.com/skills/zubair-trabzada/geo-technical
10---
11
12# GEO Technical SEO Audit
13
14## Purpose
15
16Technical SEO forms the foundation of both traditional search visibility and AI search citation. A technically broken site cannot be crawled, indexed, or cited by any platform. This skill audits 8 categories of technical health with specific attention to GEO requirements — most critically, **server-side rendering** (AI crawlers do not execute JavaScript) and **AI crawler access** (many sites inadvertently block AI crawlers in robots.txt).
17
18## How to Use This Skill
19
201. Collect the target URL (homepage + 2-3 key inner pages)
212. Fetch each page using curl/WebFetch to get raw HTML and HTTP headers
223. Run through each of the 8 audit categories below
234. Score each category using the rubric
245. Generate GEO-TECHNICAL-AUDIT.md with results
25
26---
27
28## Category 1: Crawlability (15 points)
29
30### 1.1 robots.txt Validity
31- Fetch `https://[domain]/robots.txt`
32- Check for syntactic validity: proper `User-agent`, `Allow`, `Disallow` directives
33- Check for common errors: missing User-agent, wildcards blocking important paths, Disallow: / blocking entire site
34- Verify XML sitemap is referenced: `Sitemap: https://[domain]/sitemap.xml`
35
36### 1.2 AI Crawler Access (CRITICAL for GEO)
37Check robots.txt for directives targeting these AI crawlers:
38
39| Crawler | User-Agent | Platform |
40|---|---|---|
41| GPTBot | GPTBot | ChatGPT / OpenAI |
42| Google-Extended | Google-Extended | Gemini / Google AI training |
43| Googlebot | Googlebot | Google Search + AI Overviews |
44| Bingbot | bingbot | Bing Copilot + ChatGPT (via Bing) |
45| PerplexityBot | PerplexityBot | Perplexity AI |
46| ClaudeBot | ClaudeBot | Anthropic Claude |
47| Amazonbot | Amazonbot | Alexa / Amazon AI |
48| CCBot | CCBot | Common Crawl (used by many AI models) |
49| FacebookBot | FacebookExternalHit | Meta AI |
50| Bytespider | Bytespider | TikTok / ByteDance AI |
51| Applebot-Extended | Applebot-Extended | Apple Intelligence |
52
53**Scoring for AI crawler access:**
54- All major AI crawlers allowed: 5 points
55- Some blocked but Googlebot + Bingbot allowed: 3 points
56- GPTBot or PerplexityBot blocked: 1 point (significant GEO impact)
57- Googlebot blocked: 0 points (fatal)
58
59**Important nuance**: Blocking Google-Extended does NOT block Googlebot. Google-Extended only controls AI training data usage, not search indexing. However, blocking Google-Extended may reduce presence in AI Overviews. Recommend allowing Google-Extended unless there is a specific data licensing concern.
60
61### 1.3 XML Sitemaps
62- Fetch sitemap (check robots.txt for location, or try `/sitemap.xml`, `/sitemap_index.xml`)
63- Validate XML syntax
64- Check for `<lastmod>` dates (should be present and accurate)
65- Count URLs — compare to expected number of indexable pages
66- Check for sitemap index if large site (50,000+ URLs per sitemap max)
67- Verify all sitemap URLs return 200 status codes (sample check)
68
69### 1.4 Crawl Depth
70- Homepage = depth 0. Check that all important pages are reachable within **3 clicks** (depth 3)
71- Pages at depth 4+ receive significantly less crawl budget and are less likely to be cited by AI
72- Check internal linking: are key content pages linked from the homepage or main navigation?
73
74### 1.5 Noindex Management
75- Check for `<meta name="robots" content="noindex">` on pages that SHOULD be indexed
76- Check for `X-Robots-Tag: noindex` HTTP headers
77- Common mistakes: noindex on paginated pages, category pages, or key landing pages
78
79**Category Scoring:**
80| Check | Points |
81|---|---|
82| robots.txt valid and complete | 3 |
83| AI crawlers allowed | 5 |
84| XML sitemap present and valid | 3 |
85| Crawl depth within 3 clicks | 2 |
86| No erroneous noindex directives | 2 |
87
88---
89
90## Category 2: Indexability (12 points)
91
92### 2.1 Canonical Tags
93- Every indexable page must have a `<link rel="canonical" href="...">` tag
94- Canonical must point to itself (self-referencing) for the authoritative version
95- Check for conflicting canonicals (canonical in HTML vs. HTTP header)
96- Check for canonical chains (A canonicals to B, B canonicals to C — should be A to C)
97
98### 2.2 Duplicate Content
99- Check for www vs. non-www (both should resolve, one should redirect)
100- Check for HTTP vs. HTTPS (HTTP should redirect to HTTPS)
101- Check for trailing slash consistency (pick one pattern and redirect the other)
102- Check for parameter-based duplicates (`?sort=price` creating duplicate pages)
103
104### 2.3 Pagination
105- If paginated content exists, check for `rel="next"` / `rel="prev"` (note: Google ignores these as of 2019, but Bing still uses them)
106- Preferred: use `rel="canonical"` on paginated pages pointing to a view-all page or the first page
107- Ensure paginated pages are not noindexed if they contain unique content
108
109### 2.4 Hreflang (international sites)
110- Check for `<link rel="alternate" hreflang="xx">` tags
111- Validate: reciprocal hreflang (if page A points to page B, B must point back to A)
112- Validate: x-default fallback exists
113- Check for language/region code validity (ISO 639-1 / ISO 3166-1)
114
115### 2.5 Index Bloat
116- Estimate number of indexed pages (check sitemap count, use `site:domain.com` estimate)
117- Compare indexed pages to actual valuable content pages
118- Flag if indexed pages significantly exceed content pages (index bloat from thin/duplicate/parameter pages)
119
120**Category Scoring:**
121| Check | Points |
122|---|---|
123| Canonical tags correct on all pages | 3 |
124| No duplicate content issues | 3 |
125| Pagination handled correctly | 2 |
126| Hreflang correct (if applicable) | 2 |
127| No index bloat | 2 |
128
129---
130
131## Category 3: Security (10 points)
132
133### 3.1 HTTPS Enforcement
134- Site must load over HTTPS
135- HTTP must redirect to HTTPS (301 redirect)
136- No mixed content warnings (HTTP resources on HTTPS pages)
137- SSL/TLS certificate must be valid and not expired
138
139### 3.2 Security Headers
140Check HTTP response headers for:
141
142| Header | Required Value | Purpose |
143|---|---|---|
144| `Strict-Transport-Security` | `max-age=31536000; includeSubDomains` | Forces HTTPS |
145| `Content-Security-Policy` | Appropriate policy | Prevents XSS |
146| `X-Content-Type-Options` | `nosniff` | Prevents MIME sniffing |
147| `X-Frame-Options` | `DENY` or `SAMEORIGIN` | Prevents clickjacking |
148| `Referrer-Policy` | `strict-origin-when-cross-origin` or stricter | Controls referrer data |
149| `Permissions-Policy` | Appropriate restrictions | Controls browser features |
150
151**Category Scoring:**
152| Check | Points |
153|---|---|
154| HTTPS enforced with valid cert | 4 |
155| HSTS header present | 2 |
156| X-Content-Type-Options | 1 |
157| X-Frame-Options | 1 |
158| Referrer-Policy | 1 |
159| Content-Security-Policy | 1 |
160
161---
162
163## Category 4: URL Structure (8 points)
164
165### 4.1 Clean URLs
166- URLs should be human-readable: `/blog/seo-guide` not `/blog?id=12345`
167- No session IDs in URLs
168- Lowercase only (no mixed case)
169- Hyphens for word separation (not underscores)
170- No special characters or encoded spaces
171
172### 4.2 Logical Hierarchy
173- URL path should reflect site architecture: `/category/subcategory/page`
174- Flat where appropriate — avoid unnecessarily deep nesting
175- Consistent pattern across the site
176
177### 4.3 Redirect Chains
178- Check for redirect chains (A redirects to B redirects to C)
179- Maximum 1 hop recommended (A redirects to C directly)
180- Check for redirect loops
181- All redirects should be 301 (permanent), not 302 (temporary), unless intentionally temporary
182
183### 4.4 Parameter Handling
184- URL parameters should not create duplicate indexable pages
185- Use canonical tags or `robots.txt` Disallow for parameter variations
186- Configure parameter handling in Google Search Console and Bing Webmaster Tools
187
188**Category Scoring:**
189| Check | Points |
190|---|---|
191| Clean, readable URLs | 2 |
192| Logical hierarchy | 2 |
193| No redirect chains (max 1 hop) | 2 |
194| Parameter handling configured | 2 |
195
196---
197
198## Category 5: Mobile Optimization (10 points)
199
200### Critical Context
201As of **July 2024**, Google crawls ALL sites exclusively with mobile Googlebot. There is no desktop crawling. If your site does not work on mobile, it does not work for Google. Period.
202
203### 5.1 Responsive Design
204- Check for `<meta name="viewport" content="width=device-width, initial-scale=1">`
205- Content must not require horizontal scrolling on mobile
206- No fixed-width layouts wider than viewport
207
208### 5.2 Tap Targets
209- Interactive elements (buttons, links) must be at least 48x48 CSS pixels
210- Minimum 8px spacing between tap targets
211- Check that navigation is usable on mobile
212
213### 5.3 Font Sizes
214- Base font size should be at least 16px
215- No text requiring zoom to read
216- Sufficient contrast ratio (WCAG AA: 4.5:1 for normal text, 3:1 for large text)
217
218### 5.4 Mobile Content Parity
219- All content visible on desktop must also be visible on mobile
220- No hidden content behind "read more" toggles that Googlebot cannot expand (though Google has improved at expanding these as of 2025)
221- Images and media must load on mobile
222
223**Category Scoring:**
224| Check | Points |
225|---|---|
226| Viewport meta tag correct | 3 |
227| Responsive layout (no horizontal scroll) | 3 |
228| Tap targets appropriately sized | 2 |
229| Font sizes legible | 2 |
230
231---
232
233## Category 6: Core Web Vitals (15 points)
234
235### 2026 Metrics and Thresholds
236Core Web Vitals use the **75th percentile** of real user data (field data) as the benchmark. Lab data is useful for debugging but field data determines the ranking signal.
237
238| Metric | Good | Needs Improvement | Poor | Notes |
239|---|---|---|---|---|
240| **LCP** (Largest Contentful Paint) | < 2.5s | 2.5s - 4.0s | > 4.0s | Measures loading — time until largest visible element renders |
241| **INP** (Interaction to Next Paint) | < 200ms | 200ms - 500ms | > 500ms | Replaced FID in March 2024. Measures ALL interactions, not just first |
242| **CLS** (Cumulative Layout Shift) | < 0.1 | 0.1 - 0.25 | > 0.25 | Measures visual stability — unexpected layout movements |
243
244### How to Assess Without CrUX Data
245When real user data is unavailable, estimate from page characteristics:
246- **LCP**: Check largest above-fold element. Is it an image (check size/format)? Is it text (check web font loading)? Server response time (TTFB)?
247- **INP**: Check for heavy JavaScript on page. Long tasks (>50ms) block interactivity. Check for third-party scripts.
248- **CLS**: Check for images without explicit width/height. Check for dynamically inserted content above the fold. Check for web fonts causing layout shift (FOUT/FOIT).
249
250### Common LCP Fixes
2511. Optimize hero images: WebP/AVIF format, correct sizing, preload with `<link rel="preload">`
2522. Reduce server response time (TTFB < 800ms)
2533. Eliminate render-blocking CSS/JS
2544. Preconnect to critical third-party origins
255
256### Common INP Fixes
2571. Break up long tasks (>50ms) into smaller chunks using `requestIdleCallback` or `scheduler.yield()`
2582. Reduce third-party JavaScript
2593. Use `content-visibility: auto` for off-screen content
2604. Debounce/throttle event handlers
261
262### Common CLS Fixes
2631. Always include `width` and `height` attributes on images and videos
2642. Reserve space for ads and embeds with CSS `aspect-ratio` or explicit dimensions
2653. Use `font-display: swap` with size-adjusted fallback fonts
2664. Avoid inserting content above existing content after page load
267
268**Category Scoring:**
269| Check | Points |
270|---|---|
271| LCP < 2.5s | 5 |
272| INP < 200ms | 5 |
273| CLS < 0.1 | 5 |
274
275---
276
277## Category 7: Server-Side Rendering (15 points) — CRITICAL FOR GEO
278
279### Why SSR Is Mandatory for AI Visibility
280AI crawlers (GPTBot, PerplexityBot, ClaudeBot, etc.) do **NOT execute JavaScript**. They fetch the raw HTML and parse it. If your content is rendered client-side by React, Vue, Angular, or any other JavaScript framework, AI crawlers see an empty page.
281
282Even Googlebot, which does execute JavaScript, deprioritizes JS-rendered content due to the additional crawl budget required. Google processes JS rendering in a separate "rendering queue" that can delay indexing by days or weeks.
283
284### Detection Method
2851. Fetch the page with curl (no JavaScript execution): `curl -s [URL]`
2862. Compare the raw HTML to the rendered DOM (via browser)
2873. If key content (headings, paragraphs, product info, article text) is MISSING from the curl output, the site relies on client-side rendering
288
289### What to Check
290- **Main content text**: Is the article body / product description / page content in the raw HTML?
291- **Headings**: Are H1, H2, H3 tags present in raw HTML?
292- **Navigation**: Is the main navigation server-rendered?
293- **Structured data**: Is JSON-LD in the raw HTML or injected by JavaScript?
294- **Meta tags**: Are title, description, canonical, OG tags in the raw HTML?
295- **Internal links**: Are navigation and content links in the raw HTML? (Critical for crawlability)
296
297### SSR Solutions to Recommend
298| Framework | SSR Solution |
299|---|---|
300| React | Next.js (SSR/SSG), Remix, Gatsby (SSG) |
301| Vue | Nuxt.js (SSR/SSG) |
302| Angular | Angular Universal |
303| Svelte | SvelteKit |
304| Generic | Prerender.io (prerendering service), Rendertron |
305
306### Scoring Detail
307- All key content server-rendered: 15 points
308- Main content server-rendered but some elements JS-only: 10 points
309- Critical content requires JS (product info, article text): 5 points
310- Entire page is client-rendered (empty body in raw HTML): 0 points
311
312**Category Scoring:**
313| Check | Points |
314|---|---|
315| Main content in raw HTML | 8 |
316| Meta tags + structured data in raw HTML | 4 |
317| Internal links in raw HTML | 3 |
318
319---
320
321## Category 8: Page Speed & Server Performance (15 points)
322
323### 8.1 Time to First Byte (TTFB)
324- Target: **< 800ms** (ideally < 200ms)
325- Measure with curl: `curl -o /dev/null -s -w 'TTFB: %{time_starttransfer}s\n' [URL]`
326- If TTFB > 800ms: check server location, caching, database queries, CDN usage
327
328### 8.2 Resource Optimization
329- Total page weight target: **< 2MB** (critical pages < 1MB)
330- Check for uncompressed resources (gzip/brotli compression should be enabled)
331- Check for unminified CSS and JavaScript
332- Check for unused CSS/JS (can represent 50%+ of downloaded bytes on many sites)
333
334### 8.3 Image Optimization
335- Check image formats: WebP or AVIF preferred over JPEG/PNG
336- Check for oversized images (images larger than display size)
337- Check for lazy loading: images below fold should have `loading="lazy"`
338- Check for explicit dimensions (width/height attributes prevent CLS)
339- Above-fold images should NOT be lazy loaded (harms LCP)
340
341### 8.4 Code Splitting and Lazy Loading
342- JavaScript should be code-split so each page only loads what it needs
343- Check for large JavaScript bundles (> 200KB compressed is a warning, > 500KB is critical)
344- Third-party scripts should load asynchronously (`async` or `defer`)
345- Check for render-blocking resources in `<head>`
346
347### 8.5 Caching
348- Check `Cache-Control` headers on static resources (images, CSS, JS)
349- Static assets should have long cache times: `max-age=31536000` (1 year) with content-hashed filenames
350- HTML pages should have shorter cache or `no-cache` with validation (`ETag` or `Last-Modified`)
351
352### 8.6 CDN Usage
353- Check if static resources are served from a CDN (different domain or CDN-specific headers)
354- For global audience, CDN is critical for consistent performance
355- Check for CDN-specific headers: `CF-Ray` (Cloudflare), `X-Cache` (AWS CloudFront), `X-Served-By` (Fastly)
356
357**Category Scoring:**
358| Check | Points |
359|---|---|
360| TTFB < 800ms | 3 |
361| Page weight < 2MB | 2 |
362| Images optimized (format, size, lazy) | 3 |
363| JS bundles reasonable (< 200KB compressed) | 2 |
364| Compression enabled (gzip/brotli) | 2 |
365| Cache headers on static resources | 2 |
366| CDN in use | 1 |
367
368---
369
370## Category 9: Agent-Readiness Signals (non-scoring)
371
372These checks surface emerging AI agent compatibility signals. None contribute to the numeric score — they produce a pass or a recommendation. The underlying standards are either IETF drafts or early-adoption features; penalizing absence would be unfair.
373
374### 9.1 RFC 8288 Link Headers (Service Discovery)
375
376RFC 8288 (Web Linking) defines the HTTP `Link:` response header. Servers can use it to advertise related resources — API catalog, service docs, MCP server card — in a machine-readable way, without HTML parsing.
377
378**How to check:** Capture all `Link:` response headers from the standard homepage fetch (no extra request).
379
380**What to look for:**
381- Parse `<url>; rel="relation-type"` pairs.
382- High-value rel types: `api-catalog` (RFC 9609), `describedby`, `service-doc`, `mcp-server-card`.
383
384**When to surface a recommendation:** Only for API-first sites (API docs linked in nav, `/api/` or `/developers/` paths, swagger/OpenAPI in sitemap). Omit this section entirely for standard business sites — absence is expected and not noteworthy.
385
386| State | Treatment |
387|---|---|
388| `Link:` headers present, known rel types | Informational — document what was found |
389| `Link:` headers present, unknown rel types | Informational — note and explain |
390| Absent, API-first site | Recommendation — explain and suggest implementation |
391| Absent, standard business site | Omit — do not surface |
392
393### 9.2 Markdown Content Negotiation
394
395Checks if the server responds to `Accept: text/markdown` with `Content-Type: text/markdown`. Cloudflare's "Markdown for Agents" feature enables this — AI agents receive clean Markdown instead of HTML, eliminating boilerplate stripping and improving content extraction accuracy.
396
397**How to check:** Send a GET to the homepage with `Accept: text/markdown`. This is one additional HTTP request per audit.
398
399**Evaluation:**
400- If response `Content-Type` is `text/markdown` (or `text/markdown; charset=utf-8`): pass — note as a leading-edge capability.
401- Otherwise: forward-looking recommendation, not a failure.
402- If the request errors or returns non-200: skip and note the error. Do not penalize.
403
404| State | Treatment |
405|---|---|
406| `text/markdown` returned | Bonus — note as a leading-edge capability |
407| Standard HTML returned | Forward-looking recommendation |
408| Request errors / non-200 | Skip, note the error, do not penalize |
409
410---
411
412## IndexNow Protocol
413
414### What It Is
415IndexNow is an open protocol that allows websites to notify search engines instantly when content is created, updated, or deleted. Supported by Bing, Yandex, Seznam, and Naver. Google does NOT support IndexNow but monitors the protocol.
416
417### Why It Matters for GEO
418ChatGPT uses Bing's index. Bing Copilot uses Bing's index. Faster Bing indexing means faster AI visibility on two major platforms.
419
420### Implementation Check
4211. Check for IndexNow key file: `https://[domain]/.well-known/indexnow-key.txt` or similar
4222. Check if CMS has IndexNow plugin (WordPress: IndexNow plugin; many modern CMS platforms support it natively)
4233. If not implemented, recommend adding it with instructions
424
425---
426
427## Overall Scoring
428
429| Category | Max Points | Weight |
430|---|---|---|
431| Crawlability | 15 | Core foundation |
432| Indexability | 12 | Core foundation |
433| Security | 10 | Trust signal |
434| URL Structure | 8 | Crawl efficiency |
435| Mobile Optimization | 10 | Google requirement |
436| Core Web Vitals | 15 | Ranking signal |
437| Server-Side Rendering | 15 | GEO critical |
438| Page Speed & Server | 15 | Performance |
439| **Total** | **100** | |
440
441Non-scoring checks (Category 9) appear in the output under "Agent-Readiness Signals" and do not affect this total.
442
443### Score Interpretation
444- **90-100**: Excellent — technically sound for both traditional SEO and GEO
445- **70-89**: Good — minor issues to address but fundamentally solid
446- **50-69**: Needs Work — significant technical debt impacting visibility
447- **30-49**: Poor — major issues blocking crawling, indexing, or AI visibility
448- **0-29**: Critical — fundamental technical failures requiring immediate attention
449
450---
451
452## Output Format
453
454Generate **GEO-TECHNICAL-AUDIT.md** with:
455
456```markdown
457# GEO Technical SEO Audit — [Domain]
458Date: [Date]
459
460## Technical Score: XX/100
461
462## Score Breakdown
463| Category | Score | Status |
464|---|---|---|
465| Crawlability | XX/15 | Pass/Warn/Fail |
466| Indexability | XX/12 | Pass/Warn/Fail |
467| Security | XX/10 | Pass/Warn/Fail |
468| URL Structure | XX/8 | Pass/Warn/Fail |
469| Mobile Optimization | XX/10 | Pass/Warn/Fail |
470| Core Web Vitals | XX/15 | Pass/Warn/Fail |
471| Server-Side Rendering | XX/15 | Pass/Warn/Fail |
472| Page Speed & Server | XX/15 | Pass/Warn/Fail |
473
474Status: Pass = 80%+ of category points, Warn = 50-79%, Fail = <50%
475
476## AI Crawler Access
477| Crawler | User-Agent | Status | Recommendation |
478|---|---|---|---|
479| GPTBot | GPTBot | Allowed/Blocked | [Action] |
480| Googlebot | Googlebot | Allowed/Blocked | [Action] |
481[Continue for all AI crawlers]
482
483## Critical Issues (fix immediately)
484[List with specific page URLs and what is wrong]
485
486## Warnings (fix this month)
487[List with details]
488
489## Recommendations (optimize this quarter)
490[List with details]
491
492## Agent-Readiness Signals (non-scoring)
493
494### RFC 8288 Link Headers (Service Discovery)
495
496**Status:** Present / Absent / Not Applicable
497
498<!-- If present: -->
499| Relation Type | URL | Meaning |
500|---|---|---|
501| api-catalog | /.well-known/api-catalog | Machine-readable index of available APIs |
502| mcp-server-card | /.well-known/mcp.json | MCP server capability declaration |
503
504AI agents and API clients can discover your services without parsing HTML.
505
506<!-- If absent, API-first site only: -->
507**Informational Recommendation:** This site has API/developer-oriented content but no `Link:` headers advertising discoverable services.
508
509Example: `Link: </.well-known/api-catalog>; rel="api-catalog"`
510
511Relevant for: sites with public APIs, OpenAPI docs, or MCP server integrations.
512Reference: RFC 8288, RFC 9609.
513
514<!-- If absent, standard business site: omit this section entirely -->
515
516### Markdown Content Negotiation
517
518**Status:** Supported / Not Supported
519**Test:** GET [url] with `Accept: text/markdown`
520**Response Content-Type:** [value]
521
522<!-- If supported: -->
523This site serves clean Markdown to AI agents on request. AI crawlers that support content negotiation receive formatted text without HTML boilerplate.
524
525<!-- If not supported: -->
526**Forward-Looking Recommendation:** Cloudflare Workers/Pages sites can enable Markdown content negotiation with a one-line configuration change. When an AI agent sends `Accept: text/markdown`, the server responds with clean Markdown instead of HTML.
527
528- Currently Cloudflare-specific
529- Relevant for: sites already on Cloudflare infrastructure
530- Other CDNs and frameworks expected to adopt this pattern as AI agent traffic grows
531
532## Detailed Findings
533[Per-category breakdown with evidence]
534```