# AI Search Technical Auditor > Audit front-end code for AI search readiness. Use when reviewing HTML structure, meta tags, schema markup, and technical elements that affect how AI crawlers understand and index web pages. - Skill: `diegosouzapw/ai-search-technical-auditor` (Agent Skill, multi-file: 2 files) - Install (CLI): `npx skillmds@latest add diegosouzapw/ai-search-technical-auditor` - Raw SKILL.md: https://api.skillmd.com/api/skills/diegosouzapw/ai-search-technical-auditor/raw - Safety review: pending (external: skill-scanner PASS, skillspector PASS) - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Web & Frontend - Author: diegosouzapw (https://skillmd.com/u/diegosouzapw) - Updated: 2026-09-08 - Page: https://skillmd.com/skills/diegosouzapw/ai-search-technical-auditor --- # AI Search Technical Auditor Audit HTML, meta tags, schema markup, and technical implementation to ensure pages are optimized for AI search engine crawlers. ## Understanding AI Crawlers AI search engines use different crawlers than traditional search engines. Key AI crawlers include: | Crawler | Platform | User Agent Contains | |---------|----------|---------------------| | GPTBot | OpenAI/ChatGPT | `GPTBot` | | Google-Extended | Google AI/Gemini | `Google-Extended` | | ClaudeBot | Anthropic/Claude | `ClaudeBot` | | PerplexityBot | Perplexity | `PerplexityBot` | | Bytespider | ByteDance | `Bytespider` | ## Technical Audit Checklist ### 1. robots.txt Configuration Check that AI crawlers are allowed: ```txt # Good - Allow AI crawlers User-agent: GPTBot Allow: / User-agent: Google-Extended Allow: / User-agent: ClaudeBot Allow: / User-agent: PerplexityBot Allow: / ``` ```txt # Bad - Blocking AI crawlers (unless intentional) User-agent: GPTBot Disallow: / ``` **Audit questions:** - [ ] Are AI crawlers explicitly allowed or blocked? - [ ] Is the intent to block deliberate and documented? - [ ] Are important content paths accessible? ### 2. HTML Structure #### Semantic HTML Check for proper semantic structure: ```html

Page Title

TL;DR content here

Section Heading

Content...

Page Title

Section Heading

Content...

``` **Audit questions:** - [ ] Uses semantic elements (`

`, `

`, `

`, `

`)? - [ ] Heading hierarchy is logical (H1 → H2 → H3)? - [ ] Only one `

` per page? - [ ] Main content is in `
` element? #### Content Accessibility ```html
Product Features

Feature one description

Feature two description

``` Audit questions: - [ ] Key content is in initial HTML (not JavaScript-rendered)? - [ ] Images have descriptive alt text? - [ ] Tables have proper headers (``)? - [ ] Lists use `
`/`
` elements? ### 3. Meta Tags #### Essential Meta Tags ```html What is GEO? Generative Engine Optimization Guide | Brand ``` Audit questions: - [ ] Title is descriptive and under 60 characters? - [ ] Meta description summarizes content (150-160 chars)? - [ ] Canonical URL is set correctly? - [ ] Language is declared? - [ ] Viewport meta tag is present? #### Open Graph Tags ```html ``` Audit questions: - [ ] OG title and description are set? - [ ] OG image is specified and valid? - [ ] OG URL matches canonical? ### 4. Schema Markup (JSON-LD) #### Article Schema ```html ``` #### FAQ Schema ```html ``` #### HowTo Schema ```html ``` Audit questions: - [ ] Schema type matches content type? - [ ] JSON-LD is valid (no syntax errors)? - [ ] Required properties are populated? - [ ] Dates are in ISO 8601 format? - [ ] URLs are absolute, not relative? ### 5. Page Speed AI crawlers, especially agentic ones, have tight time budgets. Target: | Metric | Target | Critical for | |--------|--------|--------------| | Time to First Byte (TTFB) | < 200ms | All crawlers | | First Contentful Paint (FCP) | < 1.8s | Content extraction | | Largest Contentful Paint (LCP) | < 2.5s | Full page analysis | | AI Crawler Target | < 0.4s FCP | Real-time agents | Audit questions: - [ ] TTFB under 200ms? - [ ] FCP under 1.8s (ideally under 0.4s)? - [ ] No render-blocking JavaScript? - [ ] Images are optimized and lazy-loaded? - [ ] CSS is minified? ### 6. Mobile Friendliness ```html ``` Audit questions: - [ ] Viewport meta tag is present? - [ ] Content is readable without zooming? - [ ] Touch targets are appropriately sized? - [ ] No horizontal scrolling required? ### 7. URL Structure ``` Good URLs: /guides/geo-optimization /products/analytics-platform /docs/api-reference Bad URLs: /page.php?id=123&cat=5 /guides/geo-optimization-best-practices-guide-2024-updated-version-3 /p/12345 ``` Audit questions: - [ ] URLs are descriptive and readable? - [ ] URLs are not excessively long? - [ ] URLs use hyphens, not underscores? - [ ] No query parameters for primary content? ## Audit Report Template ```markdown # AI Search Technical Audit Report URL: [Page URL] Date: [Audit Date] Overall Score: [X/100] ## Summary [Brief summary of findings] ## Critical Issues - [ ] Issue 1 - [ ] Issue 2 ## Warnings - [ ] Warning 1 - [ ] Warning 2 ## Passed Checks - [x] Check 1 - [x] Check 2 ## Recommendations 1. [Priority 1 recommendation] 2. [Priority 2 recommendation] 3. [Priority 3 recommendation] ``` ## Quick Reference: Schema Types | Content Type | Schema | Required Properties | |--------------|--------|---------------------| | Article | Article | headline, author, datePublished | | FAQ | FAQPage | mainEntity (array of Questions) | | How-to | HowTo | name, step (array of HowToSteps) | | Product | Product | name, description, offers | | Organization | Organization | name, url, logo | | Person | Person | name | | BreadcrumbList | BreadcrumbList | itemListElement | ## Validation Tools After implementing fixes, validate using: 1. Schema: Google Rich Results Test, Schema.org Validator 2. HTML: W3C Validator 3. Speed: Google PageSpeed Insights, WebPageTest 4. Mobile: Google Mobile-Friendly Test 5. robots.txt: Google Search Console robots.txt Tester