SEO Technical: Indexing
Guides indexing troubleshooting and fix actions. For how to find and diagnose issues in GSC, see analytics-google-search-console.
When invoking: On first use, if helpful, open with 1–2 sentences on what this skill covers and why it matters, then provide the main output. On subsequent use or when the user asks to skip, go directly to the main output.
Scope (Technical SEO)
- Fix actions: noindex, robots.txt, canonical, content quality, URL Inspection
- Noindex: Use when excluding pages intentionally; not all content needs indexing. Complements robots.txt (crawl control) and analytics-google-search-console (Coverage interpretation)
Initial Assessment
Check for product marketing context first: If .claude/product-marketing-context.md or .cursor/product-marketing-context.md exists, read it for site URL and indexing goals.
Identify issue from GSC (see analytics-google-search-console for Coverage report, issue types, diagnosis workflow). Then apply fix below.
Crawled - Currently Not Indexed
| Cause |
Action |
| Low quality, duplicate, off-topic |
Improve content, fix duplicates, set correct canonical |
| Static assets (CSS/JS) |
See below |
| Feed, share URLs with params |
Usually OK to ignore; or noindex, canonical to main URL |
| Important content pages |
Use URL Inspection, verify canonical/internal links/sitemap, Request indexing |
Static Assets (Next.js / Vercel)
Vercel adds unique dpl= params to static assets per deploy, creating many "Crawled - currently not indexed" URLs.
| Do |
Don't |
Keep robots.txt allowing /_next/ |
Do not block /_next/ (breaks CSS/JS loading) |
| Accept static assets in GSC as expected |
Do not block /_next/static/css/ or ?dpl= |
| Use X-Robots-Tag for static assets |
CSS/JS should not be indexed; no SEO impact |
Static assets in "Crawled - currently not indexed" is normal and expected.
Other Issue Types (from GSC Coverage)
| Issue |
Fix |
| Excluded by «noindex» tag |
Remove noindex if accidental; keep if intentional |
| Redirect / 404 |
Fix URL or add redirect |
| Duplicate / Canonical |
Set correct canonical; usually OK |
Noindex Usage
- When: Login, admin, duplicate content, low-value pages, legal boilerplate, thank-you pages
- How:
metadata.robots = { index: false } or X-Robots-Tag
- Rationale: Not all site content should be indexed; noindex is a valid choice for many pages
- Caution: Avoid noindex on important content pages
- With robots.txt: robots.txt controls crawl access; noindex controls indexing. Use both: robots for paths (e.g. /admin/), noindex for specific pages. See seo-technical-robots
Google Indexing API
| Type |
Typical use |
| JobPosting |
Job boards |
| BroadcastEvent |
Live platforms |
Requirements: Enable Indexing API, create service account, add owner in Search Console, request quota (default 200 URLs/day).
Output Format
Related Skills
- analytics-google-search-console: Find and diagnose indexing issues in GSC
- seo-technical-robots: Ensure robots.txt does not block indexing
- seo-technical-sitemap: Submit and maintain sitemap
- seo-technical-indexnow: Faster indexing for Bing
- seo-technical-canonical: Resolve duplicate content
1---2name: seo-technical-indexing3description: When the user wants to fix indexing issues from Search Console, use noindex, or implement Google Indexing API. Also use when the user mentions "fix indexing," "not indexed," "Crawled - currently not indexed," "noindex," or "Google Indexing API."4---5
6# SEO Technical: Indexing
7
8Guides indexing troubleshooting and fix actions. For how to find and diagnose issues in GSC, see analytics-google-search-console.
9
10**When invoking**: On **first use**, if helpful, open with 1–2 sentences on what this skill covers and why it matters, then provide the main output. On **subsequent use** or when the user asks to skip, go directly to the main output.
11
12## Scope (Technical SEO)
13
14- **Fix actions**: noindex, robots.txt, canonical, content quality, URL Inspection
15- **Noindex**: Use when excluding pages intentionally; not all content needs indexing. Complements robots.txt (crawl control) and analytics-google-search-console (Coverage interpretation)
16
17## Initial Assessment
18
19**Check for product marketing context first:** If `.claude/product-marketing-context.md` or `.cursor/product-marketing-context.md` exists, read it for site URL and indexing goals.
20
21Identify issue from GSC (see analytics-google-search-console for Coverage report, issue types, diagnosis workflow). Then apply fix below.
22
23## Crawled - Currently Not Indexed
24
25| Cause | Action |
26|-------|--------|
27| Low quality, duplicate, off-topic | Improve content, fix duplicates, set correct canonical |
28| Static assets (CSS/JS) | See below |
29| Feed, share URLs with params | Usually OK to ignore; or noindex, canonical to main URL |
30| Important content pages | Use URL Inspection, verify canonical/internal links/sitemap, Request indexing |
31
32### Static Assets (Next.js / Vercel)
33
34Vercel adds unique `dpl=` params to static assets per deploy, creating many "Crawled - currently not indexed" URLs.
35
36| Do | Don't |
37|----|-------|
38| Keep robots.txt allowing `/_next/` | Do not block `/_next/` (breaks CSS/JS loading) |
39| Accept static assets in GSC as expected | Do not block `/_next/static/css/` or `?dpl=` |
40| Use X-Robots-Tag for static assets | CSS/JS should not be indexed; no SEO impact |
41
42Static assets in "Crawled - currently not indexed" is **normal and expected**.
43
44## Other Issue Types (from GSC Coverage)
45
46| Issue | Fix |
47|-------|-----|
48| Excluded by «noindex» tag | Remove noindex if accidental; keep if intentional |
49| Redirect / 404 | Fix URL or add redirect |
50| Duplicate / Canonical | Set correct canonical; usually OK |
51
52## Noindex Usage
53
54- **When**: Login, admin, duplicate content, low-value pages, legal boilerplate, thank-you pages
55- **How**: `metadata.robots = { index: false }` or X-Robots-Tag
56- **Rationale**: Not all site content should be indexed; noindex is a valid choice for many pages
57- **Caution**: Avoid noindex on important content pages
58- **With robots.txt**: robots.txt controls crawl access; noindex controls indexing. Use both: robots for paths (e.g. /admin/), noindex for specific pages. See seo-technical-robots
59
60## Google Indexing API
61
62| Type | Typical use |
63|------|-------------|
64| JobPosting | Job boards |
65| BroadcastEvent | Live platforms |
66
67**Requirements**: Enable Indexing API, create service account, add owner in Search Console, request quota (default 200 URLs/day).
68
69## Output Format
70
71- **Action items**: Prioritized fixes
72- **References**: [Page indexing report](https://support.google.com/webmasters/answer/7440203)
73
74## Related Skills
75
76- **analytics-google-search-console**: Find and diagnose indexing issues in GSC
77- **seo-technical-robots**: Ensure robots.txt does not block indexing
78- **seo-technical-sitemap**: Submit and maintain sitemap
79- **seo-technical-indexnow**: Faster indexing for Bing
80- **seo-technical-canonical**: Resolve duplicate content
tools-only