SEO Recovery

SEO Recovery Protocol

When This Skill Activates

• "SEO traffic dropped", "organic traffic down", "lost rankings"
• "Indexing issues", "pages not indexed", "deindexed"
• "Canonical URL", "canonical mismatch", "wrong canonical"
• "Hreflang", "hreflang errors", "language tags"
• "Sitemap", "sitemap errors", "toxic sitemap", "missing sitemap"
• "Search Console", "GSC", "crawl errors", "coverage issues"
• "Robots.txt", "noindex", "blocked by robots"
• "Google not showing my pages"

Anti-Hallucination Rules (NEVER violate)

Phase 0: Confirm Scope (MANDATORY - Do This First)

Before any investigation:

1. What is the domain? (exact URL, including protocol and www vs non-www)
2. When did the traffic drop start? (exact date or date range)
3. How severe is the drop? (percentage, from X to Y)
4. Is it site-wide or specific pages/sections?
5. Were any changes deployed around that time? (redesign, migration, CMS update, hosting change)
6. Which GSC property are we using? (domain property vs URL prefix?)
7. What is the hosting platform? (Hostinger? Other? -- affects deployment workflow)

If the user cannot answer #2 and #3, start with GSC Search Analytics to establish the timeline.

Phase 1: SEO Diagnostic Protocol

Step 1: Establish the Timeline (Search Analytics)

Pull traffic data to identify exactly when the drop occurred:

GSC MCP QUERY:
- Tool: google_searchconsole (via MCP Gateway)
- Metric: clicks, impressions, CTR, position
- Date range: 90 days (or wider if drop is older)
- Dimensions: date (for timeline), page (for affected URLs), query (for lost keywords)

What to look for:

• Sudden cliff (overnight) = algorithmic penalty or technical issue
• Gradual decline over weeks = content quality, competition, or slow technical degradation
• Specific pages/sections = localized issue (canonical, noindex on section)
• All pages equally = site-wide issue (robots.txt, domain-level penalty, DNS)

Step 2: Check Indexing Status (Coverage Report)

Query GSC Coverage to understand indexing state:

KEY COVERAGE STATES:
- "Valid" = Indexed and serving in search
- "Valid with warnings" = Indexed but has issues
- "Excluded" = Not indexed (check reason!)
- "Error" = Crawl/index errors

COMMON EXCLUSION REASONS:
- "Alternate page with proper canonical tag" = Canonical pointing elsewhere
- "Duplicate without user-selected canonical" = Google chose a different canonical
- "Crawled - currently not indexed" = Google crawled but chose not to index
- "Discovered - currently not indexed" = Google knows about it but hasn't crawled
- "Page with redirect" = URL redirects to another page
- "Blocked by robots.txt" = robots.txt is blocking crawlers
- "Excluded by 'noindex' tag" = meta noindex or X-Robots-Tag

Step 3: URL Inspection (Spot-Check Key Pages)

For affected pages, run URL Inspection to get Google's view:

CHECK THESE FOR EACH AFFECTED PAGE:
1. Is the page indexed?
2. What canonical does Google see? (vs what the page declares)
3. Is it mobile-friendly?
4. Are there any crawl issues?
5. Last crawl date -- is it recent or stale?
6. Does the rendered HTML match expectations?

Step 4: Root Cause Analysis

Based on Phases 1-3, route to the appropriate diagnosis:

Phase 2: Common SEO Issues & Fixes

CANONICAL FIX: Canonical URL Mismatches

Diagnosis:

# Check canonical tag in actual HTML source
curl -s "https://example.com/page" | grep -i 'rel="canonical"'
curl -s "https://example.com/page" | grep -i "rel='canonical'"

# Check HTTP vs HTTPS canonical
curl -s "http://example.com/page" | grep -i 'canonical'
curl -s "https://example.com/page" | grep -i 'canonical'

# Check www vs non-www
curl -s "https://www.example.com/page" | grep -i 'canonical'
curl -s "https://example.com/page" | grep -i 'canonical'

# Check for canonical in HTTP headers (X-Robots-Tag)
curl -sI "https://example.com/page" | grep -i 'link.*canonical'

# Bulk check across pages (sample 20 URLs from sitemap)
curl -s "https://example.com/sitemap.xml" | grep -oP '<loc>\K[^<]+' | head -20 | while read url; do
  canonical=$(curl -s "$url" | grep -oP 'rel="canonical"[^>]*href="[^"]*"' | grep -oP 'href="\K[^"]+')
  echo "$url -> $canonical"
done

Common mismatch patterns:

• HTTP canonical on HTTPS page (or vice versa)
• www canonical on non-www page (or vice versa)
• Trailing slash mismatch (/page vs /page/)
• Query parameter pollution (/page?utm_source=... as canonical)
• Relative URL instead of absolute URL in canonical tag
• CMS generating wrong canonical (pagination, filters, search pages)

Fix approach:

1. Identify the canonical URL pattern that Google should use
2. Update canonical tags in templates/CMS to use the correct absolute URL
3. Ensure consistency: canonical URL matches the URL in sitemaps, internal links, and hreflang
4. Submit affected URLs for re-inspection in GSC after deploying fix

HREFLANG FIX: Language/Region Tag Bugs

Diagnosis:

# Check hreflang tags on page
curl -s "https://example.com/page" | grep -i 'hreflang'

# Check for return links (CRITICAL: hreflang must be reciprocal)
# If page-en links to page-fr via hreflang, page-fr MUST link back to page-en
curl -s "https://example.com/en/page" | grep -i 'hreflang'
curl -s "https://example.com/fr/page" | grep -i 'hreflang'

# Check hreflang in sitemap (alternative to on-page)
curl -s "https://example.com/sitemap.xml" | grep -i 'hreflang'

# Check x-default tag exists
curl -s "https://example.com/page" | grep -i 'x-default'

Common hreflang errors:

• Missing return links (non-reciprocal hreflang)
• Wrong language/region codes (en-uk instead of en-GB)
• Hreflang pointing to non-canonical URLs
• Missing x-default tag
• Hreflang on pages that are noindex
• Hreflang URL mismatch with canonical URL

Fix approach:

1. Map all language variants for each page
2. Ensure every hreflang annotation is reciprocal
3. Use correct ISO 639-1 (language) and ISO 3166-1 Alpha-2 (region) codes
4. Hreflang URLs must match canonical URLs exactly
5. Include x-default for fallback
6. Validate with hreflang testing tools or GSC International Targeting report

SITEMAP FIX: Toxic or Broken Sitemaps

Diagnosis:

# Check robots.txt for sitemap declarations
curl -s "https://example.com/robots.txt"

# Fetch and validate sitemap
curl -s "https://example.com/sitemap.xml" | head -50

# Check for sitemap index (multiple sitemaps)
curl -s "https://example.com/sitemap_index.xml"

# Count URLs in sitemap
curl -s "https://example.com/sitemap.xml" | grep -c '<loc>'

# Check for HTTP status of sitemap
curl -sI "https://example.com/sitemap.xml" | head -5

# Check for URLs in sitemap that return non-200
curl -s "https://example.com/sitemap.xml" | grep -oP '<loc>\K[^<]+' | head -10 | while read url; do
  status=$(curl -sI "$url" -o /dev/null -w "%{http_code}")
  echo "$status $url"
done

Toxic sitemap indicators:

• Contains URLs that return 404, 410, or 301
• Contains noindex pages
• Contains non-canonical URLs
• Contains URLs blocked by robots.txt
• Sitemap itself returns non-200
• Over 50,000 URLs per sitemap file (exceeds limit)
• Over 50MB uncompressed (exceeds limit)
• Sitemap not declared in robots.txt
• Stale <lastmod> dates (or dates in the future)

GSC Sitemap Management:

SUBMIT MISSING SITEMAPS:
- Use GSC Sitemaps API/tool to submit each sitemap URL
- Common pattern: submit /sitemap.xml, /sitemap-posts.xml, /sitemap-pages.xml, etc.
- After submitting, monitor "Submitted" vs "Indexed" counts
- If big gap between submitted and indexed, investigate excluded URLs

CHECK FOR MISSING SITEMAPS:
- Compare sitemap_index.xml entries vs what's submitted in GSC
- Any sitemap in the index but not in GSC = submit it
- Any sitemap in GSC but returning 404 = remove it

Fix approach:

1. Audit all sitemaps for toxic URLs (4xx, 5xx, noindex, non-canonical)
2. Remove or fix problematic URLs
3. Ensure all valid, indexable URLs are in sitemaps
4. Submit any missing sitemaps to GSC
5. Remove any dead sitemaps from GSC
6. Verify sitemap is declared in robots.txt
7. Regenerate sitemaps if CMS-generated ones are toxic

ROBOTS.TXT FIX: Crawler Blocking

Diagnosis:

# Fetch current robots.txt
curl -s "https://example.com/robots.txt"

# Check if specific path is blocked
# (manually parse Disallow rules against the target path)

# Check for overly broad Disallow rules
curl -s "https://example.com/robots.txt" | grep -i 'disallow'

# Check for noindex in robots.txt (NOT a valid directive but some sites use it)
curl -s "https://example.com/robots.txt" | grep -i 'noindex'

Common robots.txt mistakes:

• Disallow: / blocking entire site
• Blocking CSS/JS files needed for rendering
• Blocking entire directories that contain indexable content
• Different rules for Googlebot vs other bots causing confusion
• Noindex: directive in robots.txt (Google ignores this since 2019)
• Missing Allow: for specific paths within a Disallowed directory
• Robots.txt returns non-200 (5xx = Google assumes full block temporarily)

Fix approach:

1. Review all Disallow rules against intended indexable content
2. Remove overly broad blocks
3. Add specific Allow rules where needed
4. Ensure CSS and JS files are not blocked (needed for mobile-first indexing)
5. Test with GSC robots.txt Tester
6. Deploy updated robots.txt and request re-crawl of affected URLs

NOINDEX FIX: Accidental Noindex Tags

Diagnosis:

# Check meta robots tag
curl -s "https://example.com/page" | grep -i 'robots'

# Check X-Robots-Tag HTTP header
curl -sI "https://example.com/page" | grep -i 'x-robots'

# Bulk check for noindex across pages
curl -s "https://example.com/sitemap.xml" | grep -oP '<loc>\K[^<]+' | head -20 | while read url; do
  noindex_meta=$(curl -s "$url" | grep -i 'noindex' | head -1)
  noindex_header=$(curl -sI "$url" | grep -i 'noindex' | head -1)
  if [ -n "$noindex_meta" ] || [ -n "$noindex_header" ]; then
    echo "NOINDEX: $url"
    [ -n "$noindex_meta" ] && echo "  meta: $noindex_meta"
    [ -n "$noindex_header" ] && echo "  header: $noindex_header"
  fi
done

Common noindex causes:

• CMS "discourage search engines" setting left on from staging
• Meta robots tag added during development, never removed
• X-Robots-Tag header set in server config (.htaccess, nginx, Hostinger config)
• Plugin or middleware injecting noindex conditionally (e.g., on pagination)
• Conditional noindex based on query parameters

Fix approach:

1. Identify source of noindex (HTML meta tag vs HTTP header vs server config)
2. Remove the noindex directive at its source
3. If CMS setting, toggle it off and verify HTML output changes
4. If server config, update .htaccess/nginx config and restart
5. After fix, request re-indexing in GSC for affected pages

CRAWL BUDGET: Pages Crawled But Not Indexed

When GSC shows many pages as "Crawled - currently not indexed":

Diagnosis:

• Are these thin/duplicate/low-quality pages?
• Is the site very large (100K+ pages)?
• Are important pages buried deep in site architecture?
• Is crawl budget being wasted on faceted navigation, infinite scrolls, or parameter URLs?

Fix approach:

1. Consolidate thin/duplicate content
2. Improve internal linking to important pages
3. Use noindex or robots.txt to prevent crawling of low-value pages (facets, filters, sorts)
4. Ensure important pages are within 3 clicks of homepage
5. Improve page quality signals (content depth, E-E-A-T)

Phase 3: GSC MCP Integration

Tool Discovery

STEP 1: Discover available GSC tools
gateway_search_tools: query = "google search console"
-- or --
gateway_search_tools: query = "searchconsole"

STEP 2: Get schema for specific tool
gateway_get_tool_schema: tool_name = "[tool_name_from_step_1]"

Common GSC Operations via MCP

SEARCH ANALYTICS (Traffic Data):
- Query: clicks, impressions, CTR, position
- Dimensions: date, page, query, country, device
- Date range: specify start_date and end_date
- Filters: page contains/equals, query contains/equals

URL INSPECTION:
- Inspect specific URL for indexing status
- Get Google's canonical selection
- Check mobile usability
- See last crawl date

SITEMAP MANAGEMENT:
- List submitted sitemaps
- Submit new sitemap
- Delete sitemap
- Check sitemap status (submitted vs indexed counts)

COVERAGE REPORT (if available):
- Get indexing status breakdown
- See exclusion reasons and affected URLs

Data Interpretation Guidelines

SEARCH ANALYTICS GOTCHAS:
- Data has 24-72 hour delay (recent days may be incomplete)
- "Position" is average -- a page can rank #3 for one query and #50 for another
- "Impressions" means the URL appeared in results, NOT that the user saw it
- Property type matters: domain property captures all variants, URL prefix only captures that prefix
- Anonymous queries (privacy threshold) are excluded from query-level data but included in totals

COVERAGE REPORT GOTCHAS:
- "Valid" count can include pages indexed but not receiving traffic
- Exclusion reasons can overlap (a page can be both noindex AND non-canonical)
- Coverage data is per-URL, not per-page (www and non-www are separate URLs)

Phase 4: Hostinger Deployment Awareness

When deploying SEO fixes to a Hostinger-hosted site, follow this protocol to avoid the OPcache/CDN friction that has caused repeated deployment cycles in past sessions.

Pre-Deploy SEO Fix Checklist

1. Document current state:
   - Save current robots.txt content
   - Save current sitemap URLs and structure
   - Note current canonical patterns
   - Screenshot GSC Coverage stats as baseline

2. Stage changes in Git:
   - All SEO-relevant files (robots.txt, sitemaps, templates with canonical/hreflang)
   - Commit with descriptive message: "fix(seo): correct canonical URLs for /section/"

3. Deploy via hPanel Git mechanism:
   - NEVER create/edit files via SSH (invisible to web server due to OPcache)
   - Push to tracked branch -> hPanel auto-deploys
   - Use hostinger_hosting_deploy* MCP tools if available

Post-Deploy SEO Verification (CRITICAL)

# Wait for OPcache to clear (minimum 60 seconds)
sleep 60

# Verify at SERVER level -- NOT browser (CDN caches old content)
# Check canonical tag is updated
curl -s "https://example.com/fixed-page" | grep -i 'canonical'

# Check robots.txt is updated
curl -s "https://example.com/robots.txt"

# Check sitemap is accessible
curl -sI "https://example.com/sitemap.xml" | head -5

# Check meta robots tag is removed (if that was the fix)
curl -s "https://example.com/fixed-page" | grep -i 'noindex'

# Check hreflang tags are correct
curl -s "https://example.com/fixed-page" | grep -i 'hreflang'

Hostinger-Specific Friction Points

After Deployment: GSC Follow-Up

1. REQUEST INDEXING for key fixed pages via GSC URL Inspection
   - Prioritize high-traffic pages first
   - Max ~10-20 URL inspection requests per day (soft limit)

2. SUBMIT/RESUBMIT sitemaps if sitemap was changed
   - Use GSC Sitemaps tool to submit updated sitemap
   - Monitor "Submitted" vs "Indexed" count over next 7 days

3. MONITOR Coverage Report daily for 7 days
   - Watch for "Valid" count to increase
   - Watch for exclusion reasons to decrease
   - Note: Full re-indexing can take 2-4 weeks for large sites

4. MONITOR Search Analytics after 3-5 days
   - Compare clicks/impressions to pre-fix baseline
   - Recovery is gradual -- don't expect overnight return to previous traffic

Phase 5: Large-Scale SEO Audits

For sites with 10,000+ pages, standard page-by-page checking is impractical. Use this sampling and batching approach.

Sampling Strategy

1. Get full URL list from sitemap(s)
2. Categorize by section/template (e.g., /products/, /blog/, /categories/)
3. Sample 5-10 URLs per section for spot-checks
4. If a section has issues, then audit the full section
5. Focus effort on highest-traffic sections first (use Search Analytics to prioritize)

Batch Analysis Scripts

# Extract all URLs from sitemap index
curl -s "https://example.com/sitemap_index.xml" | grep -oP '<loc>\K[^<]+' | while read sitemap_url; do
  echo "--- $sitemap_url ---"
  curl -s "$sitemap_url" | grep -c '<loc>'
done

# Batch canonical check (outputs mismatches only)
curl -s "https://example.com/sitemap.xml" | grep -oP '<loc>\K[^<]+' | while read url; do
  canonical=$(curl -s "$url" | grep -oP 'href="\K[^"]+(?="[^>]*rel="canonical")' | head -1)
  if [ -z "$canonical" ]; then
    canonical=$(curl -s "$url" | grep -oP 'rel="canonical"[^>]*href="\K[^"]+' | head -1)
  fi
  if [ -n "$canonical" ] && [ "$canonical" != "$url" ]; then
    echo "MISMATCH: $url -> $canonical"
  fi
done

# Batch HTTP status check
curl -s "https://example.com/sitemap.xml" | grep -oP '<loc>\K[^<]+' | while read url; do
  status=$(curl -sI -o /dev/null -w "%{http_code}" "$url")
  if [ "$status" != "200" ]; then
    echo "$status $url"
  fi
done

Verification Checklist

After implementing fixes, verify each item:

• Phase 0 completed: Domain, timeline, severity, and scope confirmed with user
• GSC data pulled: Search Analytics and Coverage data retrieved (not assumed)
• Root cause identified: Specific technical issue pinpointed with evidence
• Fix implemented: Changes made to correct the issue
• Fix deployed correctly: Via Git/hPanel (NOT SSH file creation on Hostinger)
• Server-side verified: curl confirms fix is live (not relying on browser/CDN)
• Canonical tags consistent: Canonical URL matches sitemap URL, hreflang URL, and internal link URL
• Sitemaps clean: No 4xx/5xx/noindex/non-canonical URLs in sitemaps
• robots.txt permissive: Not blocking indexable content or CSS/JS
• No accidental noindex: Neither meta tag nor X-Robots-Tag header on indexable pages
• Hreflang reciprocal: Every hreflang link has a matching return link
• GSC re-indexing requested: URL Inspection used to request indexing for key pages
• Sitemaps submitted: Updated sitemaps submitted/resubmitted in GSC
• Monitoring plan: User knows to check GSC Coverage and Search Analytics over next 7-14 days
• Current state documented: Baseline metrics saved for comparison

Recovery Timeline Expectations

Set correct expectations with the user:

IMPORTANT: SEO recovery is NEVER instant. A correctly deployed fix may take weeks to reflect in traffic. Do NOT keep changing things because traffic hasn't recovered after 2 days.

Key Principle

Data first, fix second, patience third. Pull GSC data before diagnosing. Verify HTML source before assuming canonical/hreflang state. Deploy once via Git, verify via curl, then wait. SEO recovery takes weeks, not hours. The biggest mistake is panic-redeploying or making additional changes before the first fix has been crawled and processed by Google.