Programmatic SEO Guardrails
Generate a page only when the data makes it genuinely different and useful; gate everything else.
Google's spam policies define scaled content abuse as "many pages generated for the primary purpose of manipulating search rankings and not helping users", regardless of whether people, templates, or AI produce them. The method is not the problem; the lack of value is.
When to Use
- Designing a page template that will be filled from a database or API
- Reviewing an existing set of hundreds or thousands of templated pages
- Planning AI-assisted bulk content
- Indexing has stalled on a large templated section ("Crawled - currently not indexed")
When NOT to Use
- A handful of hand-written pages (normal editorial review is enough)
- Crawl or rendering bugs (use
technical-seo-audit)
Know the Policies You Are Designing Against
Summarized from Google Search spam policies (checked 2026-11-04):
| Policy |
What it describes |
Common pSEO trigger |
| Scaled content abuse |
Many pages made mainly to manipulate rankings, including AI-generated pages without added value, scraped or stitched content |
One template, swap a keyword, ship 10,000 pages |
| Doorway abuse |
Pages created to rank for similar queries that funnel users to a less useful intermediate page |
"Plumber in {city}" pages with identical copy that all link to one booking form |
| Thin affiliation |
Affiliate pages copying merchant descriptions with no original value |
Product pages built from a merchant feed only |
| Keyword stuffing |
Lists of cities/regions or repeated phrases meant to rank |
"We serve: {200 city names}" blocks |
| Site reputation abuse |
Third-party pages hosted mainly to exploit the host site's ranking signals |
Sponsored partner directory unrelated to the site |
Google's guidance on generative AI: using AI to research or structure original content is fine; generating many pages without adding value may violate the scaled content policy. All AI-generated content, including titles, descriptions, structured data, and alt text, should be fact-checked before publishing.
Workflow
Step 1: Pass the data-uniqueness test
Before writing a template, answer for a random sample of 20 rows:
- What does this page tell a user that no sibling page tells them?
- Is that difference from data (prices, specs, inventory, reviews, local facts, real usage stats) rather than reworded boilerplate?
- Would someone landing here from search be able to finish their task here, or would they have to click elsewhere?
If the honest answer to (1) is "the keyword", do not build the page set. Build one strong hub page instead.
Step 2: Design the template around unique fields
Write the template so most of the visible content comes from row-specific fields:
type PageRow = {
slug: string;
title: string;
uniqueFacts: string[]; // at least N verified, row-specific facts
comparisonTable?: Row[]; // real data, not filler
firstPartyNotes?: string; // written or reviewed by a person
sources: string[]; // where the facts came from
lastVerified: string; // ISO date
};
Avoid templates where the only variables are {city} or {tool} inside otherwise identical paragraphs.
Step 3: Add per-page quality gates
Decide in code whether each page is indexable.
function indexDecision(row: PageRow): "index" | "noindex" | "skip" {
if (row.uniqueFacts.length === 0) return "skip"; // do not generate
if (row.uniqueFacts.length < MIN_FACTS) return "noindex"; // publish for users, keep out of index
if (!row.sources.length) return "noindex";
if (isNearDuplicate(row)) return "noindex"; // or canonicalize to the parent
return "index";
}
Tune MIN_FACTS and the near-duplicate threshold per template by reading pages, not by assuming a number.
Implementation:
noindex pages: <meta name="robots" content="noindex"> and exclude them from the sitemap. Do not block them in robots.txt, or Google cannot see the noindex.
skip pages: do not generate; return 404 if the URL was ever linked.
- Near-duplicates that serve users (e.g. sort/filter variants):
rel="canonical" to the main version.
Step 4: Human review sample
Before launch and on every template change:
- Read 20 random indexable pages end to end. Fact-check every number against its source.
- Read the 5 thinnest indexable pages (lowest unique-field count). If they embarrass you, raise the gate.
- Check that titles and meta descriptions are distinct across the set.
Step 5: Stage the rollout
- Publish a small batch (the strongest pages) and submit a sitemap for just that section.
- Watch Search Console's page indexing report for that sitemap: indexed vs "Crawled - currently not indexed" vs "Duplicate, Google chose different canonical".
- Expand only if the first batch is being indexed and getting engagement. If Google is not indexing it, adding more pages of the same kind will not help; improve the template or prune.
Step 6: Maintain or prune
- Re-verify data on a schedule; show
lastVerified on the page when facts change often.
- Pages whose data goes stale or empty: update,
noindex, or remove (404/410), or 301 to the closest useful page.
- Review the section as a whole, not just page by page. Google's helpful content self-assessment asks whether you are "producing lots of content on many different topics in hopes that some of it might perform well in search results" and "using extensive automation to produce content on many topics".
Page Types: Quick Guidance
| Page type |
Good when |
Risky when |
| Integration pages |
Each page documents real setup steps, scopes, limits |
Same paragraph with logo swapped |
| Location pages |
Real local data: address, hours, staff, inventory, local pricing |
No physical presence or local service; city name swapped |
| "X vs Y" / alternatives |
Honest, sourced comparison with dates; admits where X wins |
Every page concludes "use our product"; unverified competitor claims |
| Glossary |
Definitions with examples and links to deeper pages |
One-line definitions scraped or generated in bulk |
| Data/stat pages |
First-party data with method stated |
Rewritten third-party stats |
Best Practices
- Start from data you own or have verified; the template is the last step.
- Gate indexability in code and keep gated pages out of sitemaps.
- Never use robots.txt to hide thin pages you also
noindex.
- Fact-check AI-assisted text, metadata, and structured data before publishing.
- Roll out in batches and let indexing data decide expansion.
- Prefer one excellent hub over hundreds of interchangeable leaves.
- Disclose how content was produced when it helps readers trust it.
- Never promise traffic from page count.
References
Checked 2026-11-04:
1---2name: programmatic-seo-guardrails3description: |4 Guardrails for generating many pages from data or templates (locations, integrations, comparisons, glossaries, AI-assisted pages) without creating thin, doorway, or scaled-content-abuse pages under Google's spam policies. Covers a data-uniqueness test, per-page quality gates, noindex-until-useful, canonicalization of near-duplicates, and staged rollout with monitoring.5 Use this skill when planning or reviewing template-driven pages, city/location pages, "X vs Y" or "X alternatives" pages, integration directories, or bulk AI-written content.6 Activate when: programmatic SEO, pSEO, template pages, location pages, city pages, comparison pages, alternatives pages, glossary pages, scaled content, AI-generated pages, thin content, doorway pages, bulk pages.7metadata:8 source: https://skillmd.com/skills/latestaiagents/programmatic-seo-guardrails9---1011# Programmatic SEO Guardrails1213**Generate a page only when the data makes it genuinely different and useful; gate everything else.**1415Google's spam policies define scaled content abuse as "many pages generated for the primary purpose of manipulating search rankings and not helping users", regardless of whether people, templates, or AI produce them. The method is not the problem; the lack of value is.1617## When to Use1819- Designing a page template that will be filled from a database or API20- Reviewing an existing set of hundreds or thousands of templated pages21- Planning AI-assisted bulk content22- Indexing has stalled on a large templated section ("Crawled - currently not indexed")2324## When NOT to Use2526- A handful of hand-written pages (normal editorial review is enough)27- Crawl or rendering bugs (use `technical-seo-audit`)2829## Know the Policies You Are Designing Against3031Summarized from Google Search spam policies (checked 2026-11-04):3233| Policy | What it describes | Common pSEO trigger |34|--------|-------------------|---------------------|35| Scaled content abuse | Many pages made mainly to manipulate rankings, including AI-generated pages without added value, scraped or stitched content | One template, swap a keyword, ship 10,000 pages |36| Doorway abuse | Pages created to rank for similar queries that funnel users to a less useful intermediate page | "Plumber in {city}" pages with identical copy that all link to one booking form |37| Thin affiliation | Affiliate pages copying merchant descriptions with no original value | Product pages built from a merchant feed only |38| Keyword stuffing | Lists of cities/regions or repeated phrases meant to rank | "We serve: {200 city names}" blocks |39| Site reputation abuse | Third-party pages hosted mainly to exploit the host site's ranking signals | Sponsored partner directory unrelated to the site |4041Google's guidance on generative AI: using AI to research or structure original content is fine; generating many pages without adding value may violate the scaled content policy. All AI-generated content, including titles, descriptions, structured data, and alt text, should be fact-checked before publishing.4243## Workflow4445### Step 1: Pass the data-uniqueness test4647Before writing a template, answer for a random sample of 20 rows:48491. What does this page tell a user that no sibling page tells them?502. Is that difference from **data** (prices, specs, inventory, reviews, local facts, real usage stats) rather than reworded boilerplate?513. Would someone landing here from search be able to finish their task here, or would they have to click elsewhere?5253If the honest answer to (1) is "the keyword", do not build the page set. Build one strong hub page instead.5455### Step 2: Design the template around unique fields5657Write the template so most of the visible content comes from row-specific fields:5859```ts60type PageRow = {61 slug: string;62 title: string;63 uniqueFacts: string[]; // at least N verified, row-specific facts64 comparisonTable?: Row[]; // real data, not filler65 firstPartyNotes?: string; // written or reviewed by a person66 sources: string[]; // where the facts came from67 lastVerified: string; // ISO date68};69```7071Avoid templates where the only variables are `{city}` or `{tool}` inside otherwise identical paragraphs.7273### Step 3: Add per-page quality gates7475Decide in code whether each page is indexable.7677```ts78function indexDecision(row: PageRow): "index" | "noindex" | "skip" {79 if (row.uniqueFacts.length === 0) return "skip"; // do not generate80 if (row.uniqueFacts.length < MIN_FACTS) return "noindex"; // publish for users, keep out of index81 if (!row.sources.length) return "noindex";82 if (isNearDuplicate(row)) return "noindex"; // or canonicalize to the parent83 return "index";84}85```8687Tune `MIN_FACTS` and the near-duplicate threshold per template by reading pages, not by assuming a number.8889Implementation:9091- `noindex` pages: `<meta name="robots" content="noindex">` and exclude them from the sitemap. Do not block them in robots.txt, or Google cannot see the `noindex`.92- `skip` pages: do not generate; return 404 if the URL was ever linked.93- Near-duplicates that serve users (e.g. sort/filter variants): `rel="canonical"` to the main version.9495### Step 4: Human review sample9697Before launch and on every template change:9899- Read 20 random indexable pages end to end. Fact-check every number against its source.100- Read the 5 thinnest indexable pages (lowest unique-field count). If they embarrass you, raise the gate.101- Check that titles and meta descriptions are distinct across the set.102103### Step 5: Stage the rollout1041051. Publish a small batch (the strongest pages) and submit a sitemap for just that section.1062. Watch Search Console's page indexing report for that sitemap: indexed vs "Crawled - currently not indexed" vs "Duplicate, Google chose different canonical".1073. Expand only if the first batch is being indexed and getting engagement. If Google is not indexing it, adding more pages of the same kind will not help; improve the template or prune.108109### Step 6: Maintain or prune110111- Re-verify data on a schedule; show `lastVerified` on the page when facts change often.112- Pages whose data goes stale or empty: update, `noindex`, or remove (404/410), or 301 to the closest useful page.113- Review the section as a whole, not just page by page. Google's helpful content self-assessment asks whether you are "producing lots of content on many different topics in hopes that some of it might perform well in search results" and "using extensive automation to produce content on many topics".114115## Page Types: Quick Guidance116117| Page type | Good when | Risky when |118|-----------|-----------|-----------|119| Integration pages | Each page documents real setup steps, scopes, limits | Same paragraph with logo swapped |120| Location pages | Real local data: address, hours, staff, inventory, local pricing | No physical presence or local service; city name swapped |121| "X vs Y" / alternatives | Honest, sourced comparison with dates; admits where X wins | Every page concludes "use our product"; unverified competitor claims |122| Glossary | Definitions with examples and links to deeper pages | One-line definitions scraped or generated in bulk |123| Data/stat pages | First-party data with method stated | Rewritten third-party stats |124125## Best Practices1261271. Start from data you own or have verified; the template is the last step.1282. Gate indexability in code and keep gated pages out of sitemaps.1293. Never use robots.txt to hide thin pages you also `noindex`.1304. Fact-check AI-assisted text, metadata, and structured data before publishing.1315. Roll out in batches and let indexing data decide expansion.1326. Prefer one excellent hub over hundreds of interchangeable leaves.1337. Disclose how content was produced when it helps readers trust it.1348. Never promise traffic from page count.135136## References137138Checked 2026-11-04:139140- Google Search spam policies: https://developers.google.com/search/docs/essentials/spam-policies141- Using generative AI content: https://developers.google.com/search/docs/fundamentals/using-gen-ai-content142- Creating helpful, reliable, people-first content: https://developers.google.com/search/docs/fundamentals/creating-helpful-content143- Block indexing with noindex: https://developers.google.com/search/docs/crawling-indexing/block-indexing144- Canonicalization: https://developers.google.com/search/docs/crawling-indexing/consolidate-duplicate-urls145- Sitemaps: https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap