SEO Keyword Research

Overview

Most keyword research dies at the head term. A site with DR 25 chasing "project management software" (KD 72) loses to Asana and Monday.com forever — but the same audience also searches "free task tracker for two-person agencies" (KD 8, weak SERPs, transactional intent) and that's the keyword class that converts on month-three traffic instead of year-three. This skill is the funnel that finds those keywords reliably: it forces the seed → modifier expansion → SERP-mined patterns → competitor-gap pass, then qualifies survivors against a SERP-authority rubric that beats raw KD scores, and outputs a prioritized brief mapped to content types and intent.

Mental model

Long-tail wins on three compounding axes that head-term chasing loses:

1. SERP authority profile beats KD score. KD is a dumb average — what matters is whether the actual top-10 are beatable. A "KD 35" keyword with three 1,500-word generic AI posts on top is more winnable than a "KD 18" keyword where the top 3 are a Wikipedia page, a 5-year-old definitive guide, and the official product page.
2. Intent specificity is the conversion lever. "[head term]" attracts everyone; "[head term] for [audience] under [constraint]" attracts only buyers. A 100-volume long-tail with 8% conversion beats a 10,000-volume head term with 0.2%.
3. Long-tail clusters earn topical authority. Five 200-volume long-tails covering the same sub-topic, internally linked to a pillar, signal expertise more than one head-term page. The Helpful Content System (folded into core algorithm 2024) and information-gain scoring both reward depth, original data, and entity-coverage breadth — not surface listicles.
4. Query fan-out makes long-tail coverage the design itself. Google's 2026 AI optimization guide confirms AI features fire concurrent sub-queries per prompt (Google's example: a weeds prompt expands to "best herbicides for lawns" + "how to prevent dandelions growing back"). Long-tail clusters are literally what fan-out hits — seed → modifier expansion now designs for fan-out coverage, not volume.

The funnel below operationalizes those four axes. Skip any stage and you're back to keyword-research-as-brainstorm — picking what feels obvious, missing what's winnable.

2026 algorithmic context (kill-rules upstream of the funnel)

Instructions

Core rules

• Long-tail = 4+ words OR a clear intent-specific modifier on a head term. "Best CRM" is short-tail. "Best CRM for solo financial advisors" is long-tail. The shift to specificity is what matters, not raw word count.
• SERP review is non-negotiable. For every keyword that survives expansion, look at the actual top 10. KD scores from any tool (Ahrefs, SEMrush, SE Ranking, Serpstat) are a starting estimate, not a verdict. The SERP tells you whether the keyword is beatable for THIS site.
• Realism rule (heuristic, not gospel): target keywords where the top-10 average DR is roughly within +15 of your DR. A DR-25 site can credibly rank against DR-10 to DR-40 sites; chasing keywords where the top 10 average DR 60+ is a multi-year project, not a quick win. Adjust the +15 band by content-quality gap (top-10 are weak/old → push higher; top-10 are definitive → push lower).
• Information gain is the on-page kill rule. If you cannot name what the new piece adds that the top 10 doesn't have (data, perspective, structure, freshness), the keyword isn't yours yet. Picking the keyword without an angle is the same mistake as picking a head term without DR.
• Intent must match the site's commercial position. Informational keywords on a transactional product page = mismatch (won't rank, won't convert). Commercial-investigation keywords on a thin glossary page = mismatch. Always classify intent BEFORE assigning content type.
• Five+ candidates per sub-topic, not one. Single-keyword targeting anchors to whatever came first. Always generate five long-tails per sub-topic and rank — even if four are dead ends, the comparison clarifies why the survivor wins.
• Zero-volume keywords are a real category in 2026. AI-search shifts demand to long-tails that volume tools mark "0". A 0-volume keyword ranking for 12 related semantic queries can earn 200+ visits/month. Don't auto-strike on volume = 0; check related-keyword overlap.
• GEO is not a separate discipline anymore — it's the second axis of every modern keyword. Definition / question / comparison / list queries that produce extractable answers earn AI-search citations even when traditional rank is mid-page. Note GEO potential per keyword; the on-page execution (answer-first, FAQ schema, extractable passages) is owned by another domain.
• Output a brief, not a verdict. The deliverable is a written keyword brief covering: prioritized list, intent + content-type mapping, cluster grouping, entity-coverage notes, and content-calendar slot. Writing, schema, internal-link wiring, and outreach belong to downstream domains.
• Compound over spike. Prefer evergreen long-tails over trend-cycle keywords. Trend keywords decay in 12 months; "how to [permanent task]" compounds for years.

The funnel

Stage 1 — Profile the site (kill or proceed)

Force concrete answers before generating any keyword list. If unclear, stop and ask.

1. Niche — one sentence. Sub-niche if applicable.
2. Audience — who actually buys/converts? Specific segment, not "small businesses."
3. Current DR / authority — rough estimate (Ahrefs free tool, Moz Link Explorer, or "we're new"). This sets the realism cap.
4. Existing content — count and topical coverage. New site = greenfield. Existing site = check what already ranks (Search Console queries) before generating overlap.
5. Commercial position — informational/affiliate/SaaS/e-commerce/local-services. Determines which intent classes to prioritize.
6. Geographic + language targeting — global English, US-only, multi-region, non-English. Affects SERP review and modifier set.

If the user can't name DR or commercial position, kill the engagement at this stage. Without those, qualification is blind: "I'd be guessing whether the top 10 are beatable. Get a DR estimate (Ahrefs free Backlink Checker takes 30 seconds) and tell me what conversion looks like, then I'll run the funnel."

Stage 2 — Generate seed keywords (parallel)

PARALLEL-FIRST: fire all seed sources simultaneously in ONE block. Don't serialize.

Seed sources:

• Core product/service nouns — what the site actually sells, in the language buyers use (not internal jargon).
• Problem-focused phrases — "[problem] when [context]", "[symptom] in [audience]", "stop [bad thing]".
• Solution-focused phrases — "how to [task]", "[task] without [obstacle]", "alternative to [incumbent]".
• Audience qualifiers — "[product] for [audience segment]", "[product] for [team size]", "[product] for [use case]".
• Industry / vertical terms — terminology specific to the buyer's world.
• Existing-content mining — Search Console "queries" report → keywords already pulling impressions on existing pages but ranking page 2–5. Each is a seed for related long-tails.
• Competitor SERP gaps — pick 3 competitors at the same DR tier; pull their top-200 ranking keywords (Ahrefs Site Explorer / SE Ranking competitor research / SEMrush Domain Overview); filter for keywords where the user's site does NOT rank. Each is a seed.

Output of Stage 2: a flat list of 30–80 seed terms. Don't qualify yet — qualification comes after expansion.

Stage 3 — Expand to long-tail (modifier patterns + SERP mining)

For each seed, generate 5–15 long-tail variants. Run in parallel; don't expand one seed at a time.

Modifier buckets (full library + SERP-mined sources in references/modifier-library.md):

• Question modifiers — how to, what is, why, when, which × seed. Highest GEO value.
• Intent modifiers — best, top, free, cheap, vs, alternative to, under $X. Commercial/transactional layer.
• Constraint modifiers — for [audience size], for [vertical], in [year], without [obstacle], with [feature]. Highest-yield for low-DR sites — every constraint kills a competing page.
• Outcome modifiers — in [time bound], without [obstacle], roi / cost, mistakes to avoid.
• Comparison modifiers — vs, or, difference between, when to use A vs B. Mid-funnel + AI-citation-friendly.
• Problem-trigger modifiers — why is my [thing] broken, error messages, support-forum phrasings. Highest conversion when product solves the pain.

SERP-mined sources (free, parallel — fire all in one block):

• Google autocomplete (incognito) — type seed + each letter a–z, capture suggestions.
• Google "People Also Ask" — expand 2–3 PAA branches per seed; each yields 3–5 related questions.
• Google "Related searches" — bottom-of-SERP carousel.
• Google Trends "Related queries (rising)" — momentum signals before tools assign volume.
• Reddit / forum mining — site:reddit.com [seed] for real user phrasing; Quora, Stack Exchange, niche forums for verticals where Reddit is thin.
• Lowfruits — purpose-built tool for low-competition discovery (identifies SERPs dominated by weak/forum content).
• Glimpse / Exploding Topics — emerging keywords pre-volume.

Output of Stage 3: 200–500 long-tail candidates. Dedup at Stage 4.

Stage 4 — Qualify each long-tail (SERP-first rubric)

This is the stage that separates winnable keywords from KD-low-but-unwinnable ones. Walk top-to-bottom; STRIKE on the first failure (it's a kill-rule list, not a score).

A long-tail that survives all 12 → eligible for the brief. Anything else → strike.

For zero-volume candidates: replace check #2 with "does this keyword appear in 5+ Reddit threads OR show up in PAA / autocomplete?" If yes, it's a real query with no tool data — keep. If no, strike.

Stage 5 — Cluster, map entity coverage, prioritize

Group survivors into topical clusters (one pillar + 4–8 cluster pages each), then layer entity-coverage onto each cluster:

1. Identify pillar candidates — broad seed terms with the highest survivor count clustered around them.

2. Group long-tail survivors under their pillar by sub-topic.

3. Map entity coverage per cluster — list the named entities, sub-topics, and related concepts the pillar must cover for topical authority. Source: top-10 ranking pages' H2/H3 structure (n-gram-mine them), Wikipedia "See also" sections for the seed, knowledge-panel related entities. A pillar that misses 3+ of the entities the top-10 all cover signals shallow coverage to ranking systems.

4. Score each survivor on:

   • Realism — top-10 SERP authority profile vs user DR (1 = unwinnable, 5 = clearly beatable)
   • Intent value — informational = 1, navigational = 1, commercial = 2, transactional = 3 (multiply by site's commercial position relevance)
   • GEO potential — definition / how-to / comparison / list = 1 each, none = 0 (max 4)
   • Cluster fit — pillar = 5, primary cluster page = 4, secondary cluster = 3, orphan = 1
   • AIO discount — if AI Overview present on the SERP for an informational keyword AND no plan to win the citation, subtract 2 (CTR floor risk)

   Composite priority = (realism × 3) + (intent × 2) + GEO + cluster_fit − AIO_discount. Range: 9–35.

5. Bucket into:

   • P0 — Quick wins (priority ≥ 28): publish first month
   • P1 — High-value (priority 22–27): queue for next sprint
   • P2 — Cluster fillers (priority 16–21): batch with related pillar
   • P3 — Monitor (priority < 16): not in the next quarter

A long-tail that scores 1 on realism is automatically demoted to P3 regardless of total — don't chase keywords you can't win.

Stage 6 — Output the keyword brief

Use the canonical template at references/brief-template.md. The brief covers: site profile, method notes, P0–P3 priority tables (volume, KD, SERP realism, intent, GEO, AIO, content type, cluster), topical clusters with entity-coverage notes, GEO notes (question / definition / comparison / Reddit / Bing queries), risk notes (Site Reputation Abuse, HCS, AIO cannibalization), existing-content audit (cannibalization, GSC page-2 wins), and open items for downstream domains (briefs, internal linking, schema, GEO writing, link prospecting, AI search tracking).

The brief is the deliverable. Anything not in the brief — content writing, schema wiring, internal-link execution, outreach copy — is downstream and belongs to other domains.

Decision guide — common edge cases

Tools — free vs paid

MCP-wired data sources available in this tap: serpstat, ahrefs, seo-data (SE Ranking), search-console. The free path covers the entire funnel for any user who's willing to do SERP review manually. Paid tools save time on volume + KD numbers, competitor gap analysis, and entity-coverage mining; they do NOT replace the manual SERP review in Stage 4.

Examples

Example 1: SaaS, mid-authority, finds quick wins

Profile:

• Niche: open-source self-hosted CRM
• Audience: small B2B sales teams (2–10 reps) who want data ownership
• DR: 28
• Existing content: 14 blog posts, mostly product-feature explainers
• Commercial: SaaS (free open-source + paid managed hosting tier)
• Geo: global English

Stage 2 seeds (sample — pulled in parallel from product nouns + Search Console queries + competitor gap):

• self-hosted CRM, open source CRM, CRM data ownership, lightweight CRM, simple CRM, CRM for small teams, CRM alternative, CRM without subscription, CRM under $X, GDPR compliant CRM, [competitor] alternative, [competitor] vs [user product]

Stage 3 expansion (sample of 60 long-tails generated):

• "self-hosted CRM for sales teams under 10"
• "best open source CRM 2026 self hosted"
• "CRM with data export to PostgreSQL"
• "Hubspot alternative open source self-hosted"
• "how to migrate from Hubspot to self-hosted CRM"
• "lightweight CRM for two-person agency"
• "GDPR compliant CRM for European startups"
• "CRM that works offline"
• "free CRM no subscription self-hosted"
• "self-hosted CRM Docker compose"

Stage 4 qualification (sample):

Stage 5 clustering (excerpt):

• Pillar: "Self-hosted CRM: complete guide" (target keyword: "self-hosted CRM" — KD 38, target month 4–6)
• Cluster pages (P0 / P1):
  • "Self-hosted CRM for sales teams under 10" (P0)
  • "Hubspot alternative: open source self-hosted CRMs ranked" (P0)
  • "How to migrate from Hubspot to a self-hosted CRM" (P0)
  • "GDPR-compliant CRM options for European startups" (P1)
  • "Self-hosted CRM with Docker compose: 30-min setup" (P1)

Stage 6 brief: produced per template above. Internal-link map noted as a downstream task; on-page wiring + schema + outreach handed to other domains.

Example 2: New site, no DR — refuse the head term

User: "We just launched. We sell project management software. Find me 50 keywords."

Response:

Stop. At a brand-new domain (DR ~0–5), 80% of "project management software" keywords are unwinnable for 12+ months. Asana, Monday, ClickUp, Notion own the head + shoulder with decade-old backlink graphs. 50 generic PM keywords would waste your first quarter.

Two questions first:

1. Who specifically converts that's underserved by the incumbents? Name the segment (e.g., "construction subcontractors with 3–15 person crews", "law firms doing matter management"). Narrower segment = more long-tail surface area.
2. What's the 1–2-sentence reason to switch? (Feature gap, pricing model, integration, vertical workflow.) That reason becomes the Stage 3 modifier set.

If "we don't know yet" → positioning is upstream of keyword research. Pick the niche, then run the funnel.

Example 3: GSC mining for free quick wins

For any site live ≥ 3 months with Google Search Console connected: query GSC for queries with impressions > 100 and average position 8–20 (page 2 — the "quick win" zone). For each candidate, confirm intent matches the existing page (if not, it's a NEW page candidate) and run SERP review to confirm room to climb. Output two streams:

• Existing-page improvements: 200-word section additions + intent-aligned H2 typically lift page-2 to top 10 in 4–8 weeks.
• New-page candidates: queries the existing page ranks for via incidental overlap; a dedicated page outranks the parent.

GSC mining is the highest-ROI single source on this list. Always run it first when GSC is available.

Checklist

• Stage 1 site profile captured: niche, audience, DR estimate, existing coverage, commercial position, geo
• DR / commercial position confirmed before any keyword generation (kill-gate)
• Seeds generated from ≥ 4 of the 7 source categories (parallel run)
• Long-tail expansion produced 5–15 variants per seed using the modifier library
• SERP review completed manually for every candidate that survives volume + intent filter (rubric step #3)
• Realism rule applied: top-10 average DR within ~+15 of user DR (banded by content-quality gap)
• AIO presence checked per keyword; informational keywords with AIO and no citation-win plan are demoted
• Site Reputation Abuse risk flagged on "best [thing] [year]" keywords where Forbes/CNN/Times brand pages dominate
• Information gain articulated per P0 keyword — what does the new piece add that the top 10 doesn't?
• Zero-volume candidates either kept (with Reddit/PAA validation) or struck — no auto-skip on volume = 0
• Survivors clustered into pillar-and-cluster structures, not flat lists
• Entity coverage mapped per cluster (top-10 H2/H3 mining, Wikipedia "See also", knowledge-panel related entities)
• Priority bucketing (P0/P1/P2/P3) applied with composite formula incl. AIO discount; realism = 1 demotes to P3
• GEO potential noted per keyword (question / definition / comparison / list)
• Brief uses the canonical template (references/brief-template.md); downstream concerns listed as Open Items
• Cannibalization check flagged: are any P0 keywords already targeted by an existing page?
• AI search visibility tracking listed in Open Items for any GEO-flagged P0/P1 keyword

Composition / References

In-skill references (load on demand):

• references/modifier-library.md — full long-tail modifier library + SERP-mined sources for Stage 3
• references/brief-template.md — canonical Stage 6 brief output template + field definitions

Within-domain pairings:

• Pairs with the sibling SEO skill that prospects backlinks for the cluster pillar pages (priority for any P0 cluster that lacks link equity).
• Pairs with the sibling SEO skill that validates free-tool ideas — when a keyword cluster surfaces a recurring user problem (calculator, checker, generator), it's a tool-build trigger.

Downstream concerns (owned by other domains, not this skill):

• GEO content writing (answer-first structure, FAQ schema, extractable passages) — the brief flags GEO-eligible keywords; the writing pattern lives elsewhere.
• Content briefs, internal-link wiring, on-page schema, outreach copy — consume this brief, don't appear in it.
• AI search visibility tracking setup (Profound / Otterly / AthenaHQ / Peec) — the brief lists which keywords to track; configuration is downstream.

External authoritative sources:

• Ahrefs — long-tail keyword research
• Backlinko — keyword research definitive guide
• Google Search Central — Helpful Content guidance
• Google Search Central — Site Reputation Abuse policy
• SparkToro — zero-volume keyword research framework
• Search Engine Journal — SERP analysis for keyword qualification
• Search Engine Land — AI Overviews CTR impact studies 2024–2026
• AI Platform Citation Source Index 2026 — GEO citation distribution data
• Princeton/Georgia Tech — GEO research paper (Aggarwal et al) — original framework for AI-citation optimization