psyduckler AEO AI Overview Simulator
Source: github.com/psyduckler/aeo-skills Part of: AEO Skills Suite
Simulate Google AI Overviews by running prompts through the same model that powers them — Gemini 3 Flash with Google Search grounding.
Why This Works
Google's AI Overviews and Search AI Mode are powered by Gemini 3 Flash (gemini-3-flash-preview). When you run a prompt through this model with google_search grounding enabled, you're simulating exactly what happens when a user triggers an AI Overview in Google Search. The model searches the web, selects sources, and generates a grounded answer — the same pipeline Google uses.
Influence happens at retrieval, not inside the model. You can't edit training data, but you can enter the "candidate set" the model selects from when it searches. Gemini is search-first — it searches before nearly every answer, making it more influenceable than GPT. This simulator reveals the recurring retrieval set — the sources, themes, and response patterns the model consistently draws from.
Running 20 samples per prompt captures the probabilistic nature of AI responses. A source cited in 15/20 runs is reliably featured; one cited in 3/20 is marginal. This frequency data is far more actionable than a single snapshot.
Requirements
- Gemini API key (free from aistudio.google.com) — set as
GEMINI_API_KEYenv var - Python 3.9+
- No pip dependencies (stdlib only)
Usage
# Basic — simulate an AI Overview for a query
GEMINI_API_KEY="$GEMINI_API_KEY" python3 scripts/simulate.py "best project management tools for startups"
# Track a specific domain
GEMINI_API_KEY="$GEMINI_API_KEY" python3 scripts/simulate.py "best project management tools for startups" --domain monday.com
# With response text diffing — see stable core vs volatile insertion points
GEMINI_API_KEY="$GEMINI_API_KEY" python3 scripts/simulate.py "best project management tools for startups" --domain monday.com --diff
# More runs for higher confidence
GEMINI_API_KEY="$GEMINI_API_KEY" python3 scripts/simulate.py "best project management tools for startups" --domain monday.com --runs 40
# JSON output for programmatic use
GEMINI_API_KEY="$GEMINI_API_KEY" python3 scripts/simulate.py "best project management tools for startups" --output json
Run from the skill directory. Resolve scripts/simulate.py relative to this SKILL.md.
Options
| Option | Default | Description |
|---|---|---|
prompt |
(required) | The query to simulate |
--domain |
(none) | Domain to track (e.g., example.com) |
--runs |
20 | Number of simulation runs |
--model |
gemini-3-flash-preview |
Gemini model to use |
--concurrency |
5 | Max parallel API calls (keep ≤5 to avoid rate limits) |
--output |
text |
Output format: text or json |
--diff |
off | Enable response text diffing — analyze sentence stability across runs |
Output
Text Output
AI Overview Simulation: "best project management tools for startups"
Model: gemini-3-flash-preview
Runs: 20/20 successful
Top Cited Sources:
============================================================
85% (17/20) — monday.com
70% (14/20) — asana.com
55% (11/20) — clickup.com
40% (8/20) — notion.so
25% (5/20) — trello.com
Search Queries Used:
============================================================
90% (18/20) — best project management tools for startups
45% (9/20) — project management software comparison
30% (6/20) — startup project management tools 2025
Domain Tracking: monday.com
============================================================
Mention rate: 90% (18/20 runs)
Citation rate: 85% (17/20 runs)
Cited URLs:
12x — https://monday.com/blog/project-management/best-tools/
5x — https://monday.com/s/project-management
Excerpts:
- "Monday.com offers an intuitive interface ideal for startups..."
- "For budget-conscious startups, Monday.com's free tier..."
JSON Output
Structured JSON with sources, queries, domain_tracking, and per-run detail.
Response Text Diffing (--diff)
When --diff is enabled, the simulator analyzes the response text across all runs to reveal the stable core vs volatile periphery of the recurring retrieval set:
- Stable sentences (>80%) — appear in almost every run. These form the core of the AI Overview and are hard to displace. They're backed by entrenched sources in the recurring retrieval set.
- Common sentences (40-80%) — usually included but not locked in. These are the mid-tier — present more often than not.
- Volatile sentences (10-40%) — the insertion points where your content can have impact. These sentences change between runs because the model is choosing between competing sources. Create content that wins these competitive slots.
- Rare sentences (<10%) — edge cases the model occasionally surfaces. Low priority.
- Entity volatility — brands, tools, and domains that appear inconsistently. These are competitive slots — the model is undecided about which entity to feature, meaning your brand can compete for these mentions.
Text Output with --diff
Response Stability Analysis:
============================================================
Stable (>80% of runs — core of the AI Overview, hard to displace):
[95%] Project management tools help startups organize tasks and track progress...
[90%] Key features to look for include task management, collaboration, and...
Volatile (10-40% — insertion points, your content can influence these):
[35%] For bootstrapped startups, free tiers from tools like ClickUp offer...
[25%] Integration with existing tools like Slack and Google Workspace is...
Entity Volatility (brands/entities that appear inconsistently — competitive slots):
[40%] ClickUp (8/20 runs)
[30%] Basecamp (6/20 runs)
[20%] Linear (4/20 runs)
JSON Output with --diff
Adds a response_analysis object with stable_sentences, common_sentences, volatile_sentences, rare_sentences, and entity_frequency arrays.
Note: --diff adds processing time since it compares sentences across all runs. Off by default.
How It Works
- Sends the prompt to Gemini 3 Flash with
google_searchgrounding tool enabled - Repeats this 20 times (default) — each run is independent
- From each response, extracts:
- Response text — the generated AI Overview
- Grounding sources — URLs and titles of cited web pages
- Search queries — what queries the model fired against Google Search
- Grounding support — which parts of the response are supported by which sources
- Aggregates across all runs:
- Source citation frequency (which domains/URLs appear most)
- Search query frequency (what the model searches for)
- Response consistency (how stable the answer is)
- If
--domainprovided:- Calculates mention rate (brand name in response text)
- Calculates citation rate (domain in grounding sources)
- Extracts exact excerpts where the domain is referenced
Tips
- Start with your most important queries — the ones you'd want to appear in AI Overviews for
- Compare citation rates before and after content changes (wait 2-4 weeks for re-indexing)
- A citation rate >50% means you're reliably featured; <20% means you're marginal
- Use
--output jsonto feed results into other AEO skills (analytics, gap finder) - Pair with
aeo-content-freeto improve content that isn't getting cited - Pair with
aeo-analytics-freeto track citation rates over time
Further Reading
Notes
- Gemini API key stored in macOS Keychain under
google-api-key - Retries failed requests up to 5 times with exponential backoff
- Rate-limited to respect API quotas — keep
--concurrency≤5 - Results may vary between runs — that's expected and exactly why we sample multiple times