Deep Research
Use this skill to design and run repeatable research workflows that gather evidence across many sources, preserve provenance, and synthesize results into decision-ready outputs.
This skill covers both native deep-research agents (ChatGPT Deep Research, Gemini Deep Research, Perplexity Deep Research, Claude with web search) and custom agentic research pipelines (planner / searcher / verifier / synthesizer split).
ASCII Flow
research question
|
v
research plan
scope + source targets + queries + stop criteria + freshness window
|
v
evidence gathering
primary sources first + source ledger + hostile-source checks
|
v
verification
isolated verifier checks claims against ledger, not researcher context
|
v
synthesis
evidence-tiered answer + citations + contradictions + unknowns
Quick Reference
| Question |
Default |
| When to use a native agent vs custom pipeline? |
Native for ad-hoc, open-ended questions. Custom for repeatable, auditable, or multi-source workflows. |
| What is the first artifact of any research task? |
The source ledger — never the synthesis. |
| When is a source trustworthy? |
When it is a primary document with a stable URL, author attribution, and a verifiable date. |
| What stops an unbounded research loop? |
An explicit stop criterion defined before the loop starts (saturation condition or max iterations). |
| How to handle contradictory sources? |
Separate them into evidence tiers; do not resolve by averaging. |
Use This Skill When
- You need to produce a sourced comparison, brief, memo, or research dossier.
- You need to choose between a native deep-research agent and a custom pipeline.
- You want to build a repeatable, auditable research workflow with provenance.
- You need to detect hostile sources, citation laundering, or model-output-as-source.
- You need to run a verifier subagent that has not seen the researcher's context.
Do Not Use This Skill For
- Single-lookup current facts — use ordinary browsing or direct tools.
- Long-lived retrieval serving — use ai-rag.
- Local note-vault packaging — use docs-notes-retrieval.
Current Stance
The durable pattern is not "send a single query and summarize the top results."
The default architecture (as of last_validated date above):
- Plan before searching: emit a structured research plan with queries, source targets, and success criteria before any tool call.
- Ledger as contract: the source ledger is the primary artifact — synthesis is derived from it, not the other way around.
- Verifier isolation: the verifier subagent must not share context with the researcher — it reads only the ledger and checks claims independently.
- Evidence tier separation: primary sources (original docs, filings, specs) are never mixed with secondary commentary or model-generated summaries in the same evidence bucket.
- Saturation-first synthesis: synthesize only after the research loop has reached saturation (new queries return no novel facts), not on a time budget.
- Freshness windows are explicit: every claim in the final output carries a date anchor and a freshness class (stable / volatile / unknown).
- Hostile source detection is mandatory for public-web research: adversarial SEO, AI-generated content farms, and citation-laundering chains are real threats.
Patterns
Full catalog → references/patterns-catalog.md
| ID |
Name |
When to reach for it |
| P1 |
Source-ledger-as-contract |
Always — every research task |
| P2 |
Plan-then-execute loop |
Multi-query or multi-source tasks |
| P3 |
Multi-agent research swarm |
Large corpora with parallel query paths |
| P4 |
Verifier subagent |
Any claim that will reach a user or decision |
| P5 |
Freshness-window sourcing |
Time-sensitive domains (AI, market, regulatory) |
| P6 |
Evidence-tier separation |
When primary and secondary sources both exist |
| P7 |
Citation back-pointer |
Every synthesis sentence that makes a factual claim |
| P8 |
Stop-criterion-as-eval |
To prevent unbounded research loops |
| P9 |
Hostile-source detection |
Public-web research, vendor comparisons |
| P10 |
Synthesis-after-saturation |
All multi-pass research tasks |
Anti-Patterns
Full catalog → references/anti-patterns-catalog.md
| ID |
Name |
Why it matters |
| A1 |
Pre-synthesizing the answer |
Frames all subsequent evidence gathering; produces confirmation bias |
| A2 |
Single-pass search |
Misses contradictions, minority sources, and laterally relevant evidence |
| A3 |
Mixing evidence tiers |
Primary docs and blog summaries in the same bucket degrade citation quality |
| A4 |
Model-output-as-source |
LLM summaries carry hallucination risk; they are working notes, not source truth |
| A5 |
Unbounded research loop |
Without a stop criterion, cost and time grow unboundedly with no quality signal |
| A6 |
Citation laundering |
Citing a secondary article that in turn cites the primary — provenance chain breaks |
| A7 |
Verifier-with-same-context |
Verifier that read the same sources as researcher adds no independence |
| A8 |
Vague consensus language |
"Multiple sources say" without naming them is unfalsifiable |
| A9 |
Date-free volatile claims |
Time-sensitive facts without date anchors decay silently |
| A10 |
Retrofitting citations |
Writing first, citing later consistently inflates confidence in unsupported claims |
| A11 |
Overweighting recency |
Newest is not most authoritative — prefer canonical primary sources |
| A12 |
Storing evidence with synthesis |
Mixed buckets cause RAG follow-on to cite derived text as original evidence |
| A13 |
Treating the open-access corpus as the field |
Paywalled prior work and unpublished negative results are silently absent; output skews toward hypotheses that only look promising |
Decision Matrix: Native Agent vs Custom Pipeline
See full decision guide → references/native-deep-research-agents.md
| Criterion |
Native agent |
Custom pipeline |
| Setup time |
Minutes |
Hours to days |
| Auditability |
Limited (opaque search path) |
Full (ledger + traces) |
| Repeatability |
Low (non-deterministic) |
High (seeded, logged) |
| Source control |
None |
Full |
| Freshness control |
Model-dependent |
Explicit per query |
| Multi-step planning |
Implicit |
Explicit (P2) |
| Verifier isolation |
Not available |
P4 |
| Cost predictability |
Low |
High |
| Best for |
Ad-hoc exploration, open questions, competitive overview |
Repeatable workflows, regulated outputs, source-auditable deliverables |
Decision rule: choose a native agent when speed matters and provenance does not. Choose a custom pipeline when the output will be cited, published, used in a product, or needs to be reproduced.
Native agents comparison
Full table with API surfaces and citation fidelity → references/native-deep-research-agents.md
| Agent |
API model ID / tool |
Strengths |
Limitations |
| ChatGPT Deep Research (OpenAI) |
o3-deep-research (depth) / o4-mini-deep-research (speed/cost) — Responses API |
Strongest source diversity, long synthesis, strong multi-step reasoning (RL-post-trained) |
Sunset scheduled 2026-07-23 (recommended replacement: gpt-5.5-pro) — verify at developers.openai.com/api/docs/deprecations before starting new work, the feature guide page does not surface this. Slow (5–30 min), opaque search log, citations can go stale |
| Gemini Deep Research (Google) |
deep-research-preview-04-2026 / deep-research-max-preview-04-2026 (Interactions API) |
Explicit plan review before execution, Google index, async + streaming, MCP support |
Preview model IDs include date suffix — will change; thinner synthesis depth than ChatGPT DR |
| Perplexity Deep Research |
sonar-deep-research (Chat Completions API) |
Fastest (2–5 min), paragraph-level inline citations, real-time index |
Citation laundering risk on SEO topics; shorter synthesis; model averages contradictions rather than flagging; multi-component pricing makes budgeting error-prone |
| Claude with web search |
web_search_20260318 (current: adds response-inclusion control) / web_search_20260209 (dynamic filtering, GA) / web_search_20250305 (basic) |
Full tool-call audit trail, structured output, P2/P4 composable, memory-tool-backed persistence |
No dedicated DR mode; needs P2 loop for DR-equivalent depth |
| Grok DeepSearch (xAI) |
web_search / x_search server-side tools (documented at docs.x.ai) |
Only agent with X/Twitter as a primary indexed source; now has a documented, API-invocable tool surface |
No dedicated report-generating deep-research model or endpoint comparable to the other four; citation auditability less mature |
Architecture, Workflow, and Contracts
Full depth — planner / searcher / verifier / synthesizer subagent split, canonical contract shapes (ResearchPlan, SourceLedger, EvidenceTier, VerifierReport, SynthesisArtifact), stop-criterion design, freshness window classes — in references/agentic-research-loop-architecture.md.
Default Workflow
- Frame the research question narrowly enough to be falsifiable.
- Emit a structured research plan — see assets/templates/research-plan.template.md.
- Execute the planner/searcher loop; record every source in the ledger before any synthesis.
- Run the verifier subagent (must not share context with the searcher).
- Separate evidence tiers: primary, secondary, and model-working-notes.
- Check freshness windows; date-stamp all volatile claims.
- Synthesize into the requested artifact; every factual claim carries a citation back-pointer.
- Archive the ledger separately — see assets/templates/source-ledger.template.md.
Hypothesis Generation: The Co-Scientist Role Taxonomy
The planner / searcher / verifier / synthesizer split above answers a question. A different split is needed when the task is to generate and rank candidate ideas — research directions, product hypotheses, competing explanations for an observed effect — where there is no single retrievable answer to converge on.
Google Research's AI co-scientist (2025) decomposes that job into six specialized roles under a supervisor agent, each a separate context:
| Role |
Job |
Why it is separate |
| Generation |
Produce initial hypotheses from literature exploration plus simulated scientific debate between personas |
Generation optimizes for coverage; mixing in evaluation collapses the candidate space early (see A1) |
| Reflection |
Peer-review each hypothesis for correctness, novelty, and quality |
A reviewer that also generated the idea cannot reject it — the same independence rule as P4 |
| Ranking |
Run an Elo-based tournament: hypotheses compete pairwise, ratings order the field |
Pairwise comparison is more reliable than absolute scoring, and the rating is a live signal you can watch converge |
| Evolution |
Refine the top-ranked hypotheses — simplify the concept, synthesize two ideas, or push into unconventional reasoning |
Refinement is a distinct move from generation; it operates on winners, not on a blank page |
| Proximity |
Compute a proximity graph clustering similar ideas |
Surfaces where the field is crowded vs unexplored; prevents the tournament from ranking ten paraphrases of one idea |
| Meta-review |
Synthesize patterns across all reviews and debates, feed them back as guidance |
The only role that improves the system rather than a hypothesis |
The loop is generate → debate → evolve, run iteratively: new hypotheses enter the tournament, top-ranked ones are evolved and re-enter, meta-review tightens the criteria each pass. Termination is the same discipline as P8 — stop on rating saturation (no new hypothesis breaks into the top tier for N rounds), not on a fixed round count.
How this composes with the rest of this skill. Generation and Reflection both need a ledger (P1) — a hypothesis grounded in nothing is a guess, and Reflection's novelty check requires knowing what already exists. Ranking is where evidence tiers (P6) do the work: a hypothesis supported by primary sources should outrank one supported by commentary, so feed the tier into the tournament rather than letting fluency decide. Meta-review is the natural place to catch A1 and A8 across a whole run.
Scope caution. This is a hypothesis-generation architecture. It orders candidate ideas by plausibility; it does not establish that any of them are true. The output of a tournament is a ranked queue for verification, never a finding — and see A13 for the literature bias that shapes what enters the queue at all.
Sources: Google Research, "AI co-scientist" (2025); described in Gulli, Agentic Design Patterns (Springer, 2025), Ch. 21 — Exploration and Discovery.
Known Traps
| Trap |
Resolution |
| Starting with synthesis and backfilling sources |
Freeze the ledger first; emit no synthesis until the ledger passes a minimum source count and verifier check |
| Using model outputs as sources in the next research pass |
Stamp working notes with tier: model-output; the verifier rejects them as primary evidence |
| Verifier reads the same context as researcher |
Give the verifier only the ledger JSONL, not the search session |
| Research loop never terminates |
Define the stop criterion in the plan: saturation (no novel facts in N iterations) or hard cap (M total queries) |
| Citation laundering through secondary articles |
Trace every citation to its primary document; reject chains longer than two hops |
| Mixing old and new evidence for fast-moving topics |
Apply P5 freshness-window sourcing; reject sources older than the window for volatile claims |
| Hostile SEO sources polluting vendor comparisons |
Apply P9 hostile-source detection before adding any URL to the ledger |
| S2S session-state loss on agentic research re-runs |
Persist the ledger and plan to disk before each iteration; resume from ledger state, not from conversation context |
| Trusting a vendor's feature guide as the source of truth for model availability |
A provider's "how to use X" guide and its deprecations/sunset page are maintained on different cadences and can disagree — the guide may still recommend a model days before its documented shutdown. Check the deprecations page separately before wiring a pipeline to any specific model ID |
Navigation
References
- references/patterns-catalog.md — P1–P10 named patterns
- references/anti-patterns-catalog.md — A1–A12 with detection signals
- references/native-deep-research-agents.md — Comparison table, ChatGPT DR, Gemini DR, Perplexity DR, xAI Grok DeepSearch, Claude with web search, Anthropic memory tool, URL-health recipe
- references/agentic-research-loop-architecture.md — Planner / searcher / verifier / synthesizer subagent split
- references/research-workflow.md — Staged research loop and source-plan design
- references/evidence-packaging.md — Provenance, citation, and synthesis-packaging rules
Assets / Templates
- assets/templates/research-plan.template.md — Research plan scaffold
- assets/templates/source-ledger.template.md — Source ledger scaffold
Scripts
python3 scripts/citation_verifier.py --input claims.jsonl — Checks JSONL of {claim, source_url, supporting_quote} and reports unsupported claims
Data
- data/sources.json — Curated source repos and tool references
Related Skills
Fact-Checking
- Known bugs, regressions, framework/compiler/runtime footguns, and version-specific crash or workaround guidance must be verified against current primary web sources before being treated as current fact.
- Start from
data/sources.json for workflow and tool references.
- Verify native deep-research agent capabilities and output formats before recommending them for specific use cases — features change frequently.
- Check the provider's deprecations/sunset page, not just its feature guide, before committing a pipeline to a specific model ID — the two pages update on different cadences and the guide can lag behind an already-announced shutdown by days or weeks.
- If live verification is unavailable, mark native-agent capability claims as unverified.
Learnings Loop
Before applying this skill on a non-trivial task, read learnings.consolidated.md in this directory (and learnings.md if present).
After applying it, if you encountered a pattern worth remembering, a mistake worth preventing, or a domain fact that surprised you, append one dated bullet to learnings.md via agents-skills-feedback-loop/scripts/append_learning.py. Do not modify SKILL.md itself.
1---2name: ai-deep-research3description: Builds repeatable deep-research workflows for verified synthesis. Use when producing evidence-backed briefs, comparisons, dossiers, or research pipelines.4---5
6# Deep Research
7
8Use this skill to design and run repeatable research workflows that gather evidence across many sources, preserve provenance, and synthesize results into decision-ready outputs.
9
10This skill covers both **native deep-research agents** (ChatGPT Deep Research, Gemini Deep Research, Perplexity Deep Research, Claude with web search) and **custom agentic research pipelines** (planner / searcher / verifier / synthesizer split).
11
12## ASCII Flow
13
14```text
15research question
16 |
17 v
18research plan
19 scope + source targets + queries + stop criteria + freshness window
20 |
21 v
22evidence gathering
23 primary sources first + source ledger + hostile-source checks
24 |
25 v
26verification
27 isolated verifier checks claims against ledger, not researcher context
28 |
29 v
30synthesis
31 evidence-tiered answer + citations + contradictions + unknowns
32```
33
34## Quick Reference
35
36| Question | Default |
37|----------|---------|
38| When to use a native agent vs custom pipeline? | Native for ad-hoc, open-ended questions. Custom for repeatable, auditable, or multi-source workflows. |
39| What is the first artifact of any research task? | The source ledger — never the synthesis. |
40| When is a source trustworthy? | When it is a primary document with a stable URL, author attribution, and a verifiable date. |
41| What stops an unbounded research loop? | An explicit stop criterion defined before the loop starts (saturation condition or max iterations). |
42| How to handle contradictory sources? | Separate them into evidence tiers; do not resolve by averaging. |
43
44## Use This Skill When
45
46- You need to produce a sourced comparison, brief, memo, or research dossier.
47- You need to choose between a native deep-research agent and a custom pipeline.
48- You want to build a repeatable, auditable research workflow with provenance.
49- You need to detect hostile sources, citation laundering, or model-output-as-source.
50- You need to run a verifier subagent that has not seen the researcher's context.
51
52## Do Not Use This Skill For
53
54- Single-lookup current facts — use ordinary browsing or direct tools.
55- Long-lived retrieval serving — use [ai-rag](../ai-rag/SKILL.md).
56- Local note-vault packaging — use [docs-notes-retrieval](../docs-notes-retrieval/SKILL.md).
57
58## Current Stance
59
60The durable pattern is not "send a single query and summarize the top results."
61
62The default architecture (as of last_validated date above):
63
641. **Plan before searching**: emit a structured research plan with queries, source targets, and success criteria before any tool call.
652. **Ledger as contract**: the source ledger is the primary artifact — synthesis is derived from it, not the other way around.
663. **Verifier isolation**: the verifier subagent must not share context with the researcher — it reads only the ledger and checks claims independently.
674. **Evidence tier separation**: primary sources (original docs, filings, specs) are never mixed with secondary commentary or model-generated summaries in the same evidence bucket.
685. **Saturation-first synthesis**: synthesize only after the research loop has reached saturation (new queries return no novel facts), not on a time budget.
696. **Freshness windows are explicit**: every claim in the final output carries a date anchor and a freshness class (stable / volatile / unknown).
707. **Hostile source detection is mandatory for public-web research**: adversarial SEO, AI-generated content farms, and citation-laundering chains are real threats.
71
72## Patterns
73
74Full catalog → [references/patterns-catalog.md](references/patterns-catalog.md)
75
76| ID | Name | When to reach for it |
77|----|------|----------------------|
78| P1 | Source-ledger-as-contract | Always — every research task |
79| P2 | Plan-then-execute loop | Multi-query or multi-source tasks |
80| P3 | Multi-agent research swarm | Large corpora with parallel query paths |
81| P4 | Verifier subagent | Any claim that will reach a user or decision |
82| P5 | Freshness-window sourcing | Time-sensitive domains (AI, market, regulatory) |
83| P6 | Evidence-tier separation | When primary and secondary sources both exist |
84| P7 | Citation back-pointer | Every synthesis sentence that makes a factual claim |
85| P8 | Stop-criterion-as-eval | To prevent unbounded research loops |
86| P9 | Hostile-source detection | Public-web research, vendor comparisons |
87| P10 | Synthesis-after-saturation | All multi-pass research tasks |
88
89## Anti-Patterns
90
91Full catalog → [references/anti-patterns-catalog.md](references/anti-patterns-catalog.md)
92
93| ID | Name | Why it matters |
94|----|------|----------------|
95| A1 | Pre-synthesizing the answer | Frames all subsequent evidence gathering; produces confirmation bias |
96| A2 | Single-pass search | Misses contradictions, minority sources, and laterally relevant evidence |
97| A3 | Mixing evidence tiers | Primary docs and blog summaries in the same bucket degrade citation quality |
98| A4 | Model-output-as-source | LLM summaries carry hallucination risk; they are working notes, not source truth |
99| A5 | Unbounded research loop | Without a stop criterion, cost and time grow unboundedly with no quality signal |
100| A6 | Citation laundering | Citing a secondary article that in turn cites the primary — provenance chain breaks |
101| A7 | Verifier-with-same-context | Verifier that read the same sources as researcher adds no independence |
102| A8 | Vague consensus language | "Multiple sources say" without naming them is unfalsifiable |
103| A9 | Date-free volatile claims | Time-sensitive facts without date anchors decay silently |
104| A10 | Retrofitting citations | Writing first, citing later consistently inflates confidence in unsupported claims |
105| A11 | Overweighting recency | Newest is not most authoritative — prefer canonical primary sources |
106| A12 | Storing evidence with synthesis | Mixed buckets cause RAG follow-on to cite derived text as original evidence |
107| A13 | Treating the open-access corpus as the field | Paywalled prior work and unpublished negative results are silently absent; output skews toward hypotheses that only *look* promising |
108
109## Decision Matrix: Native Agent vs Custom Pipeline
110
111See full decision guide → [references/native-deep-research-agents.md](references/native-deep-research-agents.md)
112
113| Criterion | Native agent | Custom pipeline |
114|-----------|-------------|-----------------|
115| Setup time | Minutes | Hours to days |
116| Auditability | Limited (opaque search path) | Full (ledger + traces) |
117| Repeatability | Low (non-deterministic) | High (seeded, logged) |
118| Source control | None | Full |
119| Freshness control | Model-dependent | Explicit per query |
120| Multi-step planning | Implicit | Explicit (P2) |
121| Verifier isolation | Not available | P4 |
122| Cost predictability | Low | High |
123| Best for | Ad-hoc exploration, open questions, competitive overview | Repeatable workflows, regulated outputs, source-auditable deliverables |
124
125**Decision rule:** choose a native agent when speed matters and provenance does not. Choose a custom pipeline when the output will be cited, published, used in a product, or needs to be reproduced.
126
127### Native agents comparison
128
129Full table with API surfaces and citation fidelity → [references/native-deep-research-agents.md](references/native-deep-research-agents.md)
130
131| Agent | API model ID / tool | Strengths | Limitations |
132|-------|---------------------|-----------|-------------|
133| ChatGPT Deep Research (OpenAI) | `o3-deep-research` (depth) / `o4-mini-deep-research` (speed/cost) — Responses API | Strongest source diversity, long synthesis, strong multi-step reasoning (RL-post-trained) | **Sunset scheduled 2026-07-23** (recommended replacement: `gpt-5.5-pro`) — verify at `developers.openai.com/api/docs/deprecations` before starting new work, the feature guide page does not surface this. Slow (5–30 min), opaque search log, citations can go stale |
134| Gemini Deep Research (Google) | `deep-research-preview-04-2026` / `deep-research-max-preview-04-2026` (Interactions API) | Explicit plan review before execution, Google index, async + streaming, MCP support | Preview model IDs include date suffix — will change; thinner synthesis depth than ChatGPT DR |
135| Perplexity Deep Research | `sonar-deep-research` (Chat Completions API) | Fastest (2–5 min), paragraph-level inline citations, real-time index | Citation laundering risk on SEO topics; shorter synthesis; model averages contradictions rather than flagging; multi-component pricing makes budgeting error-prone |
136| Claude with web search | `web_search_20260318` (current: adds response-inclusion control) / `web_search_20260209` (dynamic filtering, GA) / `web_search_20250305` (basic) | Full tool-call audit trail, structured output, P2/P4 composable, memory-tool-backed persistence | No dedicated DR mode; needs P2 loop for DR-equivalent depth |
137| Grok DeepSearch (xAI) | `web_search` / `x_search` server-side tools (documented at `docs.x.ai`) | Only agent with X/Twitter as a primary indexed source; now has a documented, API-invocable tool surface | No dedicated report-generating deep-research model or endpoint comparable to the other four; citation auditability less mature |
138
139## Architecture, Workflow, and Contracts
140
141Full depth — planner / searcher / verifier / synthesizer subagent split, canonical contract shapes (`ResearchPlan`, `SourceLedger`, `EvidenceTier`, `VerifierReport`, `SynthesisArtifact`), stop-criterion design, freshness window classes — in [references/agentic-research-loop-architecture.md](references/agentic-research-loop-architecture.md).
142
143### Default Workflow
144
1451. Frame the research question narrowly enough to be falsifiable.
1462. Emit a structured research plan — see [assets/templates/research-plan.template.md](assets/templates/research-plan.template.md).
1473. Execute the planner/searcher loop; record every source in the ledger before any synthesis.
1484. Run the verifier subagent (must not share context with the searcher).
1495. Separate evidence tiers: primary, secondary, and model-working-notes.
1506. Check freshness windows; date-stamp all volatile claims.
1517. Synthesize into the requested artifact; every factual claim carries a citation back-pointer.
1528. Archive the ledger separately — see [assets/templates/source-ledger.template.md](assets/templates/source-ledger.template.md).
153
154## Hypothesis Generation: The Co-Scientist Role Taxonomy
155
156The planner / searcher / verifier / synthesizer split above answers a **question**. A different split is needed when the task is to *generate and rank candidate ideas* — research directions, product hypotheses, competing explanations for an observed effect — where there is no single retrievable answer to converge on.
157
158Google Research's AI co-scientist (2025) decomposes that job into six specialized roles under a supervisor agent, each a separate context:
159
160| Role | Job | Why it is separate |
161|------|-----|--------------------|
162| **Generation** | Produce initial hypotheses from literature exploration plus simulated scientific debate between personas | Generation optimizes for coverage; mixing in evaluation collapses the candidate space early (see A1) |
163| **Reflection** | Peer-review each hypothesis for correctness, novelty, and quality | A reviewer that also generated the idea cannot reject it — the same independence rule as P4 |
164| **Ranking** | Run an **Elo-based tournament**: hypotheses compete pairwise, ratings order the field | Pairwise comparison is more reliable than absolute scoring, and the rating is a live signal you can watch converge |
165| **Evolution** | Refine the top-ranked hypotheses — simplify the concept, synthesize two ideas, or push into unconventional reasoning | Refinement is a distinct move from generation; it operates on winners, not on a blank page |
166| **Proximity** | Compute a proximity graph clustering similar ideas | Surfaces where the field is crowded vs unexplored; prevents the tournament from ranking ten paraphrases of one idea |
167| **Meta-review** | Synthesize patterns across all reviews and debates, feed them back as guidance | The only role that improves the *system* rather than a hypothesis |
168
169**The loop is generate → debate → evolve**, run iteratively: new hypotheses enter the tournament, top-ranked ones are evolved and re-enter, meta-review tightens the criteria each pass. Termination is the same discipline as P8 — stop on rating saturation (no new hypothesis breaks into the top tier for N rounds), not on a fixed round count.
170
171**How this composes with the rest of this skill.** Generation and Reflection both need a ledger (P1) — a hypothesis grounded in nothing is a guess, and Reflection's novelty check requires knowing what already exists. Ranking is where evidence tiers (P6) do the work: a hypothesis supported by primary sources should outrank one supported by commentary, so feed the tier into the tournament rather than letting fluency decide. Meta-review is the natural place to catch A1 and A8 across a whole run.
172
173**Scope caution.** This is a hypothesis-*generation* architecture. It orders candidate ideas by plausibility; it does not establish that any of them are true. The output of a tournament is a ranked queue for verification, never a finding — and see A13 for the literature bias that shapes what enters the queue at all.
174
175Sources: Google Research, "AI co-scientist" (2025); described in Gulli, *Agentic Design Patterns* (Springer, 2025), Ch. 21 — Exploration and Discovery.
176
177## Known Traps
178
179| Trap | Resolution |
180|------|-----------|
181| Starting with synthesis and backfilling sources | Freeze the ledger first; emit no synthesis until the ledger passes a minimum source count and verifier check |
182| Using model outputs as sources in the next research pass | Stamp working notes with `tier: model-output`; the verifier rejects them as primary evidence |
183| Verifier reads the same context as researcher | Give the verifier only the ledger JSONL, not the search session |
184| Research loop never terminates | Define the stop criterion in the plan: saturation (no novel facts in N iterations) or hard cap (M total queries) |
185| Citation laundering through secondary articles | Trace every citation to its primary document; reject chains longer than two hops |
186| Mixing old and new evidence for fast-moving topics | Apply P5 freshness-window sourcing; reject sources older than the window for volatile claims |
187| Hostile SEO sources polluting vendor comparisons | Apply P9 hostile-source detection before adding any URL to the ledger |
188| S2S session-state loss on agentic research re-runs | Persist the ledger and plan to disk before each iteration; resume from ledger state, not from conversation context |
189| Trusting a vendor's feature guide as the source of truth for model availability | A provider's "how to use X" guide and its deprecations/sunset page are maintained on different cadences and can disagree — the guide may still recommend a model days before its documented shutdown. Check the deprecations page separately before wiring a pipeline to any specific model ID |
190
191## Navigation
192
193### References
194
195- [references/patterns-catalog.md](references/patterns-catalog.md) — P1–P10 named patterns
196- [references/anti-patterns-catalog.md](references/anti-patterns-catalog.md) — A1–A12 with detection signals
197- [references/native-deep-research-agents.md](references/native-deep-research-agents.md) — Comparison table, ChatGPT DR, Gemini DR, Perplexity DR, xAI Grok DeepSearch, Claude with web search, Anthropic memory tool, URL-health recipe
198- [references/agentic-research-loop-architecture.md](references/agentic-research-loop-architecture.md) — Planner / searcher / verifier / synthesizer subagent split
199- [references/research-workflow.md](references/research-workflow.md) — Staged research loop and source-plan design
200- [references/evidence-packaging.md](references/evidence-packaging.md) — Provenance, citation, and synthesis-packaging rules
201
202### Assets / Templates
203
204- [assets/templates/research-plan.template.md](assets/templates/research-plan.template.md) — Research plan scaffold
205- [assets/templates/source-ledger.template.md](assets/templates/source-ledger.template.md) — Source ledger scaffold
206
207### Scripts
208
209- `python3 scripts/citation_verifier.py --input claims.jsonl` — Checks JSONL of {claim, source_url, supporting_quote} and reports unsupported claims
210
211### Data
212
213- [data/sources.json](data/sources.json) — Curated source repos and tool references
214
215## Related Skills
216
217- [../ai-rag/SKILL.md](../ai-rag/SKILL.md) — Retrieval architecture and serving indexes
218- [../docs-notes-retrieval/SKILL.md](../docs-notes-retrieval/SKILL.md) — Local note-vault packaging
219- [../agents-mcp/SKILL.md](../agents-mcp/SKILL.md) — MCP search, docs, and memory tool surfaces
220- [../qa-agent-testing/SKILL.md](../qa-agent-testing/SKILL.md) — Eval harnesses for productized research workflows
221- [../ai-agents/SKILL.md](../ai-agents/SKILL.md) — Agent architecture (planner/searcher/verifier split)
222
223## Fact-Checking
224
225- Known bugs, regressions, framework/compiler/runtime footguns, and version-specific crash or workaround guidance must be verified against current primary web sources before being treated as current fact.
226- Start from `data/sources.json` for workflow and tool references.
227- Verify native deep-research agent capabilities and output formats before recommending them for specific use cases — features change frequently.
228- Check the provider's **deprecations/sunset page**, not just its feature guide, before committing a pipeline to a specific model ID — the two pages update on different cadences and the guide can lag behind an already-announced shutdown by days or weeks.
229- If live verification is unavailable, mark native-agent capability claims as unverified.
230
231## Learnings Loop
232
233Before applying this skill on a non-trivial task, read `learnings.consolidated.md` in this directory (and `learnings.md` if present).
234
235After applying it, if you encountered a pattern worth remembering, a mistake worth preventing, or a domain fact that surprised you, append one dated bullet to `learnings.md` via `agents-skills-feedback-loop/scripts/append_learning.py`. Do not modify `SKILL.md` itself.
gabrielmoreira