adu2021 Capability Gap
Existing deep research systems (like those using live web APIs) face three critical limitations:
- Cost at scale: $5,760+ for comprehensive search synthesis limits research automation
- Reproducibility: Dynamic web content changes make trajectories non-deterministic and difficult to audit
- Analysis opacity: Live API calls obscure failure modes—difficult to diagnose why reasoning chains fail
OpenResearcher eliminates these constraints through offline infrastructure.
Core Abstractions
Three explicit browser primitives model human research behavior while enabling systematic analysis:
1. Search
Returns top-K ranked results with snippets from corpus.
- Executed against merged document pool (10K gold + 15M FineWeb)
- Deterministic ranking via local search engine
- Enables corpus-level evidence discovery
2. Open
Fetches full document content from URLs in corpus.
- Returns complete text for selected documents
- Supports multi-document integration
- Enables deep context understanding
3. Find
Locates exact string matches within opened documents.
- Precise evidence localization
- Supports substring search and regex
- Enables verification of claims
This progression from broad retrieval → content fetching → precise localization mirrors human research iterating from literature survey to evidence collection.
Design Decisions
Two-Phase Synthesis Pipeline:
Phase 1: Answer-Guided Corpus Bootstrapping
- Initial query → Serper API gathers 10K gold documents
- Merged with 15M FineWeb base corpus
- One-time cost to construct domain-specific corpus
Phase 2: Offline Trajectory Synthesis
- Teacher model (GPT-OSS-120B) generates 97K+ trajectories
- No further API calls required
- Deterministic, reproducible reasoning chains
This separation decouples expensive retrieval from synthesis, making reasoning scalable once corpus is established.
Integration Patterns
API Layer Design:
┌─────────────────────────────┐
│ Reasoning Agent │
│ (queries research tasks) │
└──────────────┬──────────────┘
│
┌──────────────▼──────────────┐
│ Three-Tool Abstraction │
│ Search / Open / Find │
└──────────────┬──────────────┘
│
┌──────────────▼──────────────┐
│ Local Search Engine │
│ (Merged corpus: 10K+15M) │
└─────────────────────────────┘
Tool Sequencing Pattern:
- Agent calls
Search(query)→ receives ranked candidates - Agent calls
Open(url)→ retrieves full document - Agent calls
Find(text, document)→ locates specific evidence - Agent integrates evidence into reasoning step
- Repeat until answer is complete or confidence sufficient
This is directly executable without external APIs, enabling both cost reduction and reproducibility.
Performance & Cost Analysis
Cost Comparison:
- Live API approach (Serper): $5,760 for comprehensive search synthesis
- OpenResearcher offline: $0 after corpus bootstrapping
- Cost reduction: 100% (amortized per-query cost approaches zero)
Reproducibility:
- Live APIs: trajectories change if web content changes
- OpenResearcher: identical results across time
- Enable corpus versioning for comparative analysis
Analysis Capability:
- Systematic gold-document hit tracking
- Identify failure modes (missed documents, insufficient evidence)
- Debug reasoning chains with perfect record of tool calls
When to Use
Apply OpenResearcher when:
- Reproducibility is critical (academic research, verification)
- Cost is a constraint (scaling reasoning across many queries)
- Offline operation is required (no internet access)
- Reasoning chains must be auditable
Less suitable for:
- Real-time current-events reasoning (corpus is static)
- Domains with rapidly evolving information
- Tasks requiring hyperlink traversal beyond merged corpus