CLI Tools Reference

CLI Tools Reference

Local Deep Research includes command-line tools for benchmarking and rate limit management.

Table of Contents

• Benchmarking CLI
• Rate Limiting CLI

Benchmarking CLI

Run benchmarks to evaluate search quality and compare configurations.

Basic Usage

python -m local_deep_research.benchmarks.cli <command> [options]

Commands

simpleqa - Run SimpleQA Benchmark

Tests factual question answering accuracy.

python -m local_deep_research.benchmarks.cli simpleqa [options]

Options:

Example:

# Run 50 examples with Ollama
python -m local_deep_research.benchmarks.cli simpleqa \
  --examples 50 \
  --search-provider ollama \
  --search-model llama3.2

browsecomp - Run BrowseComp Benchmark

Tests complex reasoning and multi-step research.

python -m local_deep_research.benchmarks.cli browsecomp [options]

Same options as simpleqa.

Example:

# Run BrowseComp with focused-iteration strategy
python -m local_deep_research.benchmarks.cli browsecomp \
  --examples 20 \
  --search-strategy iterdrag \
  --iterations 5

compare - Compare Configurations

Compare multiple search configurations on the same dataset.

python -m local_deep_research.benchmarks.cli compare [options]

Options:

Example:

# Compare configurations
python -m local_deep_research.benchmarks.cli compare \
  --dataset simpleqa \
  --examples 30

list - List Available Benchmarks

python -m local_deep_research.benchmarks.cli list

Shows available benchmark datasets and their descriptions.

Rate Limiting CLI

Monitor and manage the adaptive rate limiting system.

Basic Usage

python -m local_deep_research.web_search_engines.rate_limiting.cli <command> [options]

Commands

status - Show Rate Limit Statistics

View current rate limit data for search engines.

# All engines
python -m local_deep_research.web_search_engines.rate_limiting.cli status

# Specific engine
python -m local_deep_research.web_search_engines.rate_limiting.cli status --engine DuckDuckGoSearchEngine

Output columns:

Example output:

Rate Limit Statistics:
--------------------------------------------------------------------------------
Engine               Base Wait    Range                Success    Attempts   Updated
--------------------------------------------------------------------------------
DuckDuckGoSearchEngine 2.50        1.0s - 5.0s         95.2%      150        12-26 14:30
ArXivSearchEngine      0.50        0.5s - 1.0s         99.8%      85         12-26 12:15

reset - Reset Engine Rate Limits

Clear learned rate limit data for an engine.

python -m local_deep_research.web_search_engines.rate_limiting.cli reset --engine <engine_name>

Example:

# Reset DuckDuckGo rate limits
python -m local_deep_research.web_search_engines.rate_limiting.cli reset --engine DuckDuckGoSearchEngine

Use this when:

• Rate limits are too conservative
• After API changes
• When switching environments

export - Export Rate Limit Data

Export rate limit statistics in various formats.

# Table format (default)
python -m local_deep_research.web_search_engines.rate_limiting.cli export

# CSV format
python -m local_deep_research.web_search_engines.rate_limiting.cli export --format csv

# JSON format
python -m local_deep_research.web_search_engines.rate_limiting.cli export --format json

Formats:

Example CSV output:

engine_type,base_wait_seconds,min_wait_seconds,max_wait_seconds,last_updated,total_attempts,success_rate
DuckDuckGoSearchEngine,2.5,1.0,5.0,1703612400,150,0.952

cleanup - Remove Old Data

Clean up rate limit data older than a specified number of days.

python -m local_deep_research.web_search_engines.rate_limiting.cli cleanup --days <days>

Example:

# Remove data older than 30 days
python -m local_deep_research.web_search_engines.rate_limiting.cli cleanup --days 30

# Remove data older than 7 days
python -m local_deep_research.web_search_engines.rate_limiting.cli cleanup --days 7

Common Engine Names

When using the rate limiting CLI, use these engine class names:

Troubleshooting

Benchmark Not Starting

• Verify LLM provider is configured
• Check search engine is available
• Ensure sufficient disk space for results

Rate Limit Data Missing

• Run some searches first to generate data
• Check database file exists
• Try status without --engine flag

Export Permission Error

• Check write permissions on output directory
• Use a different output directory

See Also

• Architecture Overview - System architecture
• Troubleshooting - Common issues
• BENCHMARKING.md - Detailed benchmark documentation