Free Search Aggregator

Free Search Aggregator

Reliable, provider-diverse web search for OpenClaw with high uptime + low operator overhead.

Why use this skill

• 12 search providers, 6 requiring no API key at all
• Automatic failover: if one provider fails, the next is tried instantly
• Quota-aware: tracks daily usage, warns at 80%, skips exhausted providers
• Task search mode for multi-angle research queries
• Built-in storage lifecycle (cache / index / report), no workspace clutter

Provider Overview

Total daily quota (all keys configured): 8400+ requests/day

Credential model (important)

• No mandatory API key — DuckDuckGo + Bing RSS + Mojeek + Wikipedia work out of the box.
• API-key providers fail gracefully if key is missing (AuthError → skip, no quota consumed, no latency):
  • BRAVE_API_KEY
  • EXA_API_KEY
  • TAVILY_API_KEY
  • SERPER_API_KEY
  • SEARCHAPI_API_KEY
  • GOOGLE_API_KEY + GOOGLE_CX
  • BAIDU_API_KEY
  • MOJEEK_API_KEY (optional — without it uses HTML scraping)

Core capabilities

1. Search failover

Default provider order:

brave → exa → tavily → duckduckgo → bing_html → mojeek → serper → searchapi → google_cse → baidu → wikipedia

First successful non-empty result returns immediately.

2. Task-level multi-query search

• Expands one goal into multiple targeted queries
• Aggregates + deduplicates results
• Prefix presets:
  • default: workers=1
  • @dual ... → workers=2
  • @deep ... → workers=3 + deeper query coverage

3. Quota intelligence

• Per-provider daily tracking
• Real quota retrieval where supported (Tavily, SearchAPI, Brave via probe)
• Auto concurrency reduction at 80% quota saturation

4. Managed persistence

• memory/search-cache/YYYY-MM-DD/*.json
• memory/search-index/search-index.jsonl
• memory/search-reports/YYYY-MM-DD/*.md

Quick commands

# Normal search
scripts/search "latest AI agent frameworks 2026" --max-results 5

# Task search (multi-query, parallel)
scripts/search task "@dual Compare Claude vs GPT-4 for code generation" --max-results 5

# Deep research mode
scripts/search task "@deep autonomous vehicle safety 2026" --max-results 8 --max-queries 10

# Quota status
scripts/status

# Real quota from provider APIs
scripts/remaining --real

# Cleanup cache
python3 -m free_search gc --cache-days 14

Provider setup guides

Bing RSS (bing_html) — No key needed

Uses Bing's built-in RSS endpoint (format=rss) — bypasses bot detection. Works out of the box.

Mojeek — No key needed (API key optional)

Out-of-the-box HTML scraping. For higher quotas/stability:

1. Register at https://www.mojeek.com/services/search/api/
2. Set MOJEEK_API_KEY → automatically switches to JSON API mode

Wikipedia — No key needed

Multilingual support — change lang in providers.yaml:

wikipedia:
  lang: it   # en | zh | it | de | fr | ja ...

Exa.ai — API key required

1. Register at https://exa.ai/
2. Set EXA_API_KEY
3. Free tier: 1000 searches/month (~33/day)

Google Custom Search — API key + CX required

1. Get API key: https://developers.google.com/custom-search/v1/introduction
2. Create search engine: https://programmablesearchengine.google.com/
3. Set GOOGLE_API_KEY and GOOGLE_CX
4. Free tier: 100 queries/day

Baidu Qianfan — API key required

1. Register at https://cloud.baidu.com/
2. Set BAIDU_API_KEY
3. Best for Chinese-language content

SearXNG — Self-hosted instance required

Public instances rate-limit server-to-server requests. Use your own:

docker run -d -p 8080:8080 searxng/searxng

Then in providers.yaml:

searxng:
  endpoint: http://localhost:8080
  enabled: true

Post-install self-check

# 1) Confirm provider load
scripts/status --compact

# 2) Smoke test (uses duckduckgo/bing/mojeek out of the box)
scripts/search "openclaw" --max-results 3 --compact

# 3) Verify storage paths
ls -la /home/openclaw/.openclaw/workspace/memory/search-cache/ | tail -n 5

# 4) Check real quota (optional)
scripts/remaining --real --compact

Output contract (stable)

• Search: query, provider, results[], meta.attempted, meta.quota
• Task search: task, queries[], grouped_results[], merged_results[], meta
• Quota: date, providers[], totals; with --real: real_quota.providers[]

Operator notes

• Default mode: workers=1 — conservative for cost control
• Use @dual / @deep only for research tasks
• SearXNG and YaCy are enabled: false by default (self-hosted only)
• MOJEEK_API_KEY is optional — provider gracefully falls back to HTML scraping