Multi Search Engine

Multi Search Engine

Integration of 16 search engines for web crawling without API keys.

Workflow

1. Preparation: AI Agent initializes an empty in-memory cookie store. Cookies are only acquired dynamically during search operations when access is denied

2. Language Evaluation: Detect the language attribute of the search query. If the query is in Chinese, use Domestic search engines (Baidu, Bing CN, Bing INT, 360, Sogou, WeChat, Shenma). If the query is non-Chinese, use International search engines (Google, Google HK, DuckDuckGo, Yahoo, Startpage, Brave, Ecosia, Qwant, WolframAlpha). Select engines based on query relevance and availability.

3. Controlled Search: Use web_fetch to execute search requests with rate limiting:

   • Add 1-2 second delay between requests to respect server load
   • Batch requests in groups of 3-4 engines with sequential execution between batches
   • Include standard browser headers to identify as legitimate user agent
   • If access is denied (403/429), fetch engine homepage to obtain fresh session cookies

4. Cookie Management:

   • Cookies are stored ONLY in memory during runtime
   • Cookies are acquired on-demand when search requests fail
   • No cookies are read from or written to config.json or any file
   • Cookies are cleared after search session completes
   • Only session cookies from search engine domains are captured

5. Retry Mechanism: If a search fails due to cookie/session issues, retry once with freshly acquired cookies after a 2-second delay

6. Result Aggregation: Consolidate successful results from search engines, organize and summarize them to output a core search report

Search Engines

Domestic (7)

• Baidu: https://www.baidu.com/s?wd={keyword}
• Bing CN: https://cn.bing.com/search?q={keyword}&ensearch=0
• Bing INT: https://cn.bing.com/search?q={keyword}&ensearch=1
• 360: https://www.so.com/s?q={keyword}
• Sogou: https://sogou.com/web?query={keyword}
• WeChat: https://wx.sogou.com/weixin?type=2&query={keyword}
• Shenma: https://m.sm.cn/s?q={keyword}

International (9)

• Google: https://www.google.com/search?q={keyword}
• Google HK: https://www.google.com.hk/search?q={keyword}
• DuckDuckGo: https://duckduckgo.com/html/?q={keyword}
• Yahoo: https://search.yahoo.com/search?p={keyword}
• Startpage: https://www.startpage.com/sp/search?query={keyword}
• Brave: https://search.brave.com/search?q={keyword}
• Ecosia: https://www.ecosia.org/search?q={keyword}
• Qwant: https://www.qwant.com/?q={keyword}
• WolframAlpha: https://www.wolframalpha.com/input?i={keyword}

Quick Examples

// Basic search
web_fetch({"url": "https://www.google.com/search?q=python+tutorial"})

// Site-specific
web_fetch({"url": "https://www.google.com/search?q=site:github.com+react"})

// File type
web_fetch({"url": "https://www.google.com/search?q=machine+learning+filetype:pdf"})

// Time filter (past week)
web_fetch({"url": "https://www.google.com/search?q=ai+news&tbs=qdr:w"})

// Privacy search
web_fetch({"url": "https://duckduckgo.com/html/?q=privacy+tools"})

// DuckDuckGo Bangs
web_fetch({"url": "https://duckduckgo.com/html/?q=!gh+tensorflow"})

// Knowledge calculation
web_fetch({"url": "https://www.wolframalpha.com/input?i=100+USD+to+CNY"})

Advanced Operators

Time Filters

Privacy Engines

• DuckDuckGo: No tracking
• Startpage: Google results + privacy
• Brave: Independent index
• Qwant: EU GDPR compliant

Bangs Shortcuts (DuckDuckGo)

WolframAlpha Queries

• Math: integrate x^2 dx
• Conversion: 100 USD to CNY
• Stocks: AAPL stock
• Weather: weather in Beijing

Documentation

• references/advanced-search.md - Domestic search guide
• references/international-search.md - International search guide
• CHANGELOG.md - Version history

License

MIT

Security & Privacy Notice

Cookie Handling

• Purpose: Cookies are used ONLY to maintain search session state when access is denied (403/429 errors)
• Storage: Cookies are kept STRICTLY in memory during runtime - NEVER persisted to disk or config files
• Acquisition: Cookies are acquired on-demand from search engine homepages only when search requests fail
• Scope: Only session cookies from the specific search engine domain are captured
• Lifecycle: Cookies are cleared immediately after the search session completes
• No Pre-configuration: No cookies are loaded from config.json or any external file at startup
• No API Keys: This tool uses standard web search URLs, no authentication required

Crawling Ethics

• Rate Limiting: Implement reasonable delays between requests (recommend 1-2 seconds)
• Respect robots.txt: Honor search engine crawling policies
• Terms of Service: Users are responsible for complying with search engine ToS
• Purpose: Designed for legitimate search aggregation, not mass data scraping

Data Handling

• No Personal Data: Tool does not collect or transmit user personal information
• Local Execution: All operations run locally, no external data transmission
• Session Isolation: Cookies are session-specific and cleared after use

DSH 运行说明（本副本补充）

• 执行通道：本技能靠网页抓取跑搜索，DSH 下用 web_fetch（拉取页面）、web_search（补充/兜底）。上游写的 Browser / fetch 类工具名按此对应。
• 限速：批次之间保持 1-2 秒间隔（pwsh：Start-Sleep -Seconds 2），每批 3-4 个引擎串行，不要并发猛拉。
• 被挡时（403/429）：先取引擎首页拿新 cookie 再重试，与上游第 4 步一致。
• 资源位置：references/advanced-search.md（国内引擎指南）、references/international-search.md（国际引擎指南）相对本技能目录解析；config.json 存引擎清单；CHANGELOG.md 是版本历史。
• 默认路由提示：DSH 原生已有 web_search（Tavily）。简单查询直接用原生工具；本技能的价值在于多引擎交叉取证（尤其国内引擎 + 隐私引擎 + WolframAlpha 知识查询），或原生工具结果不足时扩大覆盖。