1999AZZAR Search Cluster
A multi-provider search aggregator designed for high-availability and security.
Setup
- Install dependencies:
python3 -m pip install -r requirements.txt. - Optionally configure provider keys in
.envor the process environment.
Security Posture
- Subprocess Isolation: Query inputs are passed as arguments to stealth_fetch.py.
- Strict TLS: Mandatory SSL verification on all providers.
- Sanitization: Integrated native internal scrubber (Path Neutral).
Requirements and Environment
Declare these variables in your environment or vault:
| Variable | Requirement | Description |
|---|---|---|
| GOOGLE_API_KEY | Optional | API Key for Google Custom Search. |
| GOOGLE_CSE_ID | Optional | Search Engine ID for Google CSE. |
| NEWSAPI_KEY | Optional | NewsAPI key for current news. |
| SEARCH_USER_AGENT | Optional | Custom User-Agent string. |
Providers
- google: Official Google Custom Search.
- wiki: Wikipedia OpenSearch API.
- reddit: Reddit JSON search API.
- news: NewsAPI structured article search.
- ddg: DuckDuckGo HTML search without an API key.
Included Scripts
- scripts/search-cluster.py: Main entry point.
- scripts/stealth_fetch.py: Optional Scrapling-based DuckDuckGo fetcher.
Workflow
- Execute:
python3 scripts/search-cluster.py all "<query>". - Parse stdout as a JSON array. Provider diagnostics are written to stderr.
- Treat an empty array as no available results, not malformed output.
Each result contains source, title, link, and snippet. Calls use bounded timeouts, retry transient failures, deduplicate URLs, and continue when individual providers fail.