# Search Cluster

> Search and research across Google CSE, DuckDuckGo, NewsAPI, Wikipedia, and Reddit with provider fallback and normalized JSON output. Use for web research, current technical information, news discovery, source collection, and resilient multi-provider searches.

- Skill: `1999azzar/search-cluster` (Agent Skill, multi-file: 4 files)
- Install (CLI): `npx skillmds@latest add 1999azzar/search-cluster`
- Raw SKILL.md: https://api.skillmd.com/api/skills/1999azzar/search-cluster/raw
- Safety review: pending (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Product & Planning
- Author: 1999AZZAR (https://skillmd.com/u/1999azzar)
- Updated: 2026-09-10
- Page: https://skillmd.com/skills/1999azzar/search-cluster

---


# Search Cluster

A multi-provider search aggregator designed for high-availability and security.

## Setup
1. Install dependencies: `python3 -m pip install -r requirements.txt`.
2. Optionally configure provider keys in `.env` or the process environment.

## Security Posture
- Subprocess Isolation: Query inputs are passed as arguments to stealth_fetch.py.
- Strict TLS: Mandatory SSL verification on all providers.
- Sanitization: Integrated native internal scrubber (Path Neutral).

## Requirements and Environment
Declare these variables in your environment or vault:

| Variable | Requirement | Description |
|---|---|---|
| GOOGLE_API_KEY | Optional | API Key for Google Custom Search. |
| GOOGLE_CSE_ID | Optional | Search Engine ID for Google CSE. |
| NEWSAPI_KEY | Optional | NewsAPI key for current news. |
| SEARCH_USER_AGENT | Optional | Custom User-Agent string. |

## Providers
- google: Official Google Custom Search.
- wiki: Wikipedia OpenSearch API.
- reddit: Reddit JSON search API.
- news: NewsAPI structured article search.
- ddg: DuckDuckGo HTML search without an API key.

## Included Scripts
- scripts/search-cluster.py: Main entry point.
- scripts/stealth_fetch.py: Optional Scrapling-based DuckDuckGo fetcher.

## Workflow
1. Execute: `python3 scripts/search-cluster.py all "<query>"`.
2. Parse stdout as a JSON array. Provider diagnostics are written to stderr.
3. Treat an empty array as no available results, not malformed output.

Each result contains `source`, `title`, `link`, and `snippet`. Calls use bounded timeouts, retry transient failures, deduplicate URLs, and continue when individual providers fail.