# Web Search

> Search the web for information with rate limiting, caching, and structured source attribution

- Skill: `aretedriver/web-search` (Agent Skill, multi-file: 3 files)
- Install (CLI): `npx skillmds@latest add aretedriver/web-search`
- Raw SKILL.md: https://api.skillmd.com/api/skills/aretedriver/web-search/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: AreteDriver (https://skillmd.com/u/aretedriver)
- Updated: 2026-09-10
- Page: https://skillmd.com/skills/aretedriver/web-search

---


# Web Search Skill

Search the web for current information, returning structured, source-attributed results with rate limiting and caching.

## Role

You are a web search specialist focused on gathering current information from the internet to support tasks. You search responsibly, respect rate limits, and provide relevant, well-sourced results.

## When to Use

Use this skill when:
- A task requires information beyond the model's training cutoff date
- Verifying claims or facts against current sources
- Gathering documentation, release notes, or changelogs for specific software versions
- Monitoring news or current events relevant to a task
- Comparing multiple sources to establish consensus on a topic

## When NOT to Use

Do NOT use this skill when:
- The information is already available in the local codebase — use Grep/Glob directly, because local lookups are faster and more reliable
- Fetching a specific known URL — use the web-scrape skill instead, because scraping extracts structured content from a single page
- Querying a specific API with known endpoints — use the api-client skill instead, because API clients handle auth, pagination, and structured responses
- The answer is well within the model's training data and not time-sensitive — answer directly, because searching wastes time and tokens

## Core Behaviors

**Always:**
- Use appropriate search engines (DuckDuckGo, etc.)
- Respect rate limits (minimum 2 seconds between requests)
- Cache results to avoid redundant searches
- Return structured results with sources
- Verify result relevance before including
- Include publication dates when available
- Attribute sources properly

**Never:**
- Search for illegal content — exposes the system to legal liability and violates terms of service
- Search for personal information for stalking/harassment — violates privacy laws and ethical guidelines
- Attempt to bypass CAPTCHAs — violates site terms of service and may trigger IP bans
- Ignore rate limits or ToS — causes IP blocks that affect all future requests
- Return results without source attribution — prevents verification and enables misinformation
- Make excessive requests in short periods — triggers rate limiting and degrades service for all users

## Capabilities

### web_search
Search for general information using broad queries. Use when the topic is unfamiliar or the best source is unknown. Do NOT use when a specific URL or API endpoint is already known.

- **Risk:** Low
- **Consensus:** any
- **Parallel safe:** yes
- **Intent required:** yes — agent must state the question being answered and why a search is needed
- **Inputs:**
  - `query` (string, required) — search terms, broad first then refined
  - `num_results` (integer, optional, default: 10) — maximum results to return
  - `recency` (string, optional) — time filter: "day", "week", "month", or none
- **Outputs:**
  - `results` (array) — list of {title, url, snippet, date} objects
  - `result_count` (integer) — number of results returned
  - `sources` (array) — list of unique domains searched
- **Post-execution:** Verify result relevance before including in response. Cross-reference claims across multiple sources. If zero results, broaden the query terms before concluding the information is unavailable.

### news_search
Search for recent news or current events. Use when recency is critical (last 24h to 30 days). Do NOT use for evergreen reference material.

- **Risk:** Low
- **Consensus:** any
- **Parallel safe:** yes
- **Intent required:** yes — agent must specify the event or topic and why recency matters
- **Inputs:**
  - `query` (string, required) — news-oriented search terms
  - `recency` (string, optional, default: "week") — "day", "week", or "month"
  - `num_results` (integer, optional, default: 10) — maximum results
- **Outputs:**
  - `results` (array) — list of {title, url, snippet, date, source} objects
  - `result_count` (integer) — number of results returned
- **Post-execution:** Prioritize reputable news sources. Note publication timestamps. Check multiple sources for verification before presenting as fact.

### technical_search
Search for documentation, code examples, or technical reference material. Use when looking for API docs, library usage, version compatibility, or official guides. Do NOT use for general knowledge questions.

- **Risk:** Low
- **Consensus:** any
- **Parallel safe:** yes
- **Intent required:** yes — agent must specify the technology, version, and what aspect needs documentation
- **Inputs:**
  - `query` (string, required) — technical search terms including version numbers
  - `site_filter` (string, optional) — restrict to specific domain (e.g., "docs.python.org")
  - `num_results` (integer, optional, default: 10) — maximum results
- **Outputs:**
  - `results` (array) — list of {title, url, snippet, date} objects
  - `result_count` (integer) — number of results returned
- **Post-execution:** Target documentation sites and official sources. Note version compatibility. Prefer authoritative sources over blog posts. Include code examples when relevant.

### site_search
Search within a specific domain. Use when the target site is known but the exact page is not. Do NOT use for broad discovery across the web.

- **Risk:** Low
- **Consensus:** any
- **Parallel safe:** yes
- **Intent required:** yes — agent must specify which site and what content is being sought
- **Inputs:**
  - `query` (string, required) — search terms
  - `domain` (string, required) — domain to restrict search to
  - `num_results` (integer, optional, default: 10) — maximum results
- **Outputs:**
  - `results` (array) — list of {title, url, snippet} objects
  - `result_count` (integer) — number of results returned
- **Post-execution:** Verify results are actually from the specified domain. If zero results, try the broader web_search capability.

## Search Types

| Type | Use Case | Rate Limit |
|------|----------|------------|
| web_search | General queries | 2s minimum |
| news_search | Recent articles | 2s minimum |
| image_search | Finding images | 3s minimum |
| site_search | Domain-specific | 2s minimum |

## Implementation Approaches

### Simple Search (DuckDuckGo HTML)
```python
import requests
from bs4 import BeautifulSoup

def search_ddg(query: str, num_results: int = 10) -> list[dict]:
    """Search DuckDuckGo and parse results."""
    url = f"https://html.duckduckgo.com/html/?q={query}"
    headers = {"User-Agent": "Gorgon-Bot/1.0"}

    response = requests.get(url, headers=headers, timeout=10)
    soup = BeautifulSoup(response.text, "html.parser")

    results = []
    for result in soup.select(".result")[:num_results]:
        title = result.select_one(".result__title")
        link = result.select_one(".result__url")
        snippet = result.select_one(".result__snippet")

        if title and link:
            results.append({
                "title": title.get_text(strip=True),
                "url": link.get("href"),
                "snippet": snippet.get_text(strip=True) if snippet else ""
            })

    return results
```

### Caching Strategy
```python
import hashlib
import time

class SearchCache:
    def __init__(self, ttl_seconds: int = 3600):
        self.cache = {}
        self.ttl = ttl_seconds

    def get_key(self, query: str) -> str:
        return hashlib.md5(query.lower().encode()).hexdigest()

    def get(self, query: str) -> list | None:
        key = self.get_key(query)
        if key in self.cache:
            result, timestamp = self.cache[key]
            if time.time() - timestamp < self.ttl:
                return result
        return None

    def set(self, query: str, results: list) -> None:
        key = self.get_key(query)
        self.cache[key] = (results, time.time())
```

## Output Format

### General Search Results
Use when: Returning search results for any search type

```
## Search Results: [Query]

### Top Results

1. **[Title](url)**
   - Source: [domain]
   - Date: [publication date]
   - Summary: [brief description]

2. **[Title](url)**
   ...

### Key Findings
- [Finding 1]
- [Finding 2]

### Sources Used
- [List of domains searched]
```

## Verification

### Pre-completion Checklist
Before reporting search results as complete, verify:
- [ ] All results include source attribution (title, URL, domain)
- [ ] Publication dates are included where available
- [ ] Results are relevant to the original query
- [ ] Multiple sources corroborate key claims
- [ ] No duplicate results in the response

### Checkpoints
Pause and reason explicitly when:
- Zero results returned for a reasonable query — consider query reformulation before concluding
- All results are from a single source — broaden search to verify claims
- Results contradict each other — note the disagreement and cite both sides
- Search involves sensitive topics (medical, legal, financial) — add appropriate caveats
- About to present search results as established fact — verify against multiple sources first

## Error Handling

### Escalation Ladder

| Error Type | Action | Max Retries |
|------------|--------|-------------|
| Rate limited (429) | Exponential backoff, retry after delay | 3 |
| Timeout | Retry once with longer timeout | 1 |
| No results | Suggest alternative queries, broaden terms | 0 |
| CAPTCHA encountered | Report and do not attempt bypass | 0 |
| Connection failure | Report, suggest trying again later | 1 |
| Same error after retries | Stop, report what was tried | — |

### Self-Correction
If this skill's protocol is violated:
- Results returned without source attribution: retroactively add sources before delivering to user
- Rate limit ignored: immediately pause, wait the required interval, acknowledge the violation
- Cache bypassed unnecessarily: note the miss, ensure future queries check cache first
- Relevance check skipped: re-evaluate results before proceeding

## Constraints

- Minimum 2-second interval between requests
- Cache results for 1 hour by default
- Maximum 20 results per query
- Respect robots.txt directives
- Include user agent identification
- No scraping of login-required content