SearXNG Advanced Search Skill

SearXNG Advanced Search Skill

---
name: searxng-skill
description: Advanced Python library for SearXNG metasearch with retry logic, timeout handling, and comprehensive search patterns.
homepage: https://github.com/yourusername/searxng-skill
metadata: {"clawdbot":{"emoji":"🔍","requires":{"python":">=3.8","packages":["requests","urllib3","python-dotenv"]},"install":[{"id":"pip","kind":"pip","package":"searxng-skill","label":"Install searxng-skill (pip)"},{"id":"local","kind":"local","command":"pip install -e .","label":"Install from source"}]}}
---

# searxng-skill

Use `searxng-skill` for privacy-focused metasearch with local/remote SearXNG instances. Supports retry logic, timeout handling, and advanced search patterns.

## Setup (once)

**Environment variables (recommended)**
```bash
# Create .env file
cat > .env << EOF
SEARXNG_URL=http://localhost:8080
SEARXNG_TIMEOUT=10
SEARXNG_MAX_RETRIES=3
SEARXNG_RETRY_DELAY=1.0
SEARXNG_BACKOFF_FACTOR=2.0
SEARXNG_VERIFY_SSL=true
SEARXNG_LANGUAGE=en
EOF

Or configuration file

# Create config.json
cat > config.json << EOF
{
  "instance_url": "http://localhost:8080",
  "default_timeout": 10,
  "max_retries": 3,
  "retry_delay": 1.0,
  "backoff_factor": 2.0,
  "verify_ssl": true,
  "default_language": "en"
}
EOF

Initialize in code

from searxng_skill import SearXNGSkill, SearXNGConfig

# From environment
config = SearXNGConfig.from_env()
skill = SearXNGSkill(config=config)

# From file
config = SearXNGConfig.from_file("config.json")
skill = SearXNGSkill(config=config)

# Direct
skill = SearXNGSkill(instance_url="http://localhost:8080")

Common Commands

Basic search

from searxng_skill import SearXNGSkill

skill = SearXNGSkill(instance_url="http://localhost:8080")
results = skill.search("Python programming")

# Access results
for result in results["results"][:10]:
    print(f"{result['title']} - {result['url']}")

Category search

from searxng_skill import SearchCategory

# Single category
results = skill.search("AI", categories=[SearchCategory.NEWS])

# Multiple categories
results = skill.search(
    "climate change",
    categories=[SearchCategory.NEWS, SearchCategory.SCIENCE]
)

Time-filtered search

from searxng_skill import TimeRange

# Recent news (last 24 hours)
news = skill.news_search("AI breakthrough", time_range=TimeRange.DAY)

# Last week
results = skill.search("Python", time_range=TimeRange.WEEK)

Image search

from searxng_skill import SafeSearch

images = skill.image_search(
    "nature photography",
    safesearch=SafeSearch.STRICT
)

for img in images[:10]:
    print(f"{img['title']}: {img.get('img_src', 'N/A')}")

Video search

videos = skill.video_search("Python tutorial")

for video in videos[:5]:
    print(f"{video['title']} ({video.get('duration', 'N/A')})")

Advanced search with operators

results = skill.advanced_search(
    query="machine learning",
    exact_phrase="deep learning",
    exclude_words=["tutorial", "beginner"],
    site="github.com",
    filetype="pdf"
)

Engine-specific search

results = skill.search(
    "quantum computing",
    engines=["google", "duckduckgo", "wikipedia"]
)

Structured search

response = skill.search_structured("artificial intelligence")

print(f"Query: {response.query}")
print(f"Total: {response.number_of_results}")

for result in response.results[:5]:
    print(f"{result.title} [{result.engine}]")
    print(f"  {result.url}")
    print(f"  Score: {result.score}\n")

Autocomplete

suggestions = skill.autocomplete("artificial int")
# ['artificial intelligence', 'artificial intelligence news', ...]

Multi-category search

categorized = skill.multi_category_search(
    "climate change",
    categories=[
        SearchCategory.GENERAL,
        SearchCategory.NEWS,
        SearchCategory.SCIENCE
    ]
)

for category, results in categorized.items():
    print(f"{category}: {len(results)} results")

Parallel searches

queries = ["Python", "JavaScript", "Go", "Rust"]
results = skill.parallel_search(queries, categories=[SearchCategory.IT])

for query, result_data in results.items():
    print(f"{query}: {len(result_data.get('results', []))} results")

Paginated search

# Get multiple pages
all_results = []
for page in range(1, 4):
    results = skill.search("AI", page=page)
    all_results.extend(results["results"])

print(f"Total results: {len(all_results)}")

Health check

if skill.health_check():
    print("✓ SearXNG instance is healthy")
else:
    print("✗ Instance unavailable")

Get engine info

engines = skill.get_engines_info()

for engine in engines[:10]:
    print(f"{engine.name} - {', '.join(engine.categories)}")
    print(f"  Enabled: {engine.enabled}")

Export results

from searxng_skill.utils import export_results_json, export_results_csv

results = skill.search("machine learning")

# Export to JSON
export_results_json(results, "results.json")

# Export to CSV
export_results_csv(results["results"], "results.csv")

Quick Reference

Search Categories

SearchCategory.GENERAL      # General web search
SearchCategory.IMAGES       # Image search
SearchCategory.VIDEOS       # Video search
SearchCategory.NEWS         # News articles
SearchCategory.MAP          # Maps
SearchCategory.MUSIC        # Music
SearchCategory.IT           # IT/Tech
SearchCategory.SCIENCE      # Scientific papers
SearchCategory.FILES        # File search
SearchCategory.SOCIAL_MEDIA # Social media

Time Ranges

TimeRange.DAY    # Last 24 hours
TimeRange.WEEK   # Last 7 days
TimeRange.MONTH  # Last 30 days
TimeRange.YEAR   # Last year
TimeRange.ALL    # All time

Safe Search Levels

SafeSearch.NONE     # No filtering (0)
SafeSearch.MODERATE # Moderate filtering (1)
SafeSearch.STRICT   # Strict filtering (2)

Notes

Environment variable shortcuts

• Set SEARXNG_URL=http://localhost:8080 to avoid repeating instance URL
• Set SEARXNG_LANGUAGE=en for default language
• Set SEARXNG_MAX_RETRIES=5 for custom retry behavior

For scripting

• Use format=OutputFormat.JSON for structured output (default)
• Set verify_ssl=False for local development only
• Implement rate limiting with time.sleep() between requests
• Use health_check() before batch operations

Performance tips

• Reuse SearXNGSkill instance for multiple searches
• Use parallel_search() for independent queries
• Cache results with functools.lru_cache for repeated queries
• Set appropriate timeouts: fast queries (5s), complex queries (30s)

Retry behavior

• Automatic retry with exponential backoff on timeout/connection errors
• Default: 3 retries, 1s initial delay, 2x backoff factor
• Customize: max_retries, retry_delay, backoff_factor
• No retry on HTTP 4xx/5xx errors

Error handling

from searxng_skill.exceptions import TimeoutException, ConnectionException

try:
    results = skill.search("query", timeout=5)
except TimeoutException:
    print("Request timed out - try increasing timeout")
except ConnectionException:
    print("Cannot connect to instance - check URL and network")

Fallback instance

try:
    skill = SearXNGSkill(instance_url="http://localhost:8080")
    results = skill.search("query")
except ConnectionException:
    # Fallback to public instance
    skill = SearXNGSkill(instance_url="https://searx.be")
    results = skill.search("query")

Local instance setup

# Docker
docker run -d -p 8080:8080 searxng/searxng

# Verify
curl http://localhost:8080/healthz

Confirm before operations

• Always validate query is not empty: if query.strip(): ...
• Check results exist: if "results" in results: ...
• Validate URLs before requests: validate_url(url)

Advanced Usage

Custom retry strategy

from searxng_skill.retry import RetryStrategy

strategy = RetryStrategy(
    max_retries=5,
    initial_delay=2.0,
    backoff_factor=1.5,
    max_delay=60.0,
    jitter=True
)

skill = SearXNGSkill(
    instance_url="http://localhost:8080",
    max_retries=5,
    retry_delay=2.0,
    backoff_factor=1.5
)

Batch processing

import time

queries = ["Python", "Java", "Go", "Rust"]

for query in queries:
    results = skill.search(query)
    print(f"{query}: {len(results['results'])} results")
    time.sleep(1)  # Rate limiting

Result deduplication

from searxng_skill.utils import deduplicate_results

results = skill.search("AI")
unique = deduplicate_results(results["results"], key="url")

Merge multiple searches

from searxng_skill.utils import merge_search_results

r1 = skill.search("Python", engines=["google"])
r2 = skill.search("Python", engines=["duckduckgo"])

merged = merge_search_results([r1, r2])

Logging

import logging

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

logger.info("Searching for: Python")
results = skill.search("Python")
logger.info(f"Found {len(results['results'])} results")

API Methods

Troubleshooting

Connection refused

# Check instance is running
curl http://localhost:8080/healthz

# Or in Python
if not skill.health_check():
    print("Instance is down")

SSL errors (local development)

skill = SearXNGSkill(
    instance_url="http://localhost:8080",
    verify_ssl=False  # Only for local dev!
)

Timeout issues

# Increase timeout
skill = SearXNGSkill(
    instance_url="http://localhost:8080",
    default_timeout=30
)

# Or per-request
results = skill.search("query", timeout=30)

No results

# Check enabled engines
engines = skill.get_engines_info()
enabled = [e for e in engines if e.enabled]
print(f"Enabled: {len(enabled)} engines")

# Try specific engines
results = skill.search("query", engines=["duckduckgo"])