SearXNG Integration for Local Deep Research

SearXNG Integration for Local Deep Research

This document explains how to configure and use the SearXNG integration with Local Deep Research.

Configuring SearXNG Access

The SearXNG search engine is disabled by default until you provide an instance URL. This ensures the system doesn't attempt to use public instances without explicit configuration.

Setting Up Access

You have two ways to enable the SearXNG search engine:

1. Environment Variable (Recommended):

   # Add to your .env file or set in your environment
   SEARXNG_INSTANCE=http://localhost:8080
   
   # Optional: Set custom delay between requests (in seconds)
   SEARXNG_DELAY=2.0
   

2. Configuration Parameter: Add to your config.py:

   # In config.py
   SEARXNG_CONFIG = {
       "instance_url": "http://localhost:8080",
       "delay_between_requests": 2.0
   }
   

Self-Hosting SearXNG (Recommended)

For the most ethical usage, we strongly recommend self-hosting your own SearXNG instance:

Using Docker (easiest method)

# Pull the SearXNG Docker image
docker pull searxng/searxng

# Run SearXNG (will be available at http://localhost:8080)
docker run -d -p 8080:8080 --name searxng searxng/searxng

Using Docker Compose (recommended for production)

1. Create a file named docker-compose.yml with the following content:

version: '3'
services:
  searxng:
    container_name: searxng
    image: searxng/searxng
    ports:
      - "8080:8080"
    volumes:
      - ./searxng:/etc/searxng
    environment:
      - SEARXNG_BASE_URL=http://localhost:8080/
    restart: unless-stopped

2. Run with Docker Compose:

docker-compose up -d

Using Public Instances

If you must use a public instance:

1. Get Permission: Always contact the administrator of any public instance
2. Respect Resources: Use a longer delay (4-5 seconds minimum) between requests
3. Limited Usage: Keep your research volume reasonable

Example configuration for a public instance:

SEARXNG_INSTANCE=https://instance.example.com
SEARXNG_DELAY=5.0

Checking Configuration

To verify if SearXNG is properly configured:

from web_search_engines.search_engine_factory import create_search_engine

# Create the engine
engine = create_search_engine("searxng")

# Check if available
if engine and hasattr(engine, 'is_available') and engine.is_available:
    print(f"SearXNG configured with instance: {engine.instance_url}")
    print(f"Delay between requests: {engine.delay_between_requests} seconds")
else:
    print("SearXNG is not properly configured or is disabled")

Network Security

SearXNG is designed for self-hosting, so Local Deep Research allows SearXNG to access private network IPs by default. This means you can run SearXNG on:

• Localhost: http://127.0.0.1:8080 or http://localhost:8080
• LAN IPs: http://192.168.1.100:8080, http://10.0.0.5:8080, http://172.16.0.2:8080
• Docker networks: http://172.17.0.2:8080
• Local hostnames: http://searxng.local:8080 (if configured in DNS/hosts)

This is intentional and secure because:

1. The SearXNG URL is admin-configured, not user input
2. Private IPs are only accessible from your local network
3. The AWS metadata endpoint (169.254.169.254) is always blocked to prevent credential theft in cloud environments

Troubleshooting

If you encounter errors:

1. Check that your instance is running
2. Verify the URL is correct in your environment variables
3. Ensure you can access the instance in your browser
4. Check firewall settings and network connectivity

Resources

• SearXNG Documentation
• SearXNG GitHub Repository
• SearXNG Docker Hub