LiteLLM Python SDK Usage

Overview

:::tip

LiteLLM follows the Perplexity API request/response for the Search API

:::

:::info

Supported from LiteLLM v1.78.7+ :::

LiteLLM Python SDK Usage

Quick Start

from litellm import search
import os

os.environ["PERPLEXITYAI_API_KEY"] = "pplx-..."

response = search(
    query="latest AI developments in 2024",
    search_provider="perplexity",
    max_results=5
)

# Access search results
for result in response.results:
    print(f"{result.title}: {result.url}")
    print(f"Snippet: {result.snippet}\n")

Async Usage

from litellm import asearch
import os, asyncio

os.environ["PERPLEXITYAI_API_KEY"] = "pplx-..."

async def search_async(): 
    response = await asearch(
        query="machine learning research papers",
        search_provider="perplexity",
        max_results=10,
        search_domain_filter=["arxiv.org", "nature.com"]
    )
    
    # Access search results
    for result in response.results:
        print(f"{result.title}: {result.url}")
        print(f"Snippet: {result.snippet}")

asyncio.run(search_async())

Optional Parameters

response = search(
    query="AI developments",
    search_provider="perplexity",
    # Unified parameters (work across all providers)
    max_results=10,                         # Maximum number of results (1-20)
    search_domain_filter=["arxiv.org"],     # Filter to specific domains
    country="US",                           # Country code filter
    max_tokens_per_page=1024                # Max tokens per page
)

LiteLLM AI Gateway Usage

LiteLLM provides a Perplexity API compatible /search endpoint for search calls.

Setup

Add this to your litellm proxy config.yaml

model_list:
  - model_name: gpt-4
    litellm_params:
      model: gpt-4
      api_key: os.environ/OPENAI_API_KEY

search_tools:
  - search_tool_name: perplexity-search
    litellm_params:
      search_provider: perplexity
      api_key: os.environ/PERPLEXITYAI_API_KEY
  
  - search_tool_name: tavily-search
    litellm_params:
      search_provider: tavily
      api_key: os.environ/TAVILY_API_KEY

Start litellm

litellm --config /path/to/config.yaml

# RUNNING on http://0.0.0.0:4000

Test Request

Option 1: Search tool name in URL (Recommended - keeps body Perplexity-compatible)

curl http://0.0.0.0:4000/v1/search/perplexity-search \
  -H "Authorization: Bearer sk-1234" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "latest AI developments 2024",
    "max_results": 5,
    "search_domain_filter": ["arxiv.org", "nature.com"],
    "country": "US"
  }'

Option 2: Search tool name in body

curl http://0.0.0.0:4000/v1/search \
  -H "Authorization: Bearer sk-1234" \
  -H "Content-Type: application/json" \
  -d '{
    "search_tool_name": "perplexity-search",
    "query": "latest AI developments 2024",
    "max_results": 5
  }'

Load Balancing

Configure multiple search providers for automatic load balancing and fallbacks:

search_tools:
  - search_tool_name: my-search
    litellm_params:
      search_provider: perplexity
      api_key: os.environ/PERPLEXITYAI_API_KEY
  
  - search_tool_name: my-search
    litellm_params:
      search_provider: tavily
      api_key: os.environ/TAVILY_API_KEY
  
  - search_tool_name: my-search
    litellm_params:
      search_provider: exa_ai
      api_key: os.environ/EXA_API_KEY

  - search_tool_name: my-search
    litellm_params:
      search_provider: brave
      api_key: os.environ/BRAVE_API_KEY

router_settings:
  routing_strategy: simple-shuffle  # or 'least-busy', 'latency-based-routing'

Test with load balancing:

curl http://0.0.0.0:4000/v1/search/my-search \
  -H "Authorization: Bearer sk-1234" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "AI developments",
    "max_results": 10
  }'

Request/Response Format

:::info

LiteLLM follows the Perplexity Search API specification.

See the official Perplexity Search documentation for complete details.

:::

Example Request

{
  "query": "latest AI developments 2024",
  "max_results": 10,
  "search_domain_filter": ["arxiv.org", "nature.com"],
  "country": "US",
  "max_tokens_per_page": 1024
}

Request Parameters

Query Format Examples:

# Single query
query = "AI developments"

# Multiple queries
query = ["AI developments", "machine learning trends"]

Response Format

The response follows Perplexity's search format with the following structure:

{
  "object": "search",
  "results": [
    {
      "title": "Latest Advances in Artificial Intelligence",
      "url": "https://arxiv.org/paper/example",
      "snippet": "This paper discusses recent developments in AI...",
      "date": "2024-01-15"
    },
    {
      "title": "Machine Learning Breakthroughs",
      "url": "https://nature.com/articles/ml-breakthrough",
      "snippet": "Researchers have achieved new milestones...",
      "date": "2024-01-10"
    }
  ]
}

Response Fields

Supported Providers

See the individual provider documentation for detailed setup instructions and provider-specific parameters.