Self Querying Retriever

Self-Querying Retriever

The Pattern

User types natural language. The LLM produces a structured (search_string, filter) tuple that the retriever executes against a vector store with metadata filtering.

"books by Asimov after 2000"
    |
    v
{ query: "books", filter: AND(eq(author, "Asimov"), gt(year, 2000)) }

Separating semantic query from symbolic filter is the key. Semantic similarity cannot express year > 2000; metadata filters cannot express "books about first contact". Together they work.

Core Flow

[NL query] -> [Structured query LLM] -> {query, filter} -> [vector store + metadata filter] -> docs
                    ^
                    |
         metadata schema description

LangChain 0.3+ SelfQueryRetriever

from langchain.chains.query_constructor.schema import AttributeInfo
from langchain.retrievers.self_query.base import SelfQueryRetriever
from langchain_chroma import Chroma
from langchain_openai import OpenAIEmbeddings
from langchain_anthropic import ChatAnthropic
from langchain_core.documents import Document

docs = [
    Document(page_content="Foundation is a 1951 novel about...",
             metadata={"title": "Foundation", "author": "Isaac Asimov",
                       "year": 1951, "genre": "science fiction", "rating": 4.4}),
    Document(page_content="The Gods Themselves explores...",
             metadata={"title": "The Gods Themselves", "author": "Isaac Asimov",
                       "year": 1972, "genre": "science fiction", "rating": 4.1}),
    Document(page_content="Prelude to Foundation is set before...",
             metadata={"title": "Prelude to Foundation", "author": "Isaac Asimov",
                       "year": 1988, "genre": "science fiction", "rating": 4.0}),
    Document(page_content="Forward the Foundation...",
             metadata={"title": "Forward the Foundation", "author": "Isaac Asimov",
                       "year": 1993, "genre": "science fiction", "rating": 4.2}),
]

vstore = Chroma.from_documents(docs, OpenAIEmbeddings(model="text-embedding-3-small"))

metadata_field_info = [
    AttributeInfo(name="title", description="Book title", type="string"),
    AttributeInfo(name="author", description="Author full name", type="string"),
    AttributeInfo(name="year", description="Publication year", type="integer"),
    AttributeInfo(name="genre",
                  description="Genre: 'science fiction', 'fantasy', 'mystery', 'non-fiction'",
                  type="string"),
    AttributeInfo(name="rating",
                  description="Average reader rating 1.0-5.0", type="float"),
]

document_content_description = "Summary of a novel"

llm = ChatAnthropic(model="claude-sonnet-4-5-20250929", temperature=0)

retriever = SelfQueryRetriever.from_llm(
    llm=llm,
    vectorstore=vstore,
    document_contents=document_content_description,
    metadata_field_info=metadata_field_info,
    enable_limit=True,              # "top 3 books..." populates `k`
    use_original_query=False,       # pass the rewritten semantic query
    verbose=True,
)

docs = retriever.invoke("Asimov novels after 1970 rated above 4")

The LLM emits a structured query that the translator converts to Chroma's filter syntax:

{
    "query": "novels",
    "filter": {
        "$and": [
            {"author": {"$eq": "Isaac Asimov"}},
            {"year": {"$gt": 1970}},
            {"rating": {"$gt": 4}},
        ]
    },
    "limit": 4
}

Supported Comparators and Operators

Built-in (availability varies per backend):

Operators: and, or, not.

Pinecone, Weaviate, Qdrant, pgvector, Elasticsearch, Milvus, Chroma, and MongoDB each have a translator class in langchain.retrievers.self_query.*.

Qdrant Example (production vector DB)

from langchain_qdrant import QdrantVectorStore

vstore = QdrantVectorStore.from_documents(
    docs, OpenAIEmbeddings(), url="http://localhost:6333", collection_name="books"
)

retriever = SelfQueryRetriever.from_llm(
    llm=llm,
    vectorstore=vstore,
    document_contents=document_content_description,
    metadata_field_info=metadata_field_info,
)

The Qdrant translator maps to native must/should/must_not clauses with range, match, match_any.

Custom Prompt for Better Filter Inference

The default prompt is good but domain-specific hints pay off on narrow schemas.

from langchain.chains.query_constructor.base import (
    StructuredQueryOutputParser, get_query_constructor_prompt,
)
from langchain.retrievers.self_query.chroma import ChromaTranslator

examples = [
    (
        "top 5 highly rated sci-fi from the 80s",
        {
            "query": "science fiction novels",
            "filter": 'and(eq("genre","science fiction"),gte("year",1980),lte("year",1989),gte("rating",4))',
            "limit": 5,
        },
    ),
    (
        "anything by Asimov except Foundation",
        {
            "query": "novels",
            "filter": 'and(eq("author","Isaac Asimov"),ne("title","Foundation"))',
        },
    ),
]

prompt = get_query_constructor_prompt(
    document_contents=document_content_description,
    attribute_info=metadata_field_info,
    examples=examples,
)
output_parser = StructuredQueryOutputParser.from_components()
query_constructor = prompt | llm | output_parser

retriever = SelfQueryRetriever(
    query_constructor=query_constructor,
    vectorstore=vstore,
    structured_query_translator=ChromaTranslator(),
)

Include 3-5 domain-specific examples. Test-time accuracy improves 10-30% vs the default prompt on non-trivial schemas.

LlamaIndex AutoRetriever

from llama_index.core.retrievers import VectorIndexAutoRetriever
from llama_index.core.vector_stores.types import MetadataInfo, VectorStoreInfo
from llama_index.llms.anthropic import Anthropic

vector_store_info = VectorStoreInfo(
    content_info="summary of a novel",
    metadata_info=[
        MetadataInfo(name="author", type="str", description="Author full name"),
        MetadataInfo(name="year", type="int", description="Publication year"),
        MetadataInfo(name="genre", type="str",
                     description="One of science fiction, fantasy, mystery, non-fiction"),
        MetadataInfo(name="rating", type="float", description="Rating 1.0-5.0"),
    ],
)

retriever = VectorIndexAutoRetriever(
    index,
    vector_store_info=vector_store_info,
    llm=Anthropic(model="claude-sonnet-4-5-20250929"),
    similarity_top_k=10,
    empty_query_top_k=10,      # if the LLM produces empty semantic query
    verbose=True,
)
nodes = retriever.retrieve("Asimov novels after 1970 rated above 4")

Combining with Hybrid Search

Self-query produces (query, filter). Pass the filter to both the dense retriever and the BM25 retriever; BM25 handles filter via post-filtering since most implementations have no native metadata filter.

from langchain_community.retrievers import BM25Retriever
from langchain.retrievers import EnsembleRetriever

def self_query_hybrid(nl_query: str):
    structured = query_constructor.invoke({"query": nl_query})
    filter_fn = make_python_filter(structured.filter)

    filtered_docs = [d for d in all_docs if filter_fn(d.metadata)]
    if not filtered_docs:
        return []

    bm25 = BM25Retriever.from_documents(filtered_docs); bm25.k = 20
    dense = vstore.as_retriever(search_kwargs={"k": 20, "filter": structured.filter})

    hybrid = EnsembleRetriever(retrievers=[bm25, dense], weights=[0.4, 0.6])
    return hybrid.invoke(structured.query)

Evaluating Filter Correctness

Build a labeled set (nl_query, expected_filter, expected_query). Score with exact filter equality and semantic match for the query.

from dataclasses import dataclass
from typing import Any

@dataclass
class FilterTest:
    nl: str
    expected_filter: dict[str, Any]
    expected_query: str

tests = [
    FilterTest("Asimov post-2000", {"$and": [{"author":"Isaac Asimov"},{"year":{"$gt":2000}}]}, "novels"),
    FilterTest("5-star fantasy", {"$and": [{"genre":"fantasy"},{"rating":{"$gte":5.0}}]}, "fantasy novels"),
]

def filter_accuracy(constructor, tests):
    correct = 0
    for t in tests:
        out = constructor.invoke({"query": t.nl})
        if normalize(out.filter) == normalize(t.expected_filter):
            correct += 1
    return correct / len(tests)

Track filter accuracy separately from retrieval accuracy. A wrong filter can produce zero results; a wrong semantic query produces low precision.

Pitfalls on Sparse/High-Cardinality Fields

genre in {sci-fi, fantasy, mystery} — the LLM must pick the canonical spelling. Describe the enum in AttributeInfo.description, or pre-validate:

VALID_GENRES = {"science fiction", "fantasy", "mystery", "non-fiction"}

def validate(filter_dict):
    # pseudo: walk the filter tree; reject unknown genres
    ...

For high-cardinality fields (author: 10k names), provide a lookup or ask the LLM to normalize with a second pass.

Anti-Patterns

Production Checklist

• Metadata schema documented with types, enums, and units
• Examples embedded in the constructor prompt (3-5 domain cases)
• Temperature 0 on the constructor LLM
• Filter accuracy eval set (>= 50 labeled queries)
• Structured output validation (Pydantic) on the constructor step
• Translator matched to the active vector backend
• Hybrid search runs with the same filter applied to BM25 and dense
• Empty-result handler loosens filter or asks user to clarify
• Generated filter logged per query for tracing
• Fallback to plain similarity when the constructor fails
• High-cardinality fields normalized (LLM second pass or lookup)
• Cost tracked — constructor is one extra LLM call per query