RAG

Knowledge Retrieval (RAG)

Retrieval-Augmented Generation (RAG) connects an LLM to your data. Since LLMs have a cutoff date and don't know your private documents, RAG solves this by first searching for relevant information in a database and then pasting it into the prompt context. This grounds the answer in facts and reduces hallucinations.

When to Use

• Domain-Specific QA: Answering questions about internal documentation, legal contracts, or medical records.
• Dynamic Data: When the information changes frequently (news, stock analysis).
• Verifiability: When the answer must cite sources ("According to policy document A...").
• Cost: To avoid fine-tuning models on new data, which is expensive and slow.

Use Cases

• Enterprise Search: "How do I reset my VPN password?" (Searches IT Wiki).
• Legal Analysis: "Summarize the liability clause in this contract." (Searches contract PDF).
• Customer Support: "What is my order status?" (Searches Order Database).

Implementation Pattern

def rag_workflow(user_query):
    # Step 1: Retrieval
    # Convert query to vector and search vector DB
    relevant_docs = vector_db.similarity_search(user_query, k=3)
    
    # Step 2: Prompt Construction
    # Combine the context with the user question
    context_text = "\n".join([doc.content for doc in relevant_docs])
    
    prompt = f"""
    You are a helpful assistant. Answer the question based ONLY on the context below.
    
    Context:
    {context_text}
    
    Question: {user_query}
    """
    
    # Step 3: Generation
    answer = llm.generate(prompt)
    
    return answer

Examples

Input: "Answer questions about our internal HR handbook using RAG."

# Index
docs = load_documents("hr-handbook.pdf")
chunks = chunk(docs, size=512, overlap=50)
vector_store.upsert(embed(chunks))

# Query
query = "What is the parental leave policy?"
relevant = vector_store.search(embed(query), top_k=5)
answer = llm.answer(query, context=relevant)

Output: "Our policy provides 16 weeks of fully paid leave for primary caregivers [HR Handbook, Section 7.2]." — with source citation.

Input: "My RAG pipeline returns irrelevant chunks."

Diagnostic: Check chunk size (too large dilutes signal), embedding model alignment, and whether query expansion is needed for short queries.

Troubleshooting