RAG Implementation
Master Retrieval-Augmented Generation (RAG) to build LLM applications that provide accurate, grounded responses using external knowledge sources.
When to Use This Skill
- Building Q&A systems over proprietary documents
- Creating chatbots with current, factual information
- Implementing semantic search with natural language queries
- Reducing hallucinations with grounded responses
- Enabling LLMs to access domain-specific knowledge
- Building documentation assistants
- Creating research tools with source citation
Core Components
1. Vector Databases
Purpose: Store and retrieve document embeddings efficiently
Options:
- Pinecone: Managed, scalable, serverless
- Weaviate: Open-source, hybrid search, GraphQL
- Milvus: High performance, on-premise
- Chroma: Lightweight, easy to use, local development
- Qdrant: Fast, filtered search, Rust-based
- pgvector: PostgreSQL extension, SQL integration
2. Embeddings
Purpose: Convert text to numerical vectors for similarity search
Models (2026):
| Model |
Dimensions |
Best For |
| voyage-3-large |
1024 |
Claude apps (Anthropic recommended) |
| voyage-code-3 |
1024 |
Code search |
| text-embedding-3-large |
3072 |
OpenAI apps, high accuracy |
| text-embedding-3-small |
1536 |
OpenAI apps, cost-effective |
| bge-large-en-v1.5 |
1024 |
Open source, local deployment |
| multilingual-e5-large |
1024 |
Multi-language support |
3. Retrieval Strategies
Approaches:
- Dense Retrieval: Semantic similarity via embeddings
- Sparse Retrieval: Keyword matching (BM25, TF-IDF)
- Hybrid Search: Combine dense + sparse with weighted fusion
- Multi-Query: Generate multiple query variations
- HyDE: Generate hypothetical documents for better retrieval
4. Reranking
Purpose: Improve retrieval quality by reordering results
Methods:
- Cross-Encoders: BERT-based reranking (ms-marco-MiniLM)
- Cohere Rerank: API-based reranking
- Maximal Marginal Relevance (MMR): Diversity + relevance
- LLM-based: Use LLM to score relevance
Quick Start with LangGraph
from langgraph.graph import StateGraph, START, END
from langchain_anthropic import ChatAnthropic
from langchain_voyageai import VoyageAIEmbeddings
from langchain_pinecone import PineconeVectorStore
from langchain_core.documents import Document
from langchain_core.prompts import ChatPromptTemplate
from langchain_text_splitters import RecursiveCharacterTextSplitter
from typing import TypedDict, Annotated
class RAGState(TypedDict):
question: str
context: list[Document]
answer: str
# Initialize components
llm = ChatAnthropic(model="claude-sonnet-4-6")
embeddings = VoyageAIEmbeddings(model="voyage-3-large")
vectorstore = PineconeVectorStore(index_name="docs", embedding=embeddings)
retriever = vectorstore.as_retriever(search_kwargs={"k": 4})
# RAG prompt
rag_prompt = ChatPromptTemplate.from_template(
"""Answer based on the context below. If you cannot answer, say so.
Context:
{context}
Question: {question}
Answer:"""
)
async def retrieve(state: RAGState) -> RAGState:
"""Retrieve relevant documents."""
docs = await retriever.ainvoke(state["question"])
return {"context": docs}
async def generate(state: RAGState) -> RAGState:
"""Generate answer from context."""
context_text = "\n\n".join(doc.page_content for doc in state["context"])
messages = rag_prompt.format_messages(
context=context_text,
question=state["question"]
)
response = await llm.ainvoke(messages)
return {"answer": response.content}
# Build RAG graph
builder = StateGraph(RAGState)
builder.add_node("retrieve", retrieve)
builder.add_node("generate", generate)
builder.add_edge(START, "retrieve")
builder.add_edge("retrieve", "generate")
builder.add_edge("generate", END)
rag_chain = builder.compile()
# Use
result = await rag_chain.ainvoke({"question": "What are the main features?"})
print(result["answer"])
Detailed patterns and worked examples
Detailed pattern documentation lives in references/details.md. Read that file when the navigation tier above is insufficient.
1---2name: rag-implementation3description: Build Retrieval-Augmented Generation (RAG) systems for LLM applications with vector databases and semantic search. Use when implementing knowledge-grounded AI, building document Q&A systems, or integrating LLMs with external knowledge bases.4metadata:5 source: https://skillmd.com/skills/ericgrill/rag-implementation6---78# RAG Implementation910Master Retrieval-Augmented Generation (RAG) to build LLM applications that provide accurate, grounded responses using external knowledge sources.1112## When to Use This Skill1314- Building Q&A systems over proprietary documents15- Creating chatbots with current, factual information16- Implementing semantic search with natural language queries17- Reducing hallucinations with grounded responses18- Enabling LLMs to access domain-specific knowledge19- Building documentation assistants20- Creating research tools with source citation2122## Core Components2324### 1. Vector Databases2526**Purpose**: Store and retrieve document embeddings efficiently2728**Options:**2930- **Pinecone**: Managed, scalable, serverless31- **Weaviate**: Open-source, hybrid search, GraphQL32- **Milvus**: High performance, on-premise33- **Chroma**: Lightweight, easy to use, local development34- **Qdrant**: Fast, filtered search, Rust-based35- **pgvector**: PostgreSQL extension, SQL integration3637### 2. Embeddings3839**Purpose**: Convert text to numerical vectors for similarity search4041**Models (2026):**42| Model | Dimensions | Best For |43|-------|------------|----------|44| **voyage-3-large** | 1024 | Claude apps (Anthropic recommended) |45| **voyage-code-3** | 1024 | Code search |46| **text-embedding-3-large** | 3072 | OpenAI apps, high accuracy |47| **text-embedding-3-small** | 1536 | OpenAI apps, cost-effective |48| **bge-large-en-v1.5** | 1024 | Open source, local deployment |49| **multilingual-e5-large** | 1024 | Multi-language support |5051### 3. Retrieval Strategies5253**Approaches:**5455- **Dense Retrieval**: Semantic similarity via embeddings56- **Sparse Retrieval**: Keyword matching (BM25, TF-IDF)57- **Hybrid Search**: Combine dense + sparse with weighted fusion58- **Multi-Query**: Generate multiple query variations59- **HyDE**: Generate hypothetical documents for better retrieval6061### 4. Reranking6263**Purpose**: Improve retrieval quality by reordering results6465**Methods:**6667- **Cross-Encoders**: BERT-based reranking (ms-marco-MiniLM)68- **Cohere Rerank**: API-based reranking69- **Maximal Marginal Relevance (MMR)**: Diversity + relevance70- **LLM-based**: Use LLM to score relevance7172## Quick Start with LangGraph7374```python75from langgraph.graph import StateGraph, START, END76from langchain_anthropic import ChatAnthropic77from langchain_voyageai import VoyageAIEmbeddings78from langchain_pinecone import PineconeVectorStore79from langchain_core.documents import Document80from langchain_core.prompts import ChatPromptTemplate81from langchain_text_splitters import RecursiveCharacterTextSplitter82from typing import TypedDict, Annotated8384class RAGState(TypedDict):85 question: str86 context: list[Document]87 answer: str8889# Initialize components90llm = ChatAnthropic(model="claude-sonnet-4-6")91embeddings = VoyageAIEmbeddings(model="voyage-3-large")92vectorstore = PineconeVectorStore(index_name="docs", embedding=embeddings)93retriever = vectorstore.as_retriever(search_kwargs={"k": 4})9495# RAG prompt96rag_prompt = ChatPromptTemplate.from_template(97 """Answer based on the context below. If you cannot answer, say so.9899 Context:100 {context}101102 Question: {question}103104 Answer:"""105)106107async def retrieve(state: RAGState) -> RAGState:108 """Retrieve relevant documents."""109 docs = await retriever.ainvoke(state["question"])110 return {"context": docs}111112async def generate(state: RAGState) -> RAGState:113 """Generate answer from context."""114 context_text = "\n\n".join(doc.page_content for doc in state["context"])115 messages = rag_prompt.format_messages(116 context=context_text,117 question=state["question"]118 )119 response = await llm.ainvoke(messages)120 return {"answer": response.content}121122# Build RAG graph123builder = StateGraph(RAGState)124builder.add_node("retrieve", retrieve)125builder.add_node("generate", generate)126builder.add_edge(START, "retrieve")127builder.add_edge("retrieve", "generate")128builder.add_edge("generate", END)129130rag_chain = builder.compile()131132# Use133result = await rag_chain.ainvoke({"question": "What are the main features?"})134print(result["answer"])135```136137## Detailed patterns and worked examples138139Detailed pattern documentation lives in `references/details.md`. Read that file when the navigation tier above is insufficient.140