Spring AI RAG Retrieval

Spring AI - Retrieval Augmented Generation (RAG)

Description

Comprehensive guide for implementing RAG systems using Spring AI. Covers query transformation, document retrieval strategies, context augmentation, reranking, and production deployment patterns.

When to Use

• Building knowledge-based Q&A systems
• Implementing domain-specific AI applications
• Reducing hallucinations in LLM responses
• Integrating external knowledge bases
• Document-grounded conversation systems
• Enterprise AI applications with proprietary data
• Multi-source information fusion

Topics Covered

1. RAG Architecture Components

• Query: User question or prompt
• Retrieval: Fetching relevant documents from vector store
• Context Assembly: Combining retrieved documents with prompt
• LLM Reasoning: Model generates response with context
• Response Generation: Final answer with citations

2. Query Processing Strategies

Query Transformation

• Query Expansion: Generate multiple query variations
• Query Rewriting: Improve query clarity
• Query Decomposition: Break complex questions
• Multi-hop Reasoning: Sequential sub-queries
• Few-shot Example Injection: Context-aware transformation

Retrieval Methods

• Dense Retrieval: Vector similarity search
• Sparse Retrieval: Keyword/BM25 search
• Hybrid Search: Combining dense + sparse
• Multi-vector Search: Different embedding perspectives
• Ensemble Methods: Voting/ranking combination

3. Context Assembly Strategies

• Simple Concatenation: Append retrieved docs
• Ranked Retrieval: By relevance score
• Window-based Assembly: K context items
• Hierarchical Compression: Summarize then include
• Dynamic Context Sizing: Based on token budget

4. Advanced Techniques

Reranking

• Cross-encoder models: Fine-tuned relevance ranking
• Semantic similarity reranking: LLM-based scoring
• MMR (Max Marginal Relevance): Diversity + relevance
• Filtering + ranking: Combined approach

Response Generation

• Structured outputs: JSON response format
• Citation/Grounding: Link to source documents
• Confidence scoring: How confident in response
• Fallback generation: No results handling

5. Evaluation & Metrics

• Retrieval Metrics: MRR, NDCG, Hit@K
• Generation Metrics: BLEU, ROUGE, factuality
• End-to-end Metrics: User satisfaction
• Latency Tracking: Performance monitoring

6. Production Considerations

• Latency Optimization: Async retrieval
• Scalability: Distributed vector stores
• Monitoring: Success rates, quality metrics
• Cost Management: Query batching, caching
• Safety Guardrails: Input validation, rate limiting

Code Patterns

Basic RAG Pipeline

@Service
public class RagService {
    @Autowired
    private VectorStore vectorStore;

    @Autowired
    private ChatClient chatClient;

    public String answerQuestion(String question) {
        // 1. Retrieve relevant documents
        List<Document> docs = vectorStore.similaritySearch(
            SearchRequest.query(question).withTopK(5)
        );

        // 2. Build context
        String context = docs.stream()
            .map(Document::getContent)
            .collect(Collectors.joining("\n---\n"));

        // 3. Generate response
        return chatClient.prompt()
            .system("Context: " + context)
            .user(question)
            .call()
            .content();
    }
}

RAG with Query Transformation

public String enhancedRag(String userQuestion) {
    // 1. Transform query
    String transformedQuery = chatClient.prompt()
        .system("Rewrite for clarity and specificity")
        .user(userQuestion)
        .call()
        .content();

    // 2. Retrieve with transformed query
    List<Document> docs = vectorStore.similaritySearch(
        transformedQuery, 10
    );

    // 3. Rerank results
    List<Document> reranked = reranker.rerank(docs, transformedQuery);

    // 4. Generate response
    return generateAnswer(userQuestion, reranked);
}

Hybrid Search RAG

public List<Document> hybridSearch(String query) {
    SearchRequest request = SearchRequest.query(query)
        .withTopK(10)
        .withHybridSearch(true)
        .withKeywordSearchWeight(0.3)
        .withSemanticSearchWeight(0.7)
        .withSimilarityThreshold(0.5);

    return vectorStore.similaritySearch(request);
}

Citation and Grounding

public ResponseWithCitations ragWithCitations(String question) {
    List<Document> docs = vectorStore.similaritySearch(question, 5);

    String context = docs.stream()
        .map(d -> "[" + d.getMetadata().get("source") + "]\n" + d.getContent())
        .collect(Collectors.joining("\n---\n"));

    String response = chatClient.prompt()
        .system("When using context, cite the source")
        .user("Question: " + question + "\nContext: " + context)
        .call()
        .content();

    return new ResponseWithCitations(response,
        docs.stream()
            .map(d -> d.getMetadata().get("source").toString())
            .toList());
}

Streaming RAG

public Flux<String> streamingRag(String question) {
    List<Document> docs = vectorStore.similaritySearch(question, 5);

    String context = docs.stream()
        .map(Document::getContent)
        .collect(Collectors.joining("\n"));

    return chatClient.prompt()
        .system("Context: " + context)
        .user(question)
        .stream()
        .content();
}

Multi-hop RAG

public String multiHopRag(String question) {
    String currentQuestion = question;
    List<String> subAnswers = new ArrayList<>();

    for (int i = 0; i < 3; i++) {
        // Retrieve context
        List<Document> docs = vectorStore.similaritySearch(currentQuestion, 3);

        // Answer sub-question
        String answer = chatClient.prompt()
            .user(currentQuestion)
            .call()
            .content();
        subAnswers.add(answer);

        // Generate next question
        currentQuestion = chatClient.prompt()
            .user("Based on: " + answer + "\nWhat next to answer: " + question)
            .call()
            .content();
    }

    // Synthesize final answer
    return synthesizeAnswers(subAnswers, question);
}

Configuration

RAG Service Bean

@Configuration
public class RagConfiguration {
    @Bean
    public RagService ragService(
            VectorStore vectorStore,
            ChatClient chatClient,
            DocumentSplitter splitter) {
        return new RagService(vectorStore, chatClient, splitter);
    }
}

Properties

spring.ai.rag.retrieval.top-k=5
spring.ai.rag.retrieval.similarity-threshold=0.75
spring.ai.rag.reranking.enabled=true
spring.ai.rag.reranking.model=cross-encoder
spring.ai.rag.streaming.enabled=true

Performance Optimization

Caching Strategy

@Cacheable(value = "rag-results", key = "#question")
public String cachedRag(String question) {
    return ragService.answer(question);
}

Batch Processing

List<String> questions = List.of(...);
List<String> answers = questions.parallelStream()
    .map(this::ragService::answer)
    .toList();

Best Practices

• Use appropriate similarity threshold
• Implement query expansion for edge cases
• Always include source citations
• Monitor retrieval quality
• Cache common queries
• Implement fallback strategies
• Regular RAG pipeline evaluation
• Document quality management

Related Skills

• vector-stores/SKILL.md - Vector storage
• embeddings/SKILL.md - Creating vectors
• chat-models/SKILL.md - LLM interaction
• document-processing/SKILL.md - Document preparation
• advisors/SKILL.md - RAG advisor pattern

References

• API: /pages/api/retrieval-augmented-generation.adoc
• Examples: /pages/guides/ (multiple RAG guides)
• Vector DB: /pages/api/vectordbs.adoc