Spring AI

Spring AI — Expert Reference (1.x)

Architecture Overview

┌──────────────────────────────────────────────────────────────────────┐
│                         Spring AI Stack                              │
│                                                                      │
│  ┌──────────────────────────────────────────────────────────────┐    │
│  │                       ChatClient                             │    │
│  │   (fluent API — the primary entry point for applications)    │    │
│  └────────────────────────┬─────────────────────────────────────┘    │
│                           │                                          │
│       ┌───────────────────┼────────────────────┐                     │
│       │                   │                    │                     │
│  ┌────▼─────┐      ┌──────▼──────┐     ┌──────▼──────┐             │
│  │ Advisors │      │  ChatModel  │     │  Prompts /  │             │
│  │ (AOP-    │      │ (provider   │     │  Templates  │             │
│  │  like    │      │  abstraction│     │             │             │
│  │  layer)  │      │  layer)     │     └─────────────┘             │
│  └──────────┘      └──────┬──────┘                                  │
│                           │                                          │
│          ┌────────────────┼────────────────┐                         │
│          │                │                │                         │
│   ┌──────▼───┐   ┌────────▼──────┐  ┌─────▼──────┐                 │
│   │ OpenAI   │   │  Anthropic    │  │   Ollama   │                 │
│   │  Model   │   │  (Claude)     │  │  (local)   │  + more         │
│   └──────────┘   └───────────────┘  └────────────┘                 │
│                                                                      │
│  ┌──────────────────────────────────────────────────────────────┐    │
│  │  RAG Pipeline                                                │    │
│  │  DocumentReader → DocumentTransformer → VectorStore          │    │
│  │                                            │                 │    │
│  │                              QuestionAnswerAdvisor ──────────┘    │
│  └──────────────────────────────────────────────────────────────┘    │
│                                                                      │
│  ┌──────────────────────────────────────────────────────────────┐    │
│  │  Observability: Micrometer metrics + traces (all model calls) │   │
│  └──────────────────────────────────────────────────────────────┘    │
└──────────────────────────────────────────────────────────────────────┘

Dependencies (Maven)

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.springframework.ai</groupId>
            <artifactId>spring-ai-bom</artifactId>
            <version>1.0.0</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

<dependencies>
    <!-- Core Spring AI + Spring Boot -->
    <dependency>
        <groupId>org.springframework.ai</groupId>
        <artifactId>spring-ai-openai-spring-boot-starter</artifactId>
    </dependency>
    <!-- Or Anthropic Claude -->
    <dependency>
        <groupId>org.springframework.ai</groupId>
        <artifactId>spring-ai-anthropic-spring-boot-starter</artifactId>
    </dependency>
    <!-- Or Ollama (local models) -->
    <dependency>
        <groupId>org.springframework.ai</groupId>
        <artifactId>spring-ai-ollama-spring-boot-starter</artifactId>
    </dependency>

    <!-- Vector store (choose one) -->
    <dependency>
        <groupId>org.springframework.ai</groupId>
        <artifactId>spring-ai-pgvector-store-spring-boot-starter</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.ai</groupId>
        <artifactId>spring-ai-redis-store-spring-boot-starter</artifactId>
    </dependency>
</dependencies>

// Gradle (Kotlin DSL)
implementation(platform("org.springframework.ai:spring-ai-bom:1.0.0"))
implementation("org.springframework.ai:spring-ai-openai-spring-boot-starter")

Configuration

# application.yml

# OpenAI
spring:
  ai:
    openai:
      api-key: ${OPENAI_API_KEY}
      chat:
        options:
          model: gpt-4o
          temperature: 0.7
          max-tokens: 2048
      embedding:
        options:
          model: text-embedding-3-small

# Anthropic Claude
spring:
  ai:
    anthropic:
      api-key: ${ANTHROPIC_API_KEY}
      chat:
        options:
          model: claude-opus-4-5
          max-tokens: 4096

# Ollama (local)
spring:
  ai:
    ollama:
      base-url: http://localhost:11434
      chat:
        options:
          model: llama3.2

# pgvector
spring:
  ai:
    vectorstore:
      pgvector:
        index-type: HNSW
        distance-type: COSINE_DISTANCE
        dimensions: 1536

ChatClient — Fluent API

ChatClient is the primary API for interacting with chat models. It wraps ChatModel with a fluent builder that handles prompts, advisors, tools, and structured output.

Basic Usage

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.model.ChatModel;

@Service
public class AssistantService {

    private final ChatClient chatClient;

    public AssistantService(ChatModel chatModel) {
        this.chatClient = ChatClient.create(chatModel);
    }

    public String ask(String question) {
        return chatClient
            .prompt()
            .system("You are a helpful assistant.")
            .user(question)
            .call()
            .content();  // returns String
    }

    // With variable substitution in prompt templates
    public String explainTopic(String topic) {
        return chatClient
            .prompt()
            .system("You are a technical educator.")
            .user(u -> u.text("Explain {topic} in simple terms").param("topic", topic))
            .call()
            .content();
    }
}

// Kotlin
@Service
class AssistantService(chatModel: ChatModel) {
    private val chatClient = ChatClient.create(chatModel)

    fun ask(question: String): String =
        chatClient.prompt()
            .system("You are a helpful assistant.")
            .user(question)
            .call()
            .content()
}

Shared ChatClient with Defaults

@Configuration
public class AiConfig {

    @Bean
    public ChatClient chatClient(ChatModel chatModel) {
        // Set defaults once — reused across all calls
        return ChatClient.builder(chatModel)
            .defaultSystem("You are a helpful assistant for Acme Corp.")
            .defaultAdvisors(
                new MessageChatMemoryAdvisor(new InMemoryChatMemory()),
                new SimpleLoggerAdvisor()
            )
            .defaultOptions(
                OpenAiChatOptions.builder()
                    .withTemperature(0.7f)
                    .build()
            )
            .build();
    }
}

@Service
public class SupportService {

    private final ChatClient chatClient;

    public SupportService(ChatClient chatClient) {
        this.chatClient = chatClient;
    }

    public String handleTicket(String issue) {
        // Overrides system only for this call
        return chatClient
            .prompt()
            .system("You are a technical support specialist.")
            .user(issue)
            .call()
            .content();
    }
}

Structured Output

import org.springframework.ai.chat.client.ChatClient;

public record MovieReview(
    String title,
    double rating,
    String summary,
    List<String> pros,
    List<String> cons
) {}

@Service
public class ReviewService {

    private final ChatClient chatClient;

    public ReviewService(ChatClient chatClient) {
        this.chatClient = chatClient;
    }

    public MovieReview extractReview(String reviewText) {
        return chatClient
            .prompt()
            .system("Extract movie review information from the text.")
            .user(reviewText)
            .call()
            .entity(MovieReview.class);  // auto-generates format instructions & parses
    }

    // List of entities
    public List<MovieReview> extractMultipleReviews(String text) {
        return chatClient
            .prompt()
            .user(text)
            .call()
            .entity(new ParameterizedTypeReference<List<MovieReview>>() {});
    }
}

// Kotlin — data class works directly
data class ProductInfo(
    val name: String,
    val price: Double,
    val inStock: Boolean,
    val description: String,
)

fun extractProduct(text: String): ProductInfo =
    chatClient.prompt()
        .user(text)
        .call()
        .entity(ProductInfo::class.java)

BeanOutputConverter (Lower-level)

import org.springframework.ai.converter.BeanOutputConverter;

var converter = new BeanOutputConverter<>(MovieReview.class);
String formatInstructions = converter.getFormat();

Prompt prompt = new PromptTemplate("""
    Extract movie info.
    {format}
    Text: {text}
    """)
    .create(Map.of("format", formatInstructions, "text", reviewText));

ChatResponse response = chatModel.call(prompt);
MovieReview review = converter.convert(response.getResult().getOutput().getContent());

Streaming

import reactor.core.publisher.Flux;

@Service
public class StreamingService {

    private final ChatClient chatClient;

    public Flux<String> streamResponse(String question) {
        return chatClient
            .prompt()
            .user(question)
            .stream()           // returns StreamResponseSpec
            .content();         // returns Flux<String>
    }

    // Full ChatResponse flux (includes metadata, usage)
    public Flux<ChatResponse> streamFullResponse(String question) {
        return chatClient
            .prompt()
            .user(question)
            .stream()
            .chatResponse();
    }
}

// Spring WebFlux controller — SSE endpoint
@RestController
public class ChatController {

    private final StreamingService streamingService;

    @GetMapping(value = "/chat/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
    public Flux<String> streamChat(@RequestParam String question) {
        return streamingService.streamResponse(question);
    }
}

// Kotlin coroutines alternative
fun streamResponse(question: String): Flow<String> =
    chatClient.prompt()
        .user(question)
        .stream()
        .content()
        .asFlow()

Tool / Function Calling

@Bean Method with @Description

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Description;

@Configuration
public class ToolConfig {

    @Bean
    @Description("Get the current weather for a given city")
    public Function<WeatherRequest, WeatherResponse> getWeather() {
        return request -> {
            // call real weather API
            return new WeatherResponse(request.city(), "Sunny", 22.0);
        };
    }

    @Bean
    @Description("Search for products in the catalog by name or category")
    public Function<ProductSearchRequest, List<Product>> searchProducts(ProductRepository repo) {
        return request -> repo.findByNameContaining(request.query());
    }
}

// Request/Response records — Spring AI reads field names as parameter schema
record WeatherRequest(
    @JsonPropertyDescription("City name, e.g. 'London'") String city
) {}

record WeatherResponse(String city, String condition, double temperatureCelsius) {}

// Register tools on ChatClient call
@Service
public class AgentService {

    private final ChatClient chatClient;

    public String answerWithTools(String question) {
        return chatClient
            .prompt()
            .user(question)
            .tools("getWeather", "searchProducts")  // bean names
            .call()
            .content();
    }
}

FunctionCallback (Programmatic, Dynamic Tools)

import org.springframework.ai.model.function.FunctionCallback;

FunctionCallback dynamicTool = FunctionCallback.builder()
    .function("calculate", (String expression) -> {
        // evaluate expression
        return "42";
    })
    .description("Evaluate a mathematical expression")
    .inputType(String.class)
    .build();

String result = chatClient
    .prompt()
    .user("What is 6 * 7?")
    .tools(dynamicTool)
    .call()
    .content();

Advisors

Advisors implement cross-cutting concerns around chat calls — analogous to Spring AOP for LLM calls.

Request flow:
  ChatClient.call()
      │
      ▼
  [Advisor 1 — before]  ← e.g. inject memory, retrieve docs
      │
      ▼
  [Advisor 2 — before]  ← e.g. content moderation check
      │
      ▼
  ChatModel.call()      ← actual LLM call
      │
      ▼
  [Advisor 2 — after]   ← e.g. save response to memory
      │
      ▼
  [Advisor 1 — after]   ← e.g. log tokens used
      │
      ▼
  ChatClient response

MessageChatMemoryAdvisor — Conversation History

import org.springframework.ai.chat.client.advisor.MessageChatMemoryAdvisor;
import org.springframework.ai.chat.memory.InMemoryChatMemory;

// In-memory (development)
var memory = new InMemoryChatMemory();
var memoryAdvisor = new MessageChatMemoryAdvisor(memory);

// Per-conversation ID (isolates sessions)
String result = chatClient
    .prompt()
    .user("What did I say earlier?")
    .advisors(a -> a
        .advisor(memoryAdvisor)
        .param(CHAT_MEMORY_CONVERSATION_ID_KEY, conversationId)  // per-user/session
        .param(CHAT_MEMORY_RETRIEVE_SIZE_KEY, 10)                // last 10 exchanges
    )
    .call()
    .content();

QuestionAnswerAdvisor — RAG Auto-Injection

import org.springframework.ai.chat.client.advisor.QuestionAnswerAdvisor;
import org.springframework.ai.vectorstore.VectorStore;

@Service
public class RagService {

    private final ChatClient chatClient;
    private final VectorStore vectorStore;

    public String askWithContext(String question) {
        return chatClient
            .prompt()
            .user(question)
            .advisors(new QuestionAnswerAdvisor(
                vectorStore,
                SearchRequest.defaults().withTopK(5)  // retrieve 5 most relevant docs
            ))
            .call()
            .content();
        // Advisor automatically:
        // 1. Embeds the question
        // 2. Searches vectorStore
        // 3. Injects results into system prompt
        // 4. Calls model
    }
}

SafeGuardAdvisor — Content Moderation

import org.springframework.ai.chat.client.advisor.SafeGuardAdvisor;

// Blocks messages containing specified sensitive patterns
var safeGuard = new SafeGuardAdvisor(
    List.of("personal data", "credit card", "ssn")
);

chatClient.prompt()
    .user(userInput)
    .advisors(safeGuard)
    .call()
    .content();

Custom Advisor

import org.springframework.ai.chat.client.advisor.api.*;

public class RequestLoggingAdvisor implements CallAroundAdvisor {

    private static final Logger log = LoggerFactory.getLogger(RequestLoggingAdvisor.class);

    @Override
    public String getName() {
        return "RequestLoggingAdvisor";
    }

    @Override
    public int getOrder() {
        return Ordered.HIGHEST_PRECEDENCE;  // run first
    }

    @Override
    public AdvisedResponse aroundCall(AdvisedRequest request, CallAroundAdvisorChain chain) {
        log.info("LLM request: model={}, messages={}",
            request.chatOptions().getModel(),
            request.messages().size()
        );

        long start = System.currentTimeMillis();
        AdvisedResponse response = chain.nextAroundCall(request);  // call next advisor/model
        long duration = System.currentTimeMillis() - start;

        log.info("LLM response: duration={}ms, tokens={}",
            duration,
            response.response().getMetadata().getUsage().getTotalTokens()
        );
        return response;
    }
}

// Streaming advisor
public class StreamingLoggingAdvisor implements StreamAroundAdvisor {

    @Override
    public String getName() { return "StreamingLoggingAdvisor"; }

    @Override
    public int getOrder() { return 0; }

    @Override
    public Flux<AdvisedResponse> aroundStream(AdvisedRequest request, StreamAroundAdvisorChain chain) {
        log.info("Stream request started");
        return chain.nextAroundStream(request)
            .doOnComplete(() -> log.info("Stream completed"));
    }
}

RAG Pipeline

┌──────────────┐    ┌─────────────────────┐    ┌─────────────┐
│ DocumentReader│───▶│ DocumentTransformer │───▶│ VectorStore │
│ (PDF, Web,   │    │ (split, enrich,     │    │ (pgvector,  │
│  DB, FS)     │    │  clean)             │    │  Chroma...) │
└──────────────┘    └─────────────────────┘    └──────┬──────┘
                                                       │
                                          ┌────────────▼──────────┐
                              Question ──▶│ QuestionAnswerAdvisor │
                                          │ (embed + search +     │
                                          │  inject into prompt)  │
                                          └───────────────────────┘

Ingestion Pipeline

import org.springframework.ai.document.Document;
import org.springframework.ai.reader.pdf.PagePdfDocumentReader;
import org.springframework.ai.transformer.splitter.TokenTextSplitter;
import org.springframework.ai.vectorstore.VectorStore;

@Service
public class DocumentIngestionService {

    private final VectorStore vectorStore;

    public void ingestPdf(Resource pdfResource) {
        // 1. Read
        var reader = new PagePdfDocumentReader(pdfResource);
        List<Document> pages = reader.get();

        // 2. Split into chunks
        var splitter = new TokenTextSplitter(
            512,   // chunk size (tokens)
            128,   // overlap
            5,     // min chunk size
            10_000,// max chunk size
            true   // keep separator
        );
        List<Document> chunks = splitter.apply(pages);

        // 3. Enrich with metadata
        chunks.forEach(doc -> doc.getMetadata().put("source", pdfResource.getFilename()));

        // 4. Embed and store
        vectorStore.add(chunks);  // auto-embeds using configured EmbeddingModel
    }

    public void ingestWebPage(String url) {
        var reader = new WebPageDocumentReader(url);
        var splitter = new TokenTextSplitter();
        vectorStore.add(splitter.apply(reader.get()));
    }
}

Query Pipeline

@Service
public class RagQueryService {

    private final ChatClient chatClient;
    private final VectorStore vectorStore;

    public String query(String question, String conversationId) {
        return chatClient
            .prompt()
            .system("""
                You are a knowledgeable assistant.
                Answer ONLY using the provided context.
                If the answer is not in the context, say 'I don't have that information.'
                """)
            .user(question)
            .advisors(
                new QuestionAnswerAdvisor(
                    vectorStore,
                    SearchRequest.defaults()
                        .withTopK(5)
                        .withSimilarityThreshold(0.7)  // filter low-relevance docs
                ),
                new MessageChatMemoryAdvisor(chatMemory)
            )
            .advisors(a -> a.param(CHAT_MEMORY_CONVERSATION_ID_KEY, conversationId))
            .call()
            .content();
    }
}

Supported Vector Stores

// SimpleVectorStore for tests / local development
@Bean
public VectorStore vectorStore(EmbeddingModel embeddingModel) {
    return new SimpleVectorStore(embeddingModel);
}

Embedding

import org.springframework.ai.embedding.EmbeddingModel;
import org.springframework.ai.embedding.EmbeddingRequest;
import org.springframework.ai.embedding.EmbeddingOptions;

@Service
public class EmbeddingService {

    private final EmbeddingModel embeddingModel;

    // Single text
    public float[] embed(String text) {
        return embeddingModel.embed(text);
    }

    // Batch
    public List<float[]> embedBatch(List<String> texts) {
        EmbeddingResponse response = embeddingModel.embedForResponse(texts);
        return response.getResults()
            .stream()
            .map(r -> r.getOutput())
            .toList();
    }

    // Cosine similarity (manual)
    public double cosineSimilarity(float[] a, float[] b) {
        double dot = 0, normA = 0, normB = 0;
        for (int i = 0; i < a.length; i++) {
            dot += a[i] * b[i];
            normA += a[i] * a[i];
            normB += b[i] * b[i];
        }
        return dot / (Math.sqrt(normA) * Math.sqrt(normB));
    }
}

Multimodal

import org.springframework.ai.chat.messages.UserMessage;
import org.springframework.ai.chat.messages.Media;
import org.springframework.util.MimeTypeUtils;

@Service
public class VisionService {

    private final ChatClient chatClient;

    public String describeImage(Resource imageResource) {
        return chatClient
            .prompt()
            .user(u -> u
                .text("Describe what you see in this image in detail.")
                .media(MimeTypeUtils.IMAGE_JPEG, imageResource)
            )
            .call()
            .content();
    }

    public String analyzeReceipt(Resource receiptImage) {
        return chatClient
            .prompt()
            .user(u -> u
                .text("Extract: merchant name, total amount, date, and line items.")
                .media(MimeTypeUtils.IMAGE_PNG, receiptImage)
            )
            .call()
            .entity(Receipt.class);  // structured output from image
    }
}

Prompt Templates

import org.springframework.ai.chat.prompt.PromptTemplate;

@Service
public class TemplateService {

    // Inline template
    public String summarize(String text, String style) {
        var template = new PromptTemplate("""
            Summarize the following text in a {style} style.
            
            Text: {text}
            
            Summary:
            """);
        Prompt prompt = template.create(Map.of("style", style, "text", text));
        return chatModel.call(prompt).getResult().getOutput().getContent();
    }
}

// Load template from classpath resource
@Value("classpath:/prompts/summarize.st")
private Resource summarizeTemplate;

public String summarizeFromFile(String text) {
    var template = new PromptTemplate(summarizeTemplate);
    return chatModel.call(template.create(Map.of("text", text)))
        .getResult().getOutput().getContent();
}

# src/main/resources/prompts/summarize.st
You are an expert summarizer.

Summarize the following text concisely:

{text}

Key points:

Observability

# application.yml — enable Micrometer tracing
management:
  tracing:
    enabled: true
    sampling:
      probability: 1.0  # 100% in dev, lower in prod
  endpoints:
    web:
      exposure:
        include: health,metrics,prometheus

// Spring AI auto-instruments all ChatModel and EmbeddingModel calls:
// - spring.ai.chat.client.operation.seconds (latency histogram)
// - spring.ai.chat.client.token.usage (prompt + completion tokens)
// - spring.ai.embedding.operation.seconds

// Access via Micrometer / Actuator
// Traces appear in Zipkin, Jaeger, or any OTel-compatible backend

Testing

import org.springframework.ai.chat.model.ChatResponse;
import org.springframework.ai.chat.model.Generation;
import org.springframework.ai.chat.messages.AssistantMessage;

// Unit test with MockChatModel
class AssistantServiceTest {

    @Test
    void askReturnsMockedResponse() {
        // Arrange
        var mockResponse = new ChatResponse(
            List.of(new Generation(new AssistantMessage("Paris")))
        );
        ChatModel mockModel = prompt -> mockResponse;
        var service = new AssistantService(ChatClient.create(mockModel));

        // Act
        String result = service.ask("What is the capital of France?");

        // Assert
        assertThat(result).isEqualTo("Paris");
    }
}

// Integration test with real model (requires API key in test env)
@SpringBootTest
class RagIntegrationTest {

    @Autowired
    private RagQueryService ragService;

    @Autowired
    private DocumentIngestionService ingestionService;

    @Test
    void queryReturnRelevantAnswer() {
        // Ingest test document
        ingestionService.ingestPdf(new ClassPathResource("test-data/spring-ai-docs.pdf"));

        // Query
        String answer = ragService.query("What is a ChatClient?", "test-session");

        assertThat(answer).isNotBlank();
        assertThat(answer.toLowerCase()).contains("chatclient");
    }
}

// Test with AutoConfigureMockMvc for streaming endpoints
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@AutoConfigureWebTestClient
class StreamingControllerTest {

    @Autowired
    private WebTestClient webTestClient;

    @Test
    void streamChatReturnsFluxOfStrings() {
        webTestClient.get()
            .uri("/chat/stream?question=Hello")
            .accept(MediaType.TEXT_EVENT_STREAM)
            .exchange()
            .expectStatus().isOk()
            .expectBodyList(String.class)
            .hasSize(GreaterThan(0));
    }
}

Model Abstraction — Swap Providers Without Code Changes

Application code uses ChatClient (provider-agnostic)
        │
        ▼
┌───────────────────────────────────┐
│          ChatClient               │
│  (same API regardless of model)   │
└───────────────┬───────────────────┘
                │  delegates to
        ┌───────▼───────┐
        │   ChatModel   │  ← interface
        └───────────────┘
                │
    ┌───────────┼────────────┐
    │           │            │
┌───▼───┐ ┌───▼───┐ ┌──────▼──────┐
│OpenAI │ │Claude │ │  Ollama     │
│ChatM..│ │ChatM..│ │  ChatModel  │
└───────┘ └───────┘ └─────────────┘

Switch via application.yml only — zero code change:
  spring.ai.anthropic.api-key + model → uses Claude
  spring.ai.openai.api-key + model    → uses OpenAI
  spring.ai.ollama.base-url + model   → uses Ollama

Supported Providers

When to Use Spring AI vs Raw SDK

Use Spring AI when:
  ✓ Spring Boot application (natural fit)
  ✓ Need to swap LLM providers without code change
  ✓ Building RAG: advisors handle retrieval automatically
  ✓ Want Micrometer observability out of the box
  ✓ Conversation memory managed declaratively via advisors
  ✓ Streaming via reactive Flux integrated in WebFlux stack

Use raw SDK (openai-java, anthropic-java) when:
  ✓ Non-Spring application (Jakarta EE, Quarkus, plain Java)
  ✓ Need deep provider-specific features (realtime API, extended thinking)
  ✓ Minimal dependency footprint required
  ✓ Ultra-low latency with no framework overhead

Anti-Patterns

1. Creating ChatClient on Every Request

// BAD: expensive, loses advisor configuration
public String ask(String question) {
    return ChatClient.create(chatModel).prompt().user(question).call().content();
}

// GOOD: inject a pre-configured ChatClient bean
@Service
public class AssistantService {
    private final ChatClient chatClient;  // injected once
    public AssistantService(ChatClient chatClient) { this.chatClient = chatClient; }
}

2. Using SimpleVectorStore in Production

// BAD: lost on restart, not thread-safe under load
@Bean
public VectorStore vectorStore(EmbeddingModel em) {
    return new SimpleVectorStore(em);  // in-memory, ephemeral
}

// GOOD: use pgvector or Redis in production
// (configured via spring.ai.vectorstore.pgvector.*)

3. Ignoring Similarity Threshold in RAG

// BAD: returns irrelevant docs when no good match exists
new QuestionAnswerAdvisor(vectorStore, SearchRequest.defaults().withTopK(5))

// GOOD: filter below relevance threshold
new QuestionAnswerAdvisor(
    vectorStore,
    SearchRequest.defaults()
        .withTopK(5)
        .withSimilarityThreshold(0.70)  // discard docs below 70% similarity
)

4. Blocking in Reactive Context

// BAD: blocks the Reactor thread in a WebFlux app
@GetMapping("/chat")
public Mono<String> chat(String q) {
    String result = chatClient.prompt().user(q).call().content();  // blocking!
    return Mono.just(result);
}

// GOOD: use streaming or wrap in subscribeOn
@GetMapping(value = "/chat", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<String> chat(String q) {
    return chatClient.prompt().user(q).stream().content();  // non-blocking
}

5. Hardcoding Model Names

// BAD: locked to one model, hard to change
ChatOptions opts = OpenAiChatOptions.builder().withModel("gpt-4o").build();

// GOOD: externalize to application.yml
// spring.ai.openai.chat.options.model=gpt-4o
// Override per-environment without code change

6. Not Handling Tool Call Failures

// BAD: tool exception propagates as 500
@Bean
@Description("Fetch stock price")
public Function<StockRequest, StockResponse> getStockPrice() {
    return request -> stockApi.getPrice(request.symbol());  // can throw
}

// GOOD: catch and return structured error
@Bean
@Description("Fetch stock price")
public Function<StockRequest, StockResponse> getStockPrice() {
    return request -> {
        try {
            return stockApi.getPrice(request.symbol());
        } catch (StockNotFoundException e) {
            return new StockResponse(request.symbol(), null, "Symbol not found");
        }
    };
}

Production Checklist

• Inject ChatClient as a Spring bean — configure defaults once via ChatClient.builder()
• Externalize all model config to application.yml (model name, temperature, max-tokens)
• Use pgvector or Redis vector store — never SimpleVectorStore in production
• Set similarityThreshold on SearchRequest to avoid injecting irrelevant context
• Add MessageChatMemoryAdvisor with a unique conversationId per user session
• Use stream().content() in WebFlux endpoints — never block a Reactor thread
• Register Micrometer metrics endpoint and alert on p99 latency and token usage
• Wrap tool/function implementations in try-catch and return structured error responses
• Write unit tests with MockChatModel and integration tests with @SpringBootTest
• Tune TokenTextSplitter chunk size for your domain — test retrieval recall before ship
• Use SafeGuardAdvisor for any user-facing chat that accepts free-form input
• Pin spring-ai-bom version and test upgrades in staging — Spring AI evolves fast