# Spring AI Chat Models

> A comprehensive guide for implementing and configuring Chat Models and ChatClient APIs in Spring AI, covering all supported LLM providers.

- Skill: `mat-garcia/spring-ai-chat-models` (Agent Skill)
- Install (CLI): `npx skillmds@latest add mat-garcia/spring-ai-chat-models`
- Raw SKILL.md: https://api.skillmd.com/api/skills/mat-garcia/spring-ai-chat-models/raw
- Safety review: pending (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- License: Complete terms in LICENSE.txt
- Author: mat-garcia (https://skillmd.com/u/mat-garcia)
- Updated: 2026-09-22
- Page: https://skillmd.com/skills/mat-garcia/spring-ai-chat-models

---


# Spring AI - Chat Models & ChatClient API

## Description

Comprehensive guide for implementing and configuring Chat Models and ChatClient APIs in Spring AI. Covers all supported LLM providers (OpenAI, Azure OpenAI, Bedrock, Ollama, etc.), configuration options, response handling, and streaming patterns.

## When to Use

- Setting up chat interactions with language models
- Configuring ChatModel or ChatClient beans
- Implementing chat completions with specific providers
- Handling chat options and model parameters
- Working with structured outputs and tool calling
- Streaming chat responses
- Integration with Spring Boot applications

## Topics Covered

### 1. Supported Chat Providers

- **OpenAI**: GPT-4o, GPT-4, GPT-3.5-turbo, o1-preview, o1-mini
- **Azure OpenAI**: Full integration with Azure deployments
- **Anthropic Claude**: Claude 3 Opus, Sonnet, Haiku
- **Google Gemini**: Gemini Pro, Gemini 2.0
- **Amazon Bedrock**: Multiple Titan, Claude, and Llama models
- **Ollama**: Local model inference
- **Mistral AI**: Open models
- **HuggingFace**: Community models
- **Other providers**: Grok, Perplexity, DeepSeek

### 2. ChatModel API Core

- **Request**: Prompt + ChatOptions
- **Response**: ChatResponse (generation + metadata)
- **Streaming**: Reactive Flux<ChatResponse> for real-time output
- **Call Metrics**: Token usage, cost, latency tracking
- **Non-streaming vs Streaming**: Performance considerations

### 3. ChatClient Fluent API

- High-level abstraction over ChatModel
- Method chaining for configuration
- `.user()`, `.system()`, `.messages()`
- `.call()`, `.stream()` for execution
- `.content()`, `.entity()` for results

### 4. Chat Configuration

- Model/deployment selection
- Temperature, max_tokens, frequency_penalty
- Top_p, top_k sampling parameters
- Stop sequences
- Function calling mode (auto, none, required)

### 5. Response Handling

- Generation with usage information
- Tool calls in response
- Finish reason interpretation (STOP, LENGTH, TOOL_CALLS, etc.)
- Metadata extraction (model info, timestamps)

### 6. Structured Output

- Type-safe JSON responses
- `StructuredOutputConverter` (JSON Schema mode)
- `Jackson` and `Gson` support
- Automatic serialization/deserialization

### 7. Tool Calling Integration

- Automatic tool detection and binding
- Request model vs response format handling
- Exception handling and retries
- OpenAI, Anthropic, Google, Bedrock support

## Code Patterns

### Basic ChatModel Usage

```java
@Configuration
public class ChatConfig {
    @Bean
    public ChatModel chatModel(OpenAiApi openAiApi) {
        return new OpenAiChatModel(openAiApi,
            OpenAiChatOptions.builder()
                .withModel("gpt-4o")
                .withTemperature(0.7f)
                .build());
    }
}
```

### ChatClient with Streaming

```java
chatClient.prompt()
    .user("Question: " + userQuery)
    .stream()
    .content()
    .doOnNext(content -> logger.info("Token: {}", content))
    .blockLast();
```

### Structured Output

```java
var response = chatClient.prompt()
    .user("Generate a person")
    .call()
    .entity(Person.class);
```

## Configuration via Properties

```properties
spring.ai.openai.api-key=${OPENAI_API_KEY}
spring.ai.openai.chat.options.model=gpt-4o
spring.ai.openai.chat.options.temperature=0.7
spring.ai.openai.chat.options.max-tokens=1000
```

## Related Skills

- `embeddings/SKILL.md` - Vector embeddings
- `rag-retrieval/SKILL.md` - RAG patterns
- `tools-agents/SKILL.md` - Tool calling
- `advisors/SKILL.md` - Chain of responsibility patterns
- `structured-output/SKILL.md` - Type-safe responses

## References

- API: `/pages/api/chatmodel.adoc`, `/pages/api/chatclient.adoc`
- Providers: `/pages/api/bedrock.adoc`, OpenAI, Azure OpenAI docs
- Tools: `/pages/api/tools.adoc`, `/pages/api/tools-migration.adoc`
- Structured Output: `/pages/api/structured-output-converter.adoc`