Spring AI Providers

Spring AI - Providers & Cloud Integrations

Description

Complete guide for integrating Spring AI with multiple cloud and AI service providers. Covers configuration, authentication, model selection, and provider-specific features.

When to Use

• Configuring AI service providers
• Switching between providers
• Multi-provider architectures
• Regional deployment
• Cost optimization
• Feature-specific provider selection
• Fallback strategies
• Hybrid cloud setups

Topics Covered

1. Major Providers

OpenAI

• Models: GPT-4o, GPT-4, GPT-3.5-turbo, o1, o1-mini
• Features: Vision, function calling, structured output
• Auth: API key
• Endpoints: Global

Azure OpenAI

• Integration: Azure cloud-native
• Deployment: Region-specific
• Models: OpenAI models on Azure infrastructure
• Auth: Azure credentials or API key
• Features: Private networking, Enterprise SLA

Anthropic Claude

• Models: Claude 3 Opus, Sonnet, Haiku, Claude Instant
• Features: Vision, extended context, tool use
• Auth: API key
• Strengths: Reasoning, safety

Google Vertex AI / Gemini

• Models: Gemini Pro, Gemini Pro Vision, PaLM
• Features: Multimodal, function calling
• Auth: Google Cloud credentials
• Integration: GCP-native

Amazon Bedrock

• Models: Claude, Titan, Llama, Mistral, Cohere
• Features: Unified API for multiple models
• Auth: AWS credentials
• Integration: AWS ecosystem

Ollama (Local)

• Setup: Run locally
• Models: Open-source models
• Features: No API calls, privacy
• Use cases: Development, edge deployment

Other Providers

• Mistral AI
• HuggingFace
• Grok (X/Musk)
• Perplexity
• DeepSeek
• Together AI

2. Authentication Patterns

API Key Authentication

spring.ai.openai.api-key=${OPENAI_API_KEY}
spring.ai.anthropic.api-key=${ANTHROPIC_API_KEY}

Cloud Credentials

# Azure
spring.cloud.azure.credential.managed-identity-enabled=true

# GCP
spring.ai.vertex.project-id=${GCP_PROJECT}
spring.ai.vertex.location=us-central1

AWS Credentials

spring.ai.bedrock.region=us-east-1

3. Provider-Specific Features

• OpenAI: Vision, functions, structured output, fine-tuning
• Claude: Extended context (200K), tool use, vision
• Gemini: Multimodal, function calling, embeddings
• Bedrock: Unified API, cross-provider fallback
• Ollama: Full local control, privacy

4. Model Selection Strategy

• Use case fit: Choose by capability
• Cost optimization: Compare pricing
• Latency requirements: Regional placement
• Feature availability: Check provider support
• Compliance: Data residency, regulations

5. Fallback & Failover

• Primary provider failure handling
• Secondary provider activation
• Graceful degradation
• Circuit breaker patterns
• Health checks

Code Patterns

OpenAI Configuration

@Configuration
public class OpenAiConfig {
    @Bean
    public ChatModel openAiChatModel(OpenAiApi openAiApi) {
        return new OpenAiChatModel(
            openAiApi,
            OpenAiChatOptions.builder()
                .withModel("gpt-4o")
                .withTemperature(0.7f)
                .build()
        );
    }

    @Bean
    public EmbeddingModel embeddingModel(OpenAiApi openAiApi) {
        return new OpenAiEmbeddingModel(
            openAiApi,
            OpenAiEmbeddingOptions.builder()
                .withModel("text-embedding-3-small")
                .build()
        );
    }
}

Azure OpenAI Configuration

@Configuration
public class AzureOpenAiConfig {
    @Bean
    public ChatModel azureOpenAiChatModel(OpenAiApi openAiApi) {
        return new OpenAiChatModel(
            openAiApi,
            OpenAiChatOptions.builder()
                .withModel("deployment-name")
                .withDeploymentName("gpt-4-deployment")
                .build()
        );
    }
}

Anthropic Claude Configuration

@Configuration
public class ClaudeConfig {
    @Bean
    public ChatModel claudeChatModel(AnthropicApi anthropicApi) {
        return new AnthropicChatModel(
            anthropicApi,
            AnthropicChatOptions.builder()
                .withModel("claude-3-opus-20240229")
                .withMaxTokens(2048)
                .build()
        );
    }
}

Google Vertex AI Configuration

@Configuration
public class VertexAiConfig {
    @Bean
    public ChatModel vertexChatModel(VertexAiApi vertexAiApi) {
        return new VertexAiChatModel(
            vertexAiApi,
            VertexAiChatOptions.builder()
                .withModel("gemini-pro")
                .build()
        );
    }
}

AWS Bedrock Configuration

@Configuration
public class BedrockConfig {
    @Bean
    public ChatModel bedrockChatModel(BedrockApi bedrockApi) {
        return new BedrockChatModel(
            bedrockApi,
            BedrockChatOptions.builder()
                .withModel("anthropic.claude-3-sonnet")
                .build()
        );
    }
}

Ollama Local Configuration

@Configuration
public class OllamaConfig {
    @Bean
    public ChatModel ollamaChatModel(OllamaApi ollamaApi) {
        return new OllamaChatModel(
            ollamaApi,
            OllamaOptions.builder()
                .withModel("mistral")
                .withTemperature(0.7f)
                .build()
        );
    }
}

Multi-Provider with Fallback

@Service
public class MultiProviderChatService {
    private final ChatModel primaryModel;
    private final ChatModel fallbackModel;

    @Autowired
    public MultiProviderChatService(
            @Qualifier("openai") ChatModel primaryModel,
            @Qualifier("claude") ChatModel fallbackModel) {
        this.primaryModel = primaryModel;
        this.fallbackModel = fallbackModel;
    }

    public String chat(String message) {
        try {
            return primaryModel.call(
                new Prompt(new UserMessage(message))
            ).getResult().getOutput().getContent();
        } catch (Exception e) {
            logger.warn("Primary provider failed, using fallback", e);
            return fallbackModel.call(
                new Prompt(new UserMessage(message))
            ).getResult().getOutput().getContent();
        }
    }
}

Provider-Specific Feature Detection

@Service
public class ProviderFeatureService {
    private final ChatModel chatModel;

    public String chatWithFeatures(String message) {
        // Check if provider supports function calling
        if (supportsFunctionCalling()) {
            return chatWithTools(message);
        }

        // Check if provider supports vision
        if (supportsVision()) {
            return chatWithVision(message);
        }

        // Fallback to basic chat
        return basicChat(message);
    }

    private boolean supportsFunctionCalling() {
        // Detect from provider type
        return chatModel instanceof OpenAiChatModel ||
               chatModel instanceof AnthropicChatModel;
    }
}

Cost-Optimized Provider Selection

@Service
public class CostOptimizedService {
    private final Map<String, ChatModel> providers;
    private final Map<String, Double> costs;

    public String chatOptimized(String message) {
        // Use cheapest provider for simple queries
        ChatModel selected = selectByQuery(message);
        return performChat(selected, message);
    }

    private ChatModel selectByQuery(String message) {
        if (isSimpleQuery(message)) {
            return providers.get("gpt-3.5-turbo");  // Cheapest
        }

        if (requiresReasoning(message)) {
            return providers.get("gpt-4o");  // Best reasoning
        }

        if (requiresVision(message)) {
            return providers.get("claude-vision");
        }

        return providers.get("default");
    }
}

Regional Provider Selection

@Service
public class RegionalProviderService {
    private final ChatModel usModel;
    private final ChatModel euModel;
    private final ChatModel apModel;

    public String chatWithRegion(String message, String region) {
        ChatModel model = switch(region) {
            case "us" -> usModel;
            case "eu" -> euModel;
            case "ap" -> apModel;
            default -> usModel;
        };

        return performChat(model, message);
    }
}

Configuration Properties

OpenAI

spring.ai.openai.api-key=${OPENAI_API_KEY}
spring.ai.openai.api-url=https://api.openai.com/v1
spring.ai.openai.chat.options.model=gpt-4o
spring.ai.openai.chat.options.temperature=0.7

Azure OpenAI

spring.ai.azure.openai.api-key=${AZURE_OPENAI_API_KEY}
spring.ai.azure.openai.endpoint=${AZURE_OPENAI_ENDPOINT}
spring.ai.azure.openai.deployment-name=gpt-4-deployment

Claude

spring.ai.anthropic.api-key=${ANTHROPIC_API_KEY}
spring.ai.anthropic.chat.options.model=claude-3-opus-20240229

Google Vertex AI

spring.ai.vertex.project-id=${GCP_PROJECT_ID}
spring.ai.vertex.location=us-central1

AWS Bedrock

spring.ai.bedrock.region=us-east-1
spring.ai.bedrock.model=anthropic.claude-3-sonnet

Ollama

spring.ai.ollama.base-url=http://localhost:11434
spring.ai.ollama.chat.options.model=mistral

Multi-Provider Configuration

spring:
  ai:
    openai:
      api-key: ${OPENAI_API_KEY}
    anthropic:
      api-key: ${ANTHROPIC_API_KEY}
    bedrock:
      region: us-east-1

Best Practices

• Use environment variables for secrets
• Implement provider abstraction
• Handle provider-specific errors
• Monitor provider availability
• Implement fallback strategies
• Test with multiple providers
• Consider cost implications
• Respect rate limits
• Use regional endpoints when possible
• Implement circuit breakers

Related Skills

• chat-models/SKILL.md - Chat implementation
• observability/SKILL.md - Monitoring
• error-handling/SKILL.md - Error management

References

• Official docs: https://docs.spring.io/spring-ai
• Provider documentation:
  • OpenAI: https://platform.openai.com/docs
  • Claude: https://docs.anthropic.com
  • Vertex AI: https://cloud.google.com/vertex-ai
  • Bedrock: https://docs.aws.amazon.com/bedrock