Deploy this version

import Image from '@theme/IdealImage'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem';

Deploy this version

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
docker.litellm.ai/berriai/litellm:v1.80.11-stable

pip install litellm==1.80.11

Key Highlights

• Gemini 3 Flash Preview - Day 0 support for Google's Gemini 3 Flash Preview with reasoning capabilities
• Stability AI Image Generation - New provider for Stability AI image generation and editing
• LiteLLM Content Filter - Built-in guardrails for harmful content, bias, and PII detection with image support
• New Provider: Venice.ai - Support for Venice.ai API via providers.json
• Unified Skills API - Skills API works across Anthropic, Vertex, Azure, and Bedrock
• Azure Sentinel Logging - New logging integration for Azure Sentinel
• Guardrails Load Balancing - Load balance between multiple guardrail providers
• Email Budget Alerts - Send email notifications when budgets are reached
• Cloudzero Integration on UI - Setup your Cloudzero Integration Directly on the UI

Cloudzero Integration on UI

<Image img={require('../../img/ui_cloudzero.png')} style={{width: '100%', display: 'block', margin: '2rem auto'}} />

Users can now configure their Cloudzero Integration directly on the UI.

Performance: 50% Reduction in Memory Usage and Import Latency for the LiteLLM SDK

We've completely restructured litellm.__init__.py to defer heavy imports until they're actually needed, implementing lazy loading for 109 components.

This refactoring includes 41 provider config classes, 40 utility functions, cache implementations (Redis, DualCache, InMemoryCache), HTTP handlers, logging, types, and other heavy dependencies. Heavy libraries like tiktoken and boto3 are now loaded on-demand rather than eagerly at import time.

This makes LiteLLM especially beneficial for serverless functions, Lambda deployments, and containerized environments where cold start times and memory footprint matter.

New Providers and Endpoints

New Providers (5 new providers)

New LLM API Endpoints (2 new endpoints)

New Models / Updated Models

New Model Support (55+ new models)

Features

• Gemini
  • Add Gemini 3 Flash Preview day 0 support with reasoning - PR #18135
  • Support extra_headers in batch embeddings - PR #18004
  • Propagate token usage when generating images - PR #17987
  • Use JSON instead of form-data for image edit requests - PR #18012
  • Fix web search requests count - PR #17921
• Anthropic
  • Use dynamic max_tokens based on model - PR #17900
  • Fix claude-3-7-sonnet max_tokens to 64K default - PR #17979
  • Add OpenAI-compatible API with modify_params=True - PR #17106
• Vertex AI
  • Add Gemini 3 Flash Preview support - PR #18164
  • Add reasoning support for gemini-3-flash-preview - PR #18175
  • Fix image edit credential source - PR #18121
  • Pass credentials to PredictionServiceClient for custom endpoints - PR #17757
  • Fix multimodal embeddings for text + base64 image combinations - PR #18172
  • Add OCR support for DeepSeek model - PR #17971
• Azure AI
  • Add Azure Cohere 4 reranking models - PR #17961
  • Add Azure DeepSeek V3.2 versions - PR #18019
  • Return AzureAnthropicConfig for Claude models in get_provider_chat_config - PR #18086
• Fireworks AI
  • Add reasoning param support for Fireworks AI models - PR #17967
• Bedrock
  • Add Qwen 2 and Qwen 3 to get_bedrock_model_id - PR #18100
  • Remove ttl field when routing to bedrock - PR #18049
  • Add Bedrock Stability image edit models - PR #18254
• Perplexity
  • Use API-provided cost instead of manual calculation - PR #17887
• OpenAI
  • Add diarize model for audio transcription - PR #18117
  • Add gpt-image-1.5-2025-12-16 in model cost map - PR #18107
  • Fix cost calculation of gpt-image-1 model - PR #17966
• GitHub Copilot
  • Add github_copilot model info - PR #17858
• Custom LLM
  • Add image_edit and aimage_edit support - PR #17999

Bug Fixes

• Gemini
  • Fix pricing for Gemini 3 Flash on Vertex AI - PR #18202
  • Add output_cost_per_image_token for gemini-2.5-flash-image models - PR #18156
  • Fix properties should be non-empty for OBJECT type - PR #18237
• Qwen
  • Add qwen3-embedding-8b input per token price - PR #18018
• General
  • Fix image URL handling - PR #18139
  • Support Signed URLs with Query Parameters in Image Processing - PR #17976
  • Add none to encoding_format instead of omitting it - PR #18042

LLM API Endpoints

Features

• Responses API
  • Add provider specific tools support - PR #17980
  • Add custom headers support - PR #18036
  • Fix tool calls transformation in completion bridge - PR #18226
  • Use list format with input_text for tool results - PR #18257
  • Add cost tracking in background mode - PR #18236
  • Fix Claude code responses API bridge errors - PR #18194
• Chat Completions API
  • Add support for agent skills - PR #18031
• Skills API
  • Unified Skills API works across Anthropic, Vertex, Azure, Bedrock - PR #18232
• Search API
  • Add new RAG Search API with rerankers - PR #18217
• Interactions API
  • Add Google Interactions API on SDK and AI Gateway - PR #18079, PR #18081
• Image Edit API
  • Add drop_params support and fix Vertex AI config - PR #18077
• General
  • Skip adding beta headers for Vertex AI as it is not supported - PR #18037
  • Fix managed files endpoint - PR #18046
  • Allow base_model for non-Azure providers in proxy - PR #18038

Bugs

• General
  • Fix basemodel import in guardrail translation - PR #17977
  • Fix No module named 'fastapi' error - PR #18239

Management Endpoints / UI

Features

• Virtual Keys
  • Add master key rotation for credentials table - PR #17952
  • Fix tag management to preserve encrypted fields in litellm_params - PR #17484
  • Fix key delete and regenerate permissions - PR #18214
• Models + Endpoints
  • Add Models Conditional Rendering in UI - PR #18071
  • Add Health Check Model for Wildcard Model in UI - PR #18269
  • Auto Resolve Vector Store Embedding Model Config - PR #18167
• Vector Stores
  • Add Milvus Vector Store UI support - PR #18030
  • Persist Vector Store Settings in Team Update - PR #18274
• Logs & Spend
  • Add LiteLLM Overhead to Logs - PR #18033
  • Show LiteLLM Overhead in Logs UI - PR #18034
  • Resolve Team ID to Team Alias in Usage Page - PR #18275
  • Fix Usage Page Top Key View Button Visibility - PR #18203
• SSO & Health
  • Add SSO Readiness Health Check - PR #18078
  • Fix /health/test_connection to resolve env variables like /chat/completions - PR #17752
• CloudZero
  • Add CloudZero Cost Tracking UI - PR #18163
  • Add Delete CloudZero Settings Route and UI - PR #18168, PR #18170
• General
  • Update UI path handling for non-root Docker - PR #17989

Bugs

• UI Fixes
  • Fix Login Page Failed To Parse JSON Error - PR #18159
  • Fix new user route user_id collision handling - PR #17559
  • Fix Callback Environment Variables Casing - PR #17912

AI Integrations

Logging

• Azure Sentinel
  • Add new Azure Sentinel Logger integration - PR #18146
• Prometheus
  • Add extraction of top level metadata for custom labels - PR #18087
• Langfuse
  • Fix not working log_failure_event - PR #18234
• Arize Phoenix
  • Fix nested spans - PR #18102
• General
  • Change extra_headers to additional_headers - PR #17950

Guardrails

• LiteLLM Content Filter
  • Add built-in guardrails for harmful content, bias, etc. - PR #18029
  • Add support for running content filters on images - PR #18044
  • Add support for Brazil PII field - PR #18076
  • Add configurable guardrail options for content filtering - PR #18007
• Guardrails API
  • Support LLM tool call response checks on /chat/completions, /v1/responses, /v1/messages - PR #17619
  • Add guardrails load balancing - PR #18181
  • Fix guardrails for passthrough endpoint - PR #18109
  • Add headers to metadata for guardrails on pass-through endpoints - PR #17992
  • Various fixes for guardrail on OpenRouter models - PR #18085
• Lakera
  • Add monitor mode for Lakera - PR #18084
• Pillar Security
  • Add masking support and MCP call support - PR #17959
• Bedrock Guardrails
  • Add support for Bedrock image guardrails - PR #18115
  • Guardrails block action takes precedence over masking - PR #17968

Secret Managers

• HashiCorp Vault
  • Add documentation for configurable Vault mount - PR #18082
  • Add per-team Vault configuration - PR #18150
• UI
  • Add secret manager settings controls to team management UI - PR #18149

Spend Tracking, Budgets and Rate Limiting

• Email Budget Alerts - Send email notifications when budgets are reached - PR #17995

MCP Gateway

• Auth Header Propagation - Add MCP auth header propagation - PR #17963
• Fix deepcopy error - Fix MCP tool call deepcopy error when processing requests - PR #18010
• Fix list tool - Fix MCP list_tools not working without database connection - PR #18161

Agent Gateway (A2A)

• New Provider: Agent Gateway - Add pydantic ai agents support - PR #18013
• VertexAI Agent Engine - Add Vertex AI Agent Engine provider - PR #18014
• Fix model extraction - Fix get_model_from_request() to extract model ID from Vertex AI passthrough URLs - PR #18097

Performance / Loadbalancing / Reliability improvements

• Lazy Imports - Use per-attribute lazy imports and extract shared constants - PR #17994
• Lazy Load HTTP Handlers - Lazy load http handlers - PR #17997
• Lazy Load Caches - Lazy load caches - PR #18001
• Lazy Load Types - Lazy load bedrock types, .types.utils, GuardrailItem - PR #18053, PR #18054, PR #18072
• Lazy Load Configs - Lazy load 41 configuration classes - PR #18267
• Lazy Load Client Decorators - Lazy load heavy client decorator imports - PR #18064
• Prisma Build Time - Download Prisma binaries at build time instead of runtime for security restricted environments - PR #17695
• Docker Alpine - Add libsndfile to Alpine image for ARM64 audio processing - PR #18092
• Security - Prevent LiteLLM API key leakage on /health endpoint failures - PR #18133

Documentation Updates

• SAP Docs - Update SAP documentation - PR #17974
• Pydantic AI Agents - Add docs on using pydantic ai agents with LiteLLM A2A gateway - PR #18026
• Vertex AI Agent Engine - Add Vertex AI Agent Engine documentation - PR #18027
• Router Order - Add router order parameter documentation - PR #18045
• Secret Manager Settings - Improve secret manager settings documentation - PR #18235
• Gemini 3 Flash - Add version requirement in Gemini 3 Flash blog - PR #18227
• README - Expand Responses API section and update endpoints - PR #17354
• Amazon Nova - Add Amazon Nova to sidebar and supported models - PR #18220
• Benchmarks - Add infrastructure recommendations to benchmarks documentation - PR #18264
• Broken Links - Fix broken link corrections - PR #18104
• README Fixes - Various README improvements - PR #18206

Infrastructure / CI/CD

• PR Templates - Add LiteLLM team PR template and CI/CD rules - PR #17983, PR #17985
• Issue Labeling - Improve issue labeling with component dropdown and more provider keywords - PR #17957
• PR Template Cleanup - Remove redundant fields from PR template - PR #17956
• Dependencies - Bump altcha-lib from 1.3.0 to 1.4.1 - PR #18017

New Contributors

• @dongbin-lunark made their first contribution in PR #17757
• @qdrddr made their first contribution in PR #18004
• @donicrosby made their first contribution in PR #17962
• @NicolaivdSmagt made their first contribution in PR #17992
• @Reapor-Yurnero made their first contribution in PR #18085
• @jk-f5 made their first contribution in PR #18086
• @castrapel made their first contribution in PR #18077
• @dtikhonov made their first contribution in PR #17484
• @opleonnn made their first contribution in PR #18175
• @eurogig made their first contribution in PR #18084

Full Changelog

View complete changelog on GitHub