Config Architecture

Configuration Architecture Guide

Overview

LiteLLM-RS loads a single YAML file into type-safe Rust models (GatewayConfig in src/config/models/gateway.rs and its children), substitutes ${VAR} / $VAR tokens from the process environment during load, fills missing fields from serde defaults, then validates the whole tree once at startup.

There is no hot reload. Configuration is read once at startup; src/utils/config/optimized.rs states this explicitly ("Hot reload is not supported"). Restart the process to pick up changes. AppState.config (src/server/state.rs) is atomically swappable by explicit code paths, but nothing watches the file.

Load Order

┌─────────────────────────────────────────────────────────────────┐
│  1. YAML file                                                   │
│     --config PATH (default: "config/gateway.yaml", src/main.rs) │
└─────────────────────────────────────────────────────────────────┘
                              ↓
┌─────────────────────────────────────────────────────────────────┐
│  2. Environment substitution (substitute_env_vars)              │
│     ${VAR} = required (hard error if unset)                     │
│     $VAR   = best-effort (left literal if unset)                │
└─────────────────────────────────────────────────────────────────┘
                              ↓
┌─────────────────────────────────────────────────────────────────┐
│  3. serde parse + defaults                                      │
│     #[serde(default ...)] fills omitted fields;                 │
│     deny_unknown_fields rejects keys on structs that opt in     │
└─────────────────────────────────────────────────────────────────┘
                              ↓
┌─────────────────────────────────────────────────────────────────┐
│  4. Validation (Validate trait)                                 │
│     runs inside Config::from_file / Config::from_env            │
│     failure aborts startup                                      │
└─────────────────────────────────────────────────────────────────┘

After loading, explicit --host / --port CLI values overwrite server.host / server.port, then the config is re-validated (load_config_with_overrides in src/main.rs).

Fallback rule: when the gateway starts without an explicit --config and the default file fails to load, run_server_with_default_config_overrides retries with Config::from_env(), which builds the config from LITELLM_* environment variables only (src/config/models/gateway.rs::GatewayConfig::from_env). An explicit --config path that fails is a hard error — no env fallback (load_explicit_config in src/server/builder.rs).

There is no search-path chain: no /etc/litellm/..., no gateway.yaml in cwd. The only default is the single constant DEFAULT_CONFIG_PATH = "config/gateway.yaml" relative to the process working directory (src/main.rs).

YAML Configuration Structure

The canonical example is config/gateway.yaml.example; unit tests assert it parses against the current schema. Top-level sections of GatewayConfig:

schema_version: "1.0"        # only "1.0" is accepted by validation

server:                      # host/port/workers/timeout/max_body_size/tls/cors/features/...
providers:                   # LIST of provider entries, not a map
  - name: "openai-primary"
    provider_type: "openai"
    api_key: "${OPENAI_API_KEY}"
    base_url: "https://api.openai.com/v1"
    models: ["gpt-4o"]
    priority: 0              # lower wins under priority_based routing

model_aliases:               # public alias -> canonical model or another alias
  production-chat: "gpt-4o"

router:                      # strategy, circuit_breaker, load_balancer
storage:                     # database, redis, vector_db
auth:                        # enable_jwt, enable_api_key, jwt_secret, rbac, allow_anonymous
monitoring:                  # metrics, tracing, health, callbacks
cache:                       # deterministic response cache (+ semantic_cache flag)
rate_limit:                  # token bucket strategy, redis_failure_mode
guardrails:                  # content safety; prompt-injection protection on by default
ip_access:                   # allowlist/blocklist; empty/default rules allow all
enterprise:                  # sso, audit_logging
pricing:                     # source, unpriced_model_policy

Unknown top-level fields are rejected at parse time (#[serde(deny_unknown_fields)] on GatewayConfig), as are unknown keys in the nested YAML models that carry the same attribute. This is not a universal serde guarantee: providers[].settings is intentionally open-ended, and some provider/callback backend payload structs do not deny unknown fields. There are no top-level logging, routing, or observability sections — logging lives under monitoring.logging, routing under router.

Type-Safe Configuration Models

Root Configuration

src/config/models/gateway.rs:

#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct GatewayConfig {
    pub schema_version: String,                    // default "1.0"
    pub server: ServerConfig,
    pub providers: Vec<ProviderConfig>,            // Vec, not a map
    pub model_aliases: HashMap<String, String>,    // default empty
    pub router: GatewayRouterConfig,
    pub storage: StorageConfig,
    pub auth: AuthConfig,
    pub monitoring: MonitoringConfig,
    pub cache: CacheConfig,                        // default
    pub rate_limit: RateLimitConfig,               // default
    pub guardrails: GuardrailConfig,               // default = enabled
    pub ip_access: IpAccessConfig,                 // default
    pub enterprise: EnterpriseConfig,              // default
    pub pricing: GatewayPricingConfig,             // default
}

Server Configuration

src/config/models/server.rs:

#[serde(deny_unknown_fields)]
pub struct ServerConfig {
    pub host: String,                      // default "0.0.0.0"
    pub port: u16,                         // default 8000
    pub workers: Option<usize>,            // None -> CPU count at runtime
    pub max_connections: Option<usize>,
    pub timeout: u64,                      // seconds, default 30
    pub max_body_size: usize,              // bytes, default 10 MiB
    pub dev_mode: bool,                    // default false
    pub tls: Option<TlsConfig>,
    pub cors: CorsConfig,
    pub features: Vec<String>,
    pub trusted_proxies: Vec<String>,      // X-Forwarded-For trust list
    pub stream_idle_timeout: u64,          // SSE idle timeout, default 300s; 0 disables
}

Shared defaults live in src/config/models/mod.rs: default_port() returns 8000, default_timeout() returns 30, default_max_body_size() returns 10 * 1024 * 1024. There is no keep_alive, request_timeout, or string-form max_request_size field.

Provider Configuration

src/config/models/provider.rs — one entry per deployment, stored as Vec<ProviderConfig>:

#[serde(deny_unknown_fields)]
pub struct ProviderConfig {
    pub name: String,                      // must be unique across entries
    pub provider_type: String,             // e.g. "openai", "anthropic"
    pub api_key: String,                   // required unless registry skips key checks
    pub base_url: Option<String>,
    pub endpoint_access: ProviderEndpointAccess, // default PublicOnly; "private_network" opt-in
    pub api_version: Option<String>,
    pub organization: Option<String>,
    pub project: Option<String>,
    pub weight: f32,                       // default 1.0, valid range (0, 100]
    pub priority: u32,                     // default 0; lower wins in priority_based
    pub rpm: u32,                          // default 1000
    pub tpm: u32,                          // default 100_000
    pub max_concurrent_requests: u32,      // default 10
    pub timeout: u64,                      // seconds, default 30, max 300
    pub max_retries: u32,                  // default 3
    pub retry: RetryConfig,                // base_delay/max_delay/backoff_multiplier/jitter
    pub health_check: ProviderHealthCheckConfig,
    pub settings: HashMap<String, serde_json::Value>, // provider-specific keys
    pub models: Vec<String>,
    pub tags: Vec<String>,
    pub enabled: bool,                     // default true
}

This is the YAML-facing model. The separate ProviderConfig trait that provider implementations satisfy lives in src/core/traits/provider/config.rs — see reference/provider-config-errors.md.

Environment Variable Substitution

substitute_env_vars in src/config/mod.rs processes the raw file text before parsing:

• ${VAR_NAME} — substituted with the env value. If unset, loading fails: all missing braced variables are collected, deduplicated, sorted, and reported in one error (Missing environment variables referenced by config: A, B). Placeholders can never silently reach runtime as literal strings.
• $VAR_NAME — shell-style bare form, also substituted. If unset, the token is left literal so ordinary dollar-containing values pass through untouched.
• ${VAR:-default} is not supported and is silently dangerous: neither recognized pattern matches a token containing :-, so the whole string stays literal in the parsed value — no substitution and no error.
• Substitution is line-based and quote/comment aware: text after an unquoted # (a YAML comment) is never substituted, and values pulled from the environment are not re-expanded.
• Errors surface as GatewayError::Config (the runtime loader does not use ConfigError; see reference/provider-config-errors.md).

# Works
api_key: "${OPENAI_API_KEY}"        # fails startup if OPENAI_API_KEY is unset
url: "$REDIS_URL"                   # left literal if REDIS_URL is unset

# Does NOT work - no fallback syntax; url becomes the literal
# string "${REDIS_URL:-redis://localhost:6379}"
url: "${REDIS_URL:-redis://localhost:6379}"

Separately, when the implicit-default serve path falls back to env-only configuration, GatewayConfig::from_env reads dedicated LITELLM_* variables (LITELLM_HOST, LITELLM_PORT, LITELLM_DATABASE_URL, LITELLM_JWT_SECRET, LITELLM_PROVIDERS, ... — see the constants atop src/config/models/gateway.rs). This is independent of ${VAR} substitution inside YAML.

Configuration Loading

Actual chain, Config::from_file in src/config/mod.rs:

pub async fn from_file<P: AsRef<Path>>(path: P) -> Result<Self> {
    let content = tokio::fs::read_to_string(path).await
        .map_err(|e| GatewayError::Config(format!("Failed to read config file: {}", e)))?;
    let content = substitute_env_vars(&content)?;          // hard error on missing ${VAR}
    let gateway: GatewayConfig = serde_yml::from_str(&content)
        .map_err(|e| GatewayError::Config(format!("Failed to parse config: {}", e)))?;
    let config = Self { gateway };
    config.validate()?;                                    // Validate trait, single entry point
    Ok(config)
}

Entry points:

Secret redaction happens only on export: to_json / to_yaml replace known secrets (provider API keys, jwt_secret, HMAC secret, S3/vector/SSO credentials) with [REDACTED] before serializing.

References

• reference/validation.md — Validate trait, validation module layout, and per-section validation rules.
• reference/provider-config-errors.md — Provider configuration trait and ConfigError variants.
• reference/best-practices.md — Defaults, early validation, secret handling, type safety, and documentation practices.