Claude Code Litellm Gateway

Claude Code + LiteLLM Gateway

A battle-tested skill for wiring Claude Code to an existing LiteLLM gateway.

This is especially useful when:

• LiteLLM already works for another agent
• you want Claude Code to use a LiteLLM-routed model such as gemini-3.1-pro-preview
• you must not break an existing Hermes / OpenClaw / direct-Vertex setup
• Claude Code has conflicting user-level settings in ~/.claude/settings.json
• you want either:
  • a parallel wrapper such as claude-gemini, or
  • a main-command takeover so plain claude uses LiteLLM by default

What this skill solves

Claude Code expects Anthropic-style configuration, while LiteLLM may be routing to Anthropic, Vertex AI, OpenAI, or something else underneath.

The core trick is:

• Claude Code speaks to LiteLLM using Anthropic-style environment variables
• LiteLLM translates the request to the real upstream model
• Claude Code still hits Anthropic-style endpoints such as /v1/messages
• LiteLLM translates both request and response formats

For Toby's working setup, Claude Code was successfully routed to:

gemini-3.1-pro-preview

through a local LiteLLM gateway at:

http://127.0.0.1:4000

backed by:

/Users/toby/TobyLab/litellm-vertex-proxy

This was verified both as:

• a dedicated wrapper command: claude-gemini
• a direct main claude configuration in ~/.claude/settings.json

Official-docs-aligned mental model

This skill matches the public Claude Code + LiteLLM pattern:

• LiteLLM exposes a gateway with virtual model names in config.yaml
• Claude Code is pointed at that gateway with:
  • ANTHROPIC_BASE_URL
  • ANTHROPIC_AUTH_TOKEN
  • optionally ANTHROPIC_MODEL
• Claude Code sends Anthropic Messages API requests
• LiteLLM translates those requests to the actual upstream provider/model

In other words, this is not teaching Claude Code to natively understand Gemini. It is teaching Claude Code to talk to LiteLLM, while LiteLLM handles Gemini.

Quick start

1. Confirm LiteLLM itself is healthy.
2. Confirm LiteLLM accepts Anthropic-format /v1/messages requests.
3. Inspect ~/.claude/settings.json for conflicting direct-Vertex or other user-level Claude settings.
4. Choose an integration mode:
   • Mode A — wrapper path: keep existing claude, add claude-gemini
   • Mode B — main-command takeover: back up ~/.claude/settings.json, then make plain claude use LiteLLM by default
5. Test with claude -p before recommending interactive use.
6. Keep a rollback path.

When to use this skill

• Claude Code is installed, but you want it to use an existing LiteLLM proxy
• Hermes Agent or another agent already works through LiteLLM and must remain untouched
• Claude Code is currently configured for direct Vertex / Anthropic access in ~/.claude/settings.json
• claude --model gemini-3.1-pro-preview fails even though LiteLLM already exposes that model
• you want a reversible, low-blast-radius integration path
• or you explicitly want plain claude itself to default to LiteLLM + Gemini

The exact integration pattern

1. LiteLLM side prerequisites

LiteLLM must already expose the target model in model_list.

For Toby's working setup:

- model_name: gemini-3.1-pro-preview
  litellm_params:
    model: vertex_ai/gemini-3.1-pro-preview
    vertex_project: os.environ/VERTEXAI_PROJECT
    vertex_location: os.environ/VERTEXAI_LOCATION

And LiteLLM must be listening locally at:

http://127.0.0.1:4000

2. Claude Code talks to LiteLLM with Anthropic-style variables

Use:

export ANTHROPIC_BASE_URL=http://127.0.0.1:4000
export ANTHROPIC_AUTH_TOKEN="$LITELLM_MASTER_KEY"
export ANTHROPIC_MODEL=gemini-3.1-pro-preview

Important:

• ANTHROPIC_BASE_URL points to the LiteLLM root, not /v1
• Claude Code then talks to endpoints like /v1/messages
• LiteLLM handles the translation underneath

3. Disable Claude Code's experimental betas for this path

In the real working setup, this was included:

export CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1

This is a safe compatibility choice when routing Claude Code through LiteLLM to non-Anthropic backends.

The real conflict that blocked success

Symptom

Claude Code was installed and LiteLLM was healthy, but Claude Code did not follow the LiteLLM route cleanly.

Root cause

~/.claude/settings.json already contained a user-level direct-Vertex Claude setup:

{
  "env": {
    "CLAUDE_CODE_USE_VERTEX": "1",
    "ANTHROPIC_VERTEX_PROJECT_ID": "...",
    "CLOUD_ML_REGION": "global",
    "ANTHROPIC_DEFAULT_SONNET_MODEL": "claude-sonnet-4-5@20250929",
    "ANTHROPIC_DEFAULT_OPUS_MODEL": "claude-opus-4-7",
    "ANTHROPIC_DEFAULT_HAIKU_MODEL": "claude-haiku-4-5@20251001"
  }
}

That meant plain claude could prefer its own direct Vertex path rather than the LiteLLM gateway path.

Consequence

Even when LiteLLM had gemini-3.1-pro-preview, a naive invocation such as:

claude -p "..." --model gemini-3.1-pro-preview

could fail with a model-selection error or route through the wrong provider path.

Two supported integration modes

Mode A — dedicated wrapper (lowest risk)

Use this when:

• the user's existing claude setup works and should stay untouched
• the user wants a reversible parallel entrypoint
• other tooling may depend on the current claude behavior

Recommended wrapper pattern

Use a dedicated launcher such as:

~/.local/bin/claude-gemini

Reference implementation:

#!/usr/bin/env bash
set -euo pipefail

PROXY_DIR="/path/to/litellm-project"

source "$PROXY_DIR/scripts/env.sh" >/dev/null 2>&1

unset CLAUDE_CODE_USE_VERTEX
unset ANTHROPIC_VERTEX_PROJECT_ID
unset CLOUD_ML_REGION
unset ANTHROPIC_DEFAULT_SONNET_MODEL
unset ANTHROPIC_DEFAULT_OPUS_MODEL
unset ANTHROPIC_DEFAULT_HAIKU_MODEL

export ANTHROPIC_BASE_URL="${ANTHROPIC_BASE_URL:-http://127.0.0.1:4000}"
export ANTHROPIC_AUTH_TOKEN="$LITELLM_MASTER_KEY"
export ANTHROPIC_MODEL="${ANTHROPIC_MODEL:-gemini-3.1-pro-preview}"
export CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1

extra_args=()
has_setting_sources=0
for arg in "$@"; do
  if [ "$arg" = "--setting-sources" ]; then
    has_setting_sources=1
    break
  fi
done
if [ "$has_setting_sources" -eq 0 ]; then
  extra_args+=(--setting-sources project,local)
fi

exec claude "${extra_args[@]}" "$@"

Why this works:

• keeps the existing global Claude configuration intact
• avoids inheriting the direct-Vertex user settings
• routes only this wrapper invocation through LiteLLM
• preserves reversibility

Mode B — switch plain claude to LiteLLM + Gemini

Use this when:

• the user explicitly wants plain claude to route through LiteLLM
• backing up and replacing ~/.claude/settings.json is acceptable
• the user still wants a parallel wrapper or rollback path preserved

Safe workflow

1. Back up the current ~/.claude/settings.json
2. Replace the env block with LiteLLM-backed variables
3. Keep claude-gemini or another wrapper as a fallback/explicit entrypoint
4. Verify plain claude -p succeeds against the LiteLLM model

Reference ~/.claude/settings.json shape:

{
  "theme": "dark",
  "env": {
    "ANTHROPIC_BASE_URL": "http://127.0.0.1:4000",
    "ANTHROPIC_AUTH_TOKEN": "YOUR_LITELLM_MASTER_KEY",
    "ANTHROPIC_MODEL": "gemini-3.1-pro-preview",
    "CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS": "1"
  }
}

Notes:

• preserve other top-level user preferences such as theme
• the switch is simplest when env is replaced cleanly rather than merged with conflicting direct-provider variables
• always keep a timestamped backup file

Reusable helper:

scripts/switch-main-to-litellm.sh

This helper:

• backs up the current settings file
• rewrites the env block for LiteLLM use
• preserves other top-level JSON fields

Verification flow

1. Verify LiteLLM Anthropic compatibility directly

Before involving Claude Code, verify LiteLLM accepts Anthropic-style requests:

curl -i http://127.0.0.1:4000/v1/messages \
  -H "x-api-key: $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini-3.1-pro-preview",
    "max_tokens": 32,
    "messages": [{"role": "user", "content": "Reply with exactly: ok"}]
  }'

Expected success shape:

{
  "type": "message",
  "model": "gemini-3.1-pro-preview",
  "content": [{"type": "text", "text": "ok"}]
}

This proves the gateway path is good.

2. Verify Claude Code in print mode first

Wrapper mode:

claude-gemini -p "Reply with exactly: wrapper-ok" --output-format json

Main-command mode:

claude -p "Reply with exactly: main-ok" --output-format json

Expected signal:

• command succeeds
• result text matches
• modelUsage shows gemini-3.1-pro-preview

3. Only then recommend interactive use

After print mode succeeds:

claude

or:

claude-gemini

Important lessons from the real incident

Lesson 1: ANTHROPIC_BASE_URL is the LiteLLM root, not /v1

Correct:

ANTHROPIC_BASE_URL=http://127.0.0.1:4000

Not:

ANTHROPIC_BASE_URL=http://127.0.0.1:4000/v1

Lesson 2: A working LiteLLM /v1/models route is not the whole proof

Claude Code uses Anthropic-style flows like /v1/messages.

So verify /v1/messages, not just /v1/models.

Lesson 3: Wrapper-first is the safer default

If the user already has:

• Claude direct Vertex config
• Claude Max / OAuth config
• other provider-specific settings

then a dedicated wrapper is safer than editing the main settings file.

Lesson 4: But main-command takeover is valid when explicitly requested

If the user wants plain claude to default to LiteLLM + Gemini, a backed-up settings.json replacement is a clean approach.

Lesson 5: Test with -p first

claude -p is the smallest, safest proof that the routing works before you ask the user to trust interactive sessions.

Lesson 6: Preserve existing working agents

If Hermes Agent or OpenClaw already uses LiteLLM successfully, Claude Code should be integrated alongside it unless the user explicitly requests a main-command switch.

Nice usability additions

Optional shell alias:

alias cgemini="claude-gemini"

This gives the user two explicit entrypoints:

• claude -> current default path (which may be original Claude or LiteLLM depending on chosen mode)
• cgemini or claude-gemini -> explicit LiteLLM + Gemini path

Common traps

• pointing ANTHROPIC_BASE_URL at /v1
• testing only /v1/models and forgetting /v1/messages
• overwriting ~/.claude/settings.json without a backup
• forgetting to unset CLAUDE_CODE_USE_VERTEX in wrapper mode
• forgetting --setting-sources project,local in wrapper mode
• assuming a non-Anthropic model name will work through plain claude if user settings still force a different provider path
• trying to “fix” Hermes Agent even though Hermes/OpenClaw was already working

Files in this skill

• scripts/claude-gemini-wrapper.sh — reference wrapper for LiteLLM-backed Claude Code
• scripts/switch-main-to-litellm.sh — helper to back up and rewrite ~/.claude/settings.json
• references/quickstart-zh.md — Chinese quickstart and decision guide
• references/toby-working-example.md — exact working pattern and test commands from the real setup

Output standard

When reporting a Claude Code + LiteLLM setup, include:

1. which mode was used: wrapper, main-command takeover, or both
2. whether the existing agent setup was preserved
3. which LiteLLM base URL was used
4. which model Claude Code was routed to
5. whether /v1/messages was verified directly
6. whether claude -p and/or claude-gemini -p was verified successfully
7. what launch command the user should actually run
8. where the rollback backup lives if the main settings file was changed