use-freellmapi
Point a project at FreeLLMAPI — a local proxy that aggregates the free tiers of ~30 LLM providers
(Google, Groq, Cerebras, NVIDIA, Mistral, OpenRouter, GitHub Models, Cohere, Cloudflare, Z.ai,
ModelScope, Ollama, Kilo, OVH, AI Horde, and more) behind a single endpoint — roughly 4B tokens/month
across 251 model families / 358 endpoints. A router picks the best available model per request and
fails over when one is rate-limited. The payoff: prototype against real models for free, with zero code
changes beyond a base URL and a key.
Don't hardcode the roster or the numbers into advice. Providers come and go, the catalog
self-updates from a signed feed, and free installs run on a 30-day catalog trail. Read the live
/v1/models and the dashboard rather than trusting any list — including this one.
Tiering note: a FreeLLMAPI project is the toolkit's one sanctioned multi-provider setup. The
model & effort tiering policy (model-adaptation, Model & effort tiering) is provider-relative —
within a normal build you pick tiers inside ONE provider's ladder and never mix vendors to save
tokens — and FreeLLMAPI is its explicit carve-out: here the aggregated free tiers ARE the ladder,
and the scarce resource is rate/quota headroom, not dollars.
The whole integration, in three facts
FreeLLMAPI is an OpenAI-compatible proxy. Wiring a project to it is almost always just three values:
| What |
Value |
| base_url |
http://localhost:3001/v1 (default; PORT may differ). Anthropic, Gemini, and Ollama clients use the bare origin — http://localhost:3001, no /v1 |
| api_key (bearer) |
freellmapi-… — the install's unified key — or better, a per-client key (sk-cp-…) minted for this project |
| model |
"auto" — let the router pick and fail over. Steer it per request with "auto:fast" / "auto:smart" / "auto:reliable" / "auto:<profile-name>"; pin one like "gemini-2.5-flash"; or "fusion" to blend a panel of models into one answer |
Because the surface is OpenAI-compatible, any OpenAI client library works unchanged — chat, streaming,
tool calling, vision, and embeddings all route through the same endpoint. But it is not only
OpenAI-shaped: the proxy also speaks the native Anthropic Messages API (/v1/messages — Claude Code
and the Anthropic SDKs), the native Gemini API (/v1beta — Gemini CLI), and an opt-in Ollama
surface (/api/* — Zed, JetBrains AI). Point each client at the wire format it already speaks instead
of forcing everything through chat completions. Everything past this point is about doing the swap
cleanly, reversibly, and verified — not about anything exotic.
Read references/capabilities.md for the exact supported surface (all six wire formats, the virtual
models auto/fusion and auto:* steering, embeddings families, vision, tools, structured outputs,
media, the /mcp server, prompt compression, the response cache, analytics) and the short list of
things it still does not do (moderation, n > 1, multi-tenant auth). Check it before promising a
capability — and when it matters, confirm against GET /v1/docs on the user's own install, since
older pinned images lag the catalog.
Workflow
Work top to bottom. Steps 1–2 stand up the proxy and confirm it can actually serve a request; steps
3–5 rewire the project and prove it works end to end. Don't skip the verify — "it should work" is not
the same as a 200 with a completion in it.
Step 1 — Make sure the proxy is running
Probe the unauthenticated liveness endpoint:
curl -fsS http://localhost:3001/api/ping
# {"status":"ok","timestamp":"..."} → something is answering, but see the check below
A 200 on /api/ping does not prove the container answered. A stray host-side npm run dev
from a source checkout binds *:3001 on the IPv6 wildcard, which the OS prefers over the container's
127.0.0.1:3001 when resolving localhost. That dev server answers /api/ping happily but has no
built client (ENOENT … client/dist/index.html, with a host path in the error) and its own empty
database (a bogus "create an account" page) — which reads exactly like a wiped install after a
rebuild. If the dashboard looks freshly-installed or errors on refresh, check who owns the port
before touching any data:
lsof -nP -iTCP:3001 -sTCP:LISTEN # a `node` row next to `com.docker` is the bug
Kill the npm run dev parent (not just the node child — tsx watch respawns it), then re-probe.
Never reach for docker compose down -v to "reset" this; that is what actually destroys keys.
If nothing answers, install and start it with the official one-liner (needs Docker running):
curl -fsSL https://freellmapi.co/install.sh | bash
This sets up ~/freellmapi, generates an at-rest ENCRYPTION_KEY, pulls
ghcr.io/tashfeenahmed/freellmapi:latest, starts the container on :3001, and waits for /api/ping.
Re-running is safe — it preserves the existing .env. When it finishes, probe /api/ping again to
confirm. If Docker isn't available, the project also ships a desktop app bundling the same server
(macOS, Windows installer or zip, Linux AppImage/deb/tar.xz — the repo's Releases page), or fall back
to a source install (git clone + npm install + npm run dev); see the Local development section
of the repo README.
If the user already runs it on a non-default port or another host, use that base_url everywhere below.
Step 2 — Get the unified key and make sure a provider can serve requests
Unified key (freellmapi-…): the install-wide bearer token. Get it from:
- the Keys page header at
http://localhost:3001 (the dashboard), or
- the first-run container logs:
cd ~/freellmapi && docker compose logs 2>&1 | grep -i "unified api key"
Since v0.6.9 the Keys page can also mint per-client keys (sk-cp-…) — one per downstream app or
tool, independently revocable, each with an optional system prompt the proxy enforces server-side.
For wiring a single project, prefer minting one: analytics attribute traffic by key, and revoking it
later doesn't touch any other client's credential. Everything below works the same with either key.
A fresh proxy has no provider keys, so chat requests fail until at least one provider is enabled.
This is the one part you can't do for the user — adding keys and toggling providers happens in the
browser dashboard. Make it explicit and offer to wait. Two paths:
- Fastest, zero-key smoke test: on the dashboard, enable a keyless provider — as of v0.6.9
that's Kilo
:free, OVH, or AI Horde, which need no API key at all. Good enough to prove
the pipe end to end in under a minute. (Pollinations is no longer keyless — it validates a key
now, so don't offer it as the zero-key path. AI Horde is queue-based and can take tens of seconds.)
- Real prototyping: add free provider keys on the Keys page (Google AI Studio, Groq, Cerebras,
etc. — each has a free signup), then reorder the Fallback Chain to taste.
Hold here until the user confirms at least one provider is enabled — otherwise Step 5 will fail with a
"no model available" error and look like a wiring bug when it isn't. The Keys page's provider
checklist (status icons per provider, flagging any that need a key but have none) is the quickest
way to confirm, beating guesswork about which toggle actually took.
Step 3 — Detect the target: coding agent, or application code?
First ask which kind of target this is — the two paths diverge completely.
If the target is a coding agent or CLI (Claude Code, Codex, Gemini CLI, Cline, Aider, Continue,
OpenCode, Goose, Qwen, Roo, Kilo, Crush, Cursor, Zed, JetBrains AI) — do not hand-edit its config.
The proxy ships generators that read the live catalog, merge with existing config, and write a
timestamped backup first:
npx freellmapi setup-claude --url http://localhost:3001 --dry-run # show the diff
npx freellmapi setup-claude --url http://localhost:3001 # apply, with backup
Read references/agent-clients.md for the full command table, the per-client base URLs, and the traps
that waste the most time (Claude Code needs the origin not /v1, and ANTHROPIC_AUTH_TOKEN not
ANTHROPIC_API_KEY or it refuses to start; Ollama emulation is off by default and open-loopback
doesn't work in Docker). Then skip to Step 5 and verify — Step 4's env-toggle pattern is for
application code, not agent config.
If the target is application code, continue here. Find how the project talks to an LLM today so you
pick the right recipe and the smallest diff:
# language + client library + where the client is built
# (quote the --include globs — zsh expands bare *.py before grep sees it and the command fails)
grep -rniE "openai|anthropic|langchain|llama_?index|@?ai-sdk|base_?url|api_?key" \
--include="*.py" --include="*.ts" --include="*.tsx" --include="*.js" --include="*.mjs" . \
| grep -viE "node_modules|/dist/|/build/" | head -40
Determine: the language (Python / JS-TS), the client library, the file(s) where the client is
constructed, and where its base_url / api_key / model come from (env vars vs hardcoded). Then open
references/recipes.md and read only the section for the detected stack — it has copy-paste wiring
for OpenAI (Python/Node), LangChain, LlamaIndex, Vercel AI SDK, Continue, raw HTTP/curl, and the
Anthropic-SDK-via-x-api-key case.
Step 4 — Rewire it, reversibly (the env-toggle pattern)
The point of prototyping with free models is that you'll flip back and forth — free models for cheap
iteration, the paid provider for a quality check or a demo. A hardcoded swap forces a code edit every
time and risks losing the original config. So drive the three values from environment variables and
keep the original provider as a documented fallback, rather than editing call sites.
Concretely:
Introduce (or reuse) env vars for the base URL, key, and model. Many OpenAI clients already read
OPENAI_BASE_URL / OPENAI_API_KEY from the environment, so reusing those names is ideal. But "no
code change at all" only holds when both are true: (a) the client is built with no explicit
base_url/api_key args, and (b) something actually loads .env into the process — a shell
set -a; source .env, a process manager's env_file, or a dotenv loader the app already calls.
Plain Python and plain Node do not auto-read a .env file. If neither is true (very common — a
prototype with the key hardcoded, and no dotenv), you will edit the call site and/or add a loader;
that's expected, not a failure. Each recipe in references/recipes.md notes how its stack loads env.
Point the client constructor at those env vars instead of literals (only if it wasn't already).
In .env, set them to the FreeLLMAPI values and keep the original provider's values commented
right above them, labeled, so flipping back is uncommenting two lines:
# --- FreeLLMAPI (free, local proxy) — active ---
OPENAI_BASE_URL=http://localhost:3001/v1
OPENAI_API_KEY=freellmapi-xxxxxxxxxxxxxxxxxxxxxxxx
LLM_MODEL=auto
# --- Original provider (paid) — flip back by swapping which block is active ---
# OPENAI_BASE_URL=https://api.openai.com/v1
# OPENAI_API_KEY=sk-...
# LLM_MODEL=gpt-4o-mini
Default the model to "auto" so the router chooses and fails over, but keep the option to pin a
specific model where a call needs a known capability (e.g. a vision model).
Write or update a committed .env.example with the same two labeled blocks but placeholder
values (freellmapi-xxxx, sk-...). Without it, moving a hardcoded key into .env leaves the repo
with no record of which env vars it now needs — the example is the documentation.
Show the user the .env block and the (usually tiny) code diff. Never write a real provider key into a
file that's tracked by git — confirm .env is gitignored, and if a key was previously hardcoded in
source, move it to .env as part of this step.
Step 5 — Verify with a live call
Prove the pipe works before declaring success. First a direct smoke test (substitute the real key):
curl -i -s http://localhost:3001/v1/chat/completions \
-H "Authorization: Bearer freellmapi-xxxx" \
-H "Content-Type: application/json" \
-d '{"model":"auto","messages":[{"role":"user","content":"Reply with the single word: ok"}]}'
A healthy response is a 200 with a chat completion, and an X-Routed-Via: <platform>/<model>
header telling you which provider actually served it. Then run the project's own code path (its
test, its entry script, a representative call) so you've confirmed the app — not just curl — talks to
the proxy.
Map failures to causes instead of guessing:
| Symptom |
Cause |
Fix |
401 invalid API key |
wrong/missing unified key |
re-copy it from the dashboard Keys header |
4xx no model / no provider |
no provider enabled on the proxy |
back to Step 2 — enable a provider |
429 |
the picked key is rate-limited |
router fails over; add more provider keys for headroom |
| connection refused |
proxy not running / wrong port |
back to Step 1; check PORT |
Step 6 — Hand off
Tell the user, briefly:
- How to flip back — uncomment the original
.env block (or re-run the agent's own config backup).
- Where the dashboard is (
http://localhost:3001) and what's on it: Models (Chat / Embeddings /
Image / Audio / Fusion tabs, with editable intelligence-and-speed ranks and per-model rate limits),
Playground (now accepts image and text-file attachments), Keys (provider checklist,
per-client sk-cp-… keys, per-key model scopes and proxy overrides, bulk key actions), Agents
(per-tool setup blocks, Anthropic/Gemini family maps, Ollama emulation mode, URL tokens),
Analytics, and Premium. Named routing profiles auto-sort a chain by
intelligence/speed/budget and are selectable per request as auto:<profile>. Settings live in a
sidebar dialog and include an automatic update check with release notes; a forgotten dashboard
password is recoverable with a code printed in the server logs. The UI ships in 60 languages.
- What the model values do —
auto routes and fails over, auto:fast / auto:smart /
auto:reliable steer one request without touching the dashboard, fusion blends a panel of models
for a quality bump on hard prompts.
- What analytics will tell them later — p50/p95 latency, time-to-first-token, success rate,
estimated savings, and pin-honor rate (the number that answers "I asked for model X, why did I get
Y?"), plus per-request failover traces.
- The relevant not-supported caveats from
references/capabilities.md — moderation, n > 1, and
multi-tenant auth won't route, so those calls still need the real provider. Legacy
/v1/completions, image generation, text-to-speech, and fusion tool-calling do work on current
releases; only image input to /v1/responses and to fusion is still missing.
- If the catalog looks stale — free installs run on a 30-day catalog trail (the live signed feed is
the paid tier), so a missing new model or a provider whose free tier changed is usually catalog lag,
not a wiring bug.
- Optionally, prompt compression — off by default, but worth mentioning for long agent sessions:
it shrinks re-sent context before routing so more small-context models stay eligible.
References
references/recipes.md — per-framework, copy-paste env-toggle wiring for application code.
Read the one section that matches the detected stack; ignore the rest.
references/agent-clients.md — wiring for coding agents and CLIs: the npx freellmapi setup-* generator table, Claude Code / Codex / Gemini CLI / Ollama-client / headerless setups, the
MCP server, and the per-client traps. Read this whenever the target is a tool rather than a codebase.
references/capabilities.md — exact supported surface across all six wire formats (chat,
streaming, tools, vision, structured outputs, embeddings + families, Responses, legacy completions,
media, transcription, Anthropic Messages, native Gemini, Ollama, /mcp, URL tokens, ops endpoints),
plus routing behavior and auto:* steering, prompt compression, the response cache, analytics, the
not-supported list, and the gotchas (sticky sessions, fresh-install-has-no-keys, which providers are
actually keyless, catalog lag, the X-Routed-Via / X-Fallback-Trail / X-Request-ID headers and
their percent-encoding trap).
1---2name: use-freellmapi3version: 1.4.04description: |5 Wire any project or coding agent to FreeLLMAPI — a local proxy aggregating ~30 free LLM provider6 tiers behind one endpoint — so you can prototype without paying for API calls. Use to switch a7 project off paid OpenAI/Anthropic/etc. onto free models, point an app or CLI agent at a local LLM8 proxy, or stand up FreeLLMAPI itself. Speaks OpenAI, Anthropic Messages, native Gemini, and Ollama9 wire formats, covering Claude Code, Codex CLI, Gemini CLI, Cline, Aider, Continue, Zed, and10 JetBrains AI. Detects the project's LLM client, ensures the proxy is running, rewires base_url +11 api_key behind an env toggle, and verifies with a live test call. Trigger on "use freellmapi",12 "switch this to free models", "point this at freellmapi", "stop paying for openai here", "free llm13 proxy", "prototype without api costs", "run claude code on free models", "point claude14 code/codex/gemini cli at freellmapi", "free models for my coding agent".15requires_agent_teams: false16requires_claude_code: false17min_plan: starter18owns:19 directories: []20 patterns: []21 shared_read: ["*"]22allowed-tools: ["Read", "Write", "Edit", "Bash", "Grep", "Glob"]23composes_with: ["project-profiler", "model-adaptation", "use-pxpipe"]24spawned_by: []25metadata:26 source: https://skillmd.com/skills/ivy00johns/use-freellmapi27---2829# use-freellmapi3031Point a project at **FreeLLMAPI** — a local proxy that aggregates the free tiers of ~30 LLM providers32(Google, Groq, Cerebras, NVIDIA, Mistral, OpenRouter, GitHub Models, Cohere, Cloudflare, Z.ai,33ModelScope, Ollama, Kilo, OVH, AI Horde, and more) behind a single endpoint — roughly **4B tokens/month34across 251 model families / 358 endpoints**. A router picks the best available model per request and35fails over when one is rate-limited. The payoff: prototype against real models for free, with zero code36changes beyond a base URL and a key.3738> **Don't hardcode the roster or the numbers into advice.** Providers come and go, the catalog39> self-updates from a signed feed, and free installs run on a 30-day catalog trail. Read the live40> `/v1/models` and the dashboard rather than trusting any list — including this one.4142> **Tiering note:** a FreeLLMAPI project is the toolkit's one sanctioned multi-provider setup. The43> model & effort tiering policy (`model-adaptation`, *Model & effort tiering*) is provider-relative —44> within a normal build you pick tiers inside ONE provider's ladder and never mix vendors to save45> tokens — and FreeLLMAPI is its explicit carve-out: here the aggregated free tiers ARE the ladder,46> and the scarce resource is rate/quota headroom, not dollars.4748## The whole integration, in three facts4950FreeLLMAPI is an **OpenAI-compatible** proxy. Wiring a project to it is almost always just three values:5152| What | Value |53| --- | --- |54| **base_url** | `http://localhost:3001/v1` (default; `PORT` may differ). **Anthropic, Gemini, and Ollama clients use the bare origin** — `http://localhost:3001`, no `/v1` |55| **api_key** (bearer) | `freellmapi-…` — the install's *unified key* — or better, a **per-client key** (`sk-cp-…`) minted for this project |56| **model** | `"auto"` — let the router pick and fail over. Steer it per request with `"auto:fast"` / `"auto:smart"` / `"auto:reliable"` / `"auto:<profile-name>"`; pin one like `"gemini-2.5-flash"`; or `"fusion"` to blend a panel of models into one answer |5758Because the surface is OpenAI-compatible, any OpenAI client library works unchanged — chat, streaming,59tool calling, vision, and embeddings all route through the same endpoint. But it is **not only**60OpenAI-shaped: the proxy also speaks the **native Anthropic Messages API** (`/v1/messages` — Claude Code61and the Anthropic SDKs), the **native Gemini API** (`/v1beta` — Gemini CLI), and an opt-in **Ollama**62surface (`/api/*` — Zed, JetBrains AI). Point each client at the wire format it already speaks instead63of forcing everything through chat completions. Everything past this point is about doing the swap64*cleanly, reversibly, and verified* — not about anything exotic.6566> Read `references/capabilities.md` for the exact supported surface (all six wire formats, the virtual67> models `auto`/`fusion` and `auto:*` steering, embeddings families, vision, tools, structured outputs,68> media, the `/mcp` server, prompt compression, the response cache, analytics) and the short list of69> things it still does **not** do (moderation, `n > 1`, multi-tenant auth). Check it before promising a70> capability — and when it matters, confirm against `GET /v1/docs` on the user's own install, since71> older pinned images lag the catalog.7273## Workflow7475Work top to bottom. Steps 1–2 stand up the proxy and confirm it can actually serve a request; steps763–5 rewire the project and prove it works end to end. Don't skip the verify — "it should work" is not77the same as a 200 with a completion in it.7879### Step 1 — Make sure the proxy is running8081Probe the unauthenticated liveness endpoint:8283```bash84curl -fsS http://localhost:3001/api/ping85# {"status":"ok","timestamp":"..."} → something is answering, but see the check below86```8788> **A 200 on `/api/ping` does not prove the *container* answered.** A stray host-side `npm run dev`89> from a source checkout binds `*:3001` on the IPv6 wildcard, which the OS prefers over the container's90> `127.0.0.1:3001` when resolving `localhost`. That dev server answers `/api/ping` happily but has no91> built client (`ENOENT … client/dist/index.html`, with a **host** path in the error) and its own empty92> database (a bogus "create an account" page) — which reads exactly like a wiped install after a93> rebuild. If the dashboard looks freshly-installed or errors on refresh, check who owns the port94> **before** touching any data:95>96> ```bash97> lsof -nP -iTCP:3001 -sTCP:LISTEN # a `node` row next to `com.docker` is the bug98> ```99>100> Kill the `npm run dev` parent (not just the node child — `tsx watch` respawns it), then re-probe.101> Never reach for `docker compose down -v` to "reset" this; that is what actually destroys keys.102103If nothing answers, install and start it with the official one-liner (needs Docker running):104105```bash106curl -fsSL https://freellmapi.co/install.sh | bash107```108109This sets up `~/freellmapi`, generates an at-rest `ENCRYPTION_KEY`, pulls110`ghcr.io/tashfeenahmed/freellmapi:latest`, starts the container on `:3001`, and waits for `/api/ping`.111Re-running is safe — it preserves the existing `.env`. When it finishes, probe `/api/ping` again to112confirm. If Docker isn't available, the project also ships a **desktop app** bundling the same server113(macOS, Windows installer or zip, Linux AppImage/deb/tar.xz — the repo's Releases page), or fall back114to a source install (`git clone` + `npm install` + `npm run dev`); see the *Local development* section115of the [repo README](https://github.com/tashfeenahmed/freellmapi).116117If the user already runs it on a non-default port or another host, use that `base_url` everywhere below.118119### Step 2 — Get the unified key and make sure a provider can serve requests120121**Unified key** (`freellmapi-…`): the install-wide bearer token. Get it from:122123- the **Keys page header** at `http://localhost:3001` (the dashboard), or124- the first-run container logs:125 `cd ~/freellmapi && docker compose logs 2>&1 | grep -i "unified api key"`126127Since v0.6.9 the Keys page can also mint **per-client keys** (`sk-cp-…`) — one per downstream app or128tool, independently revocable, each with an optional system prompt the proxy enforces server-side.129For wiring a single project, prefer minting one: analytics attribute traffic by key, and revoking it130later doesn't touch any other client's credential. Everything below works the same with either key.131132**A fresh proxy has no provider keys, so chat requests fail until at least one provider is enabled.**133This is the one part you can't do for the user — adding keys and toggling providers happens in the134browser dashboard. Make it explicit and offer to wait. Two paths:135136- **Fastest, zero-key smoke test:** on the dashboard, enable a **keyless** provider — as of v0.6.9137 that's **Kilo `:free`**, **OVH**, or **AI Horde**, which need no API key at all. Good enough to prove138 the pipe end to end in under a minute. (**Pollinations is no longer keyless** — it validates a key139 now, so don't offer it as the zero-key path. AI Horde is queue-based and can take tens of seconds.)140- **Real prototyping:** add free provider keys on the **Keys** page (Google AI Studio, Groq, Cerebras,141 etc. — each has a free signup), then reorder the **Fallback Chain** to taste.142143Hold here until the user confirms at least one provider is enabled — otherwise Step 5 will fail with a144"no model available" error and look like a wiring bug when it isn't. The Keys page's **provider145checklist** (status icons per provider, flagging any that need a key but have none) is the quickest146way to confirm, beating guesswork about which toggle actually took.147148### Step 3 — Detect the target: coding agent, or application code?149150**First ask which kind of target this is — the two paths diverge completely.**151152**If the target is a coding agent or CLI** (Claude Code, Codex, Gemini CLI, Cline, Aider, Continue,153OpenCode, Goose, Qwen, Roo, Kilo, Crush, Cursor, Zed, JetBrains AI) — **do not hand-edit its config.**154The proxy ships generators that read the live catalog, merge with existing config, and write a155timestamped backup first:156157```bash158npx freellmapi setup-claude --url http://localhost:3001 --dry-run # show the diff159npx freellmapi setup-claude --url http://localhost:3001 # apply, with backup160```161162Read `references/agent-clients.md` for the full command table, the per-client base URLs, and the traps163that waste the most time (Claude Code needs the **origin** not `/v1`, and `ANTHROPIC_AUTH_TOKEN` **not**164`ANTHROPIC_API_KEY` or it refuses to start; Ollama emulation is off by default and `open-loopback`165doesn't work in Docker). Then skip to Step 5 and verify — Step 4's env-toggle pattern is for166application code, not agent config.167168**If the target is application code**, continue here. Find how the project talks to an LLM today so you169pick the right recipe and the smallest diff:170171```bash172# language + client library + where the client is built173# (quote the --include globs — zsh expands bare *.py before grep sees it and the command fails)174grep -rniE "openai|anthropic|langchain|llama_?index|@?ai-sdk|base_?url|api_?key" \175 --include="*.py" --include="*.ts" --include="*.tsx" --include="*.js" --include="*.mjs" . \176 | grep -viE "node_modules|/dist/|/build/" | head -40177```178179Determine: the language (Python / JS-TS), the client library, the file(s) where the client is180constructed, and where its `base_url` / `api_key` / `model` come from (env vars vs hardcoded). Then open181`references/recipes.md` and read **only the section for the detected stack** — it has copy-paste wiring182for OpenAI (Python/Node), LangChain, LlamaIndex, Vercel AI SDK, Continue, raw HTTP/curl, and the183Anthropic-SDK-via-`x-api-key` case.184185### Step 4 — Rewire it, reversibly (the env-toggle pattern)186187The point of prototyping with free models is that you'll flip *back and forth* — free models for cheap188iteration, the paid provider for a quality check or a demo. A hardcoded swap forces a code edit every189time and risks losing the original config. So **drive the three values from environment variables and190keep the original provider as a documented fallback**, rather than editing call sites.191192Concretely:1931941. Introduce (or reuse) env vars for the base URL, key, and model. Many OpenAI clients already read195 `OPENAI_BASE_URL` / `OPENAI_API_KEY` from the environment, so reusing those names is ideal. But "no196 code change at all" only holds when **both** are true: (a) the client is built with no explicit197 `base_url`/`api_key` args, **and** (b) something actually loads `.env` into the process — a shell198 `set -a; source .env`, a process manager's `env_file`, or a dotenv loader the app already calls.199 Plain Python and plain Node do **not** auto-read a `.env` file. If neither is true (very common — a200 prototype with the key hardcoded, and no dotenv), you will edit the call site and/or add a loader;201 that's expected, not a failure. Each recipe in `references/recipes.md` notes how its stack loads env.2022. Point the client constructor at those env vars instead of literals (only if it wasn't already).2033. In `.env`, set them to the FreeLLMAPI values and **keep the original provider's values commented204 right above them**, labeled, so flipping back is uncommenting two lines:205206 ```dotenv207 # --- FreeLLMAPI (free, local proxy) — active ---208 OPENAI_BASE_URL=http://localhost:3001/v1209 OPENAI_API_KEY=freellmapi-xxxxxxxxxxxxxxxxxxxxxxxx210 LLM_MODEL=auto211212 # --- Original provider (paid) — flip back by swapping which block is active ---213 # OPENAI_BASE_URL=https://api.openai.com/v1214 # OPENAI_API_KEY=sk-...215 # LLM_MODEL=gpt-4o-mini216 ```2172184. Default the model to `"auto"` so the router chooses and fails over, but keep the option to pin a219 specific model where a call needs a known capability (e.g. a vision model).2205. Write or update a committed **`.env.example`** with the same two labeled blocks but placeholder221 values (`freellmapi-xxxx`, `sk-...`). Without it, moving a hardcoded key into `.env` leaves the repo222 with no record of which env vars it now needs — the example is the documentation.223224Show the user the `.env` block and the (usually tiny) code diff. Never write a real provider key into a225file that's tracked by git — confirm `.env` is gitignored, and if a key was previously hardcoded in226source, move it to `.env` as part of this step.227228### Step 5 — Verify with a live call229230Prove the pipe works before declaring success. First a direct smoke test (substitute the real key):231232```bash233curl -i -s http://localhost:3001/v1/chat/completions \234 -H "Authorization: Bearer freellmapi-xxxx" \235 -H "Content-Type: application/json" \236 -d '{"model":"auto","messages":[{"role":"user","content":"Reply with the single word: ok"}]}'237```238239A healthy response is a `200` with a chat completion, and an **`X-Routed-Via: <platform>/<model>`**240header telling you which provider actually served it. Then run the **project's own** code path (its241test, its entry script, a representative call) so you've confirmed the app — not just curl — talks to242the proxy.243244Map failures to causes instead of guessing:245246| Symptom | Cause | Fix |247| --- | --- | --- |248| `401` invalid API key | wrong/missing unified key | re-copy it from the dashboard Keys header |249| `4xx` no model / no provider | no provider enabled on the proxy | back to Step 2 — enable a provider |250| `429` | the picked key is rate-limited | router fails over; add more provider keys for headroom |251| connection refused | proxy not running / wrong port | back to Step 1; check `PORT` |252253### Step 6 — Hand off254255Tell the user, briefly:256257- **How to flip back** — uncomment the original `.env` block (or re-run the agent's own config backup).258- **Where the dashboard is** (`http://localhost:3001`) and what's on it: **Models** (Chat / Embeddings /259 Image / Audio / Fusion tabs, with editable intelligence-and-speed ranks and per-model rate limits),260 **Playground** (now accepts image and text-file attachments), **Keys** (provider checklist,261 per-client `sk-cp-…` keys, per-key model scopes and proxy overrides, bulk key actions), **Agents**262 (per-tool setup blocks, Anthropic/Gemini family maps, Ollama emulation mode, URL tokens),263 **Analytics**, and **Premium**. Named **routing profiles** auto-sort a chain by264 intelligence/speed/budget and are selectable per request as `auto:<profile>`. Settings live in a265 sidebar dialog and include an automatic update check with release notes; a forgotten dashboard266 password is recoverable with a code printed in the server logs. The UI ships in 60 languages.267- **What the model values do** — `auto` routes and fails over, `auto:fast` / `auto:smart` /268 `auto:reliable` steer one request without touching the dashboard, `fusion` blends a panel of models269 for a quality bump on hard prompts.270- **What analytics will tell them later** — p50/p95 latency, time-to-first-token, success rate,271 estimated savings, and **pin-honor rate** (the number that answers "I asked for model X, why did I get272 Y?"), plus per-request failover traces.273- **The relevant not-supported caveats** from `references/capabilities.md` — moderation, `n > 1`, and274 multi-tenant auth won't route, so those calls still need the real provider. Legacy275 `/v1/completions`, image generation, text-to-speech, and fusion tool-calling **do** work on current276 releases; only image *input* to `/v1/responses` and to `fusion` is still missing.277- **If the catalog looks stale** — free installs run on a 30-day catalog trail (the live signed feed is278 the paid tier), so a missing new model or a provider whose free tier changed is usually catalog lag,279 not a wiring bug.280- **Optionally, prompt compression** — off by default, but worth mentioning for long agent sessions:281 it shrinks re-sent context before routing so more small-context models stay eligible.282283## References284285- **`references/recipes.md`** — per-framework, copy-paste env-toggle wiring for **application code**.286 Read the one section that matches the detected stack; ignore the rest.287- **`references/agent-clients.md`** — wiring for **coding agents and CLIs**: the `npx freellmapi288 setup-*` generator table, Claude Code / Codex / Gemini CLI / Ollama-client / headerless setups, the289 MCP server, and the per-client traps. Read this whenever the target is a tool rather than a codebase.290- **`references/capabilities.md`** — exact supported surface across all six wire formats (chat,291 streaming, tools, vision, structured outputs, embeddings + families, Responses, legacy completions,292 media, transcription, Anthropic Messages, native Gemini, Ollama, `/mcp`, URL tokens, ops endpoints),293 plus routing behavior and `auto:*` steering, prompt compression, the response cache, analytics, the294 not-supported list, and the gotchas (sticky sessions, fresh-install-has-no-keys, which providers are295 actually keyless, catalog lag, the `X-Routed-Via` / `X-Fallback-Trail` / `X-Request-ID` headers and296 their percent-encoding trap).