# Archestra Dev LLM Providers > Use when adding an LLM provider, changing proxy adapters or provider routes, fixing streaming/tool-call translation bugs, editing model fetchers or model handling, or touching provider credentials/enums and model constants. - Skill: `archestra-ai/archestra-dev-llm-providers` (Agent Skill) - Install (CLI): `npx skillmds@latest add archestra-ai/archestra-dev-llm-providers` - Raw SKILL.md: https://api.skillmd.com/api/skills/archestra-ai/archestra-dev-llm-providers/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: AI & ML - Author: archestra-ai (https://skillmd.com/u/archestra-ai) - Updated: 2026-09-10 - Page: https://skillmd.com/skills/archestra-ai/archestra-dev-llm-providers --- # Archestra LLM Providers & Proxy Use this skill before adding an LLM provider or changing provider translation, streaming, or model handling. Run commands from `platform/` unless specifically instructed otherwise. ## Provider surface map For a chat-capable provider, check all of these (use `github-copilot` as the worked example — it is OpenAI-compatible, so it follows the default path; `microsoft-365-copilot` shows the harder shape, with its own graph translator): - `backend/src/types/llm-providers//` — `api.ts`, `messages.ts`, `tools.ts`, `index.ts` (some also have `models.ts`). `index.ts` default-exports a namespace (e.g. `GithubCopilot`) with `API`/`Messages`/`Tools` plus a `Types` sub-namespace; register it in `types/llm-providers/index.ts`. OpenAI-compatible providers re-export OpenAI schemas with `.passthrough()`. - `backend/src/routes/proxy/adapters/.ts` — exports `AdapterFactory`; re-export it from `adapters/index.ts`. - `backend/src/routes/proxy/routes/.ts` — Fastify plugin: `fastifyHttpProxy` catch-all with `createProxyPreHandler` (from `proxy-prehandler.ts`), explicit `POST .../chat/completions` handlers (default-agent and `:agentId` variants) calling `handleLLMProxy`, and model-listing GETs via `proxy-model-listing.ts`. Register the plugin in BOTH places: re-export it from `backend/src/routes/index.ts` (the main API surface iterates `Object.values(routes)`) AND add it to `registerWorkerRoutes` in `server.ts`. - `shared/model-constants.ts` — add to `SupportedProvidersSchema`, `SupportedProvidersDiscriminatorSchema` (`:chatCompletions` for OpenAI-compatible; others name their API shape, e.g. `anthropic:messages`, `bedrock:converse`), and `providerDisplayNames`. Membership in `PROVIDERS_WITH_OPTIONAL_API_KEY`, `PROVIDERS_REQUIRING_BASE_URL`, and `PROVIDERS_REQUIRING_PER_USER_CREDENTIAL` silently changes auth behavior: per-user-credential providers get personal-scope keys only, no team/org/env fallback (see the `github-copilot` rationale comment there). - `backend/src/routes/chat/model-fetchers/` — add a fetcher and register it in the `modelFetchers` record in `model-fetchers/index.ts`; its `Record` type makes a missing provider a compile error. `registry.ts#testProviderApiKey` uses it to validate keys on creation. Simple bearer `/models` endpoints reuse `makeBearerFetcher`/`makeStaticFetcher` from `bearer-fetcher.ts`. - Message normalization for the chat feature lives in `backend/src/routes/chat/normalization/` (notably `prepare-for-provider.ts`) and `prepare-model-messages.ts` — provider-specific message-shape rules go here, not in the proxy adapters. - Frontend: provider key management at `frontend/src/app/llm/model-providers/page.tsx` + `frontend/src/components/create-llm-provider-api-key-dialog.tsx`; provider icon at `frontend/public/icons/.png`; model pickers (`components/llm-model-select.tsx`, `components/chat/model-selector.tsx`) use `providerDisplayNames`. - Also: `backend/src/config.ts` + `.env.example` for base-URL/key env vars, `../docs/pages/platform-supported-llm-providers.md`. For embeddings-only providers, follow `voyage`: register the provider and model fetcher, wire the embedding client, and add it to `EMBEDDING_ONLY_PROVIDER_LIST` in `shared/model-constants.ts`. Chat surfaces use `providerSupportsChat` and `ChatProvider`; do not invent chat routes, adapters, or chat-matrix entries for a provider with no chat API. ## Default path: OpenAI-compatible - Most new providers are OpenAI-compatible. Do not hand-roll a translator: call `createOpenAiCompatibleAdapterFactory` from `adapters/openai-compatible-adapter.ts` with `provider`, `interactionType`, `getBaseUrl`, and `createClient` — it reuses `OpenAIRequestAdapter`/`OpenAIResponseAdapter`/`OpenAIStreamAdapter` wholesale. See `adapters/deepseek.ts` (minimal) and `adapters/github-copilot.ts` (custom auth via a fetch wrapper, since `createClient` is synchronous). - Providers with genuinely different wire formats get translator modules next to the adapter (`gemini-openai-translator.ts`, `bedrock-openai-translator.ts`, `cohere-openai-translator.ts`, `anthropic-openai-translator.ts`) — fix translation bugs there, with a matching `*.test.ts`. ## Guard rails - `backend/src/routes/proxy/routes/provider-matrix.test.ts` — `providerConfigsByProvider` is `satisfies Record`, so adding a chat-capable provider without a matrix entry (route plugin + adapter factory + endpoints) fails typecheck. The suite then exercises every chat provider's real route with a mocked client: declared-tool persistence, execution IDs, streaming tool calls, TOON compression, and limit blocking. - The `modelFetchers` record (above) enforces the same exhaustiveness for model listing. ## Translation gotchas (real handling, check before "fixing") - **Empty assistant turns**: `convertToModelMessages` can produce assistant messages with empty content that providers reject. `buildModelMessagesForProvider` in `routes/chat/prepare-model-messages.ts` filters them (`isEmptyAssistantModelMessage`) and then repairs unanswered tool calls (`ensureToolCallsHaveResults`) so `tool_use`/`tool_result` adjacency holds. The Cohere proxy adapter (`adapters/cohere.ts`) does its own empty-assistant filtering. - **Tool-call name repair**: harmony-format models leak reasoning-channel sentinels into tool names (`name<|channel|>commentary`). `routes/chat/tool-call-repair.ts#repairHarmonyToolName` strips them, gated on an exact match against registered tools; wired via `experimental_repairToolCall` in `routes/chat/routes.ts`. - **Provider message-shape rules**: Gemini and Bedrock require the first non-system turn to be from the user — `ensureLeadingUserTurn` in `prepare-model-messages.ts` prepends one for both (the provider set gating it lives next to the function). Bedrock content rules (every message non-empty, user messages need a text part) are enforced in `normalization/prepare-for-provider.ts` (`ensureBedrockMessageHasContent`, `ensureBedrockUserMessageHasTextPart`); the same file decides per provider whether text documents stay native `document` blocks (Anthropic/Bedrock) or are inlined as decoded text (everyone else). - **Output-token ceilings**: `agents/agent-output-budget.ts#resolveAgentMaxOutputTokens` clamps `maxOutputTokens` to the model's real output limit from model metadata (`sanitizeOutputLimit` from `clients/models-dev-client.ts`, 8192 fallback) and the operator ceiling — don't hardcode max-token values. ## Validation ```bash cd backend && npx vitest run src/routes/proxy/routes/provider-matrix.test.ts cd backend && npx vitest run src/routes/proxy/adapters/*.test.ts # adapter/translator unit tests pnpm type-check ``` - Manual end-to-end check: `PROVIDER_SMOKE_TEST.md` (repo root of `platform/`) is a browser-automation smoke runbook covering chat, policies, TOON, and proxy flows — run it after provider/proxy changes that unit tests can't cover. ## Related skills - `archestra-dev-backend` — general route/codegen/permission conventions (route shape, `RouteId`, endpoint permissions). - `archestra-dev-backend-tests` — vitest projects, mocking rules, DB fixtures for the tests above.