Local LLM Agentic Coding

Local LLM → Claude Code (or OpenCode)

Connect a coding agent to a locally-served open model. One Ollama server, two doors: Claude Code speaks Ollama's Anthropic API (/v1/messages, since Ollama v0.14+) via three env vars; OpenCode speaks Ollama's OpenAI-compatible API (/v1) via a opencode.json provider block. Either way, no proxy is needed (no LiteLLM, no claude-code-router).

The easy button (offer it first, then open the hood): Ollama ships one-command setup — ollama launch claude and ollama launch opencode wire and start the tool for you. The scripts below are the understand-every-step path, plus the VPS-lean install and the honest verdict the easy button skips. For the OpenCode door specifically, see references/opencode-wiring.md.

Prerequisite: a coding agent must be installed

The user needs the target agent already installed — Claude Code (claude) or OpenCode (opencode). This skill is normally invoked BY that agent, so the driver is present. But if the user asks to wire a tool that is NOT installed (for example, setting up Claude Code while driving OpenCode), tell them to install it first — Claude Code: https://docs.claude.com/en/docs/claude-code, OpenCode: https://opencode.ai/docs — before wiring. launch.sh and launch-opencode.sh both guard for this and exit with that pointer if the CLI is missing.

The one thing to say up front (set expectations honestly)

Making this work and making it usable are different. Two independent thresholds must BOTH be cleared, and cheap hardware usually misses both:

1. Capability — the model must emit valid structured tool calls, turn after turn. Small models (≤1.7B) malform them (args object vs string, skipped tools). Fixed by a bigger model (≥14B, ideally a 30B-class MoE), NOT by faster hardware.
2. Throughput — the box must chew Claude Code's ~18,000-token system prompt every turn fast enough to beat its request timeout. CPU-only = minutes/turn. Fixed by a GPU, NOT by a smarter model.

Say this to the user before installing, then read references/hardware-sizing.md and run scripts/check-hardware.sh to give them a specific, honest verdict for their box.

Workflow

Run these in order. Each script is idempotent and needs no root (user-space install).

1. Reality-check the hardware

bash scripts/check-hardware.sh

Reports CPU/RAM/GPU/disk and recommends a model tier (or warns it'll only be a learning rig). Details + the sizing table: references/hardware-sizing.md.

2. Install Ollama

bash scripts/install-ollama.sh

This script is Linux/VPS-only — it streams the release tarball through zstd and excludes the CUDA/ROCm/MLX GPU libraries, so a CPU box installs in ~112 MB instead of ~3 GB. Starts ollama serve on :11434. (On a GPU box, drop the --exclude flags — see the script's header.)

On macOS or Windows, install the official Ollama app from ollama.com/download (or brew install ollama), which handles GPU/Metal automatically — then skip to step 3. Simplest of all on any OS: ollama launch claude / ollama launch opencode installs, pulls, wires, and launches in one command.

3. Pull + tune a model

bash scripts/make-model.sh <base-model> <derived-name>
# e.g. bash scripts/make-model.sh qwen3:14b qwen3-cc

Pulls the base model and builds a derived model with a Modelfile that sets num_ctx large enough for Claude Code's ~18K prompt (default 32768). This is the #1 cause of "local Claude Code is broken" — the Ollama default of 4096 silently truncates the prompt.

4. Launch a coding agent against it

Claude Code (Anthropic door):

bash scripts/launch.sh <derived-name>     # e.g. qwen3-cc

Exports the env vars and runs claude --model <name>. The wiring:

export ANTHROPIC_BASE_URL="http://localhost:11434"   # bare host:port; CC appends /v1/messages
export ANTHROPIC_AUTH_TOKEN="ollama"   # required placeholder, accepted-but-ignored
unset ANTHROPIC_API_KEY                # canonical (empty string also works). If a real key is
                                       # set, CC prefers it and tries to use it as a real key → fails.

On slow (CPU) boxes the launcher also raises API_TIMEOUT_MS so turns don't 500 mid-prefill. For a persistent wiring that applies everywhere (including background agents), put ANTHROPIC_BASE_URL / ANTHROPIC_AUTH_TOKEN / API_TIMEOUT_MS in the env block of ~/.claude/settings.json instead of shell exports.

OpenCode (OpenAI door):

bash scripts/launch-opencode.sh <derived-name>   # writes opencode.json, then launches

OpenCode does NOT use the env vars — it needs a opencode.json provider block pointed at http://localhost:11434/v1 (note the required /v1). Full details + gotchas: references/opencode-wiring.md.

5. Verify end-to-end

bash scripts/verify.sh <derived-name>

Tests, in order: native /v1/messages chat → a real tool call → count_tokens (a 404 here is FINE on v0.14+, not a hang) → measures prompt-eval tok/s so you can predict per-turn latency.

Hard-won gotchas (all verified the hard way)

• No proxy needed. Ollama v0.14+ serves the Anthropic API natively. Do NOT install LiteLLM or claude-code-router for Ollama. (LM Studio ≥0.4.1 and llama.cpp's llama-server are ALSO native now; only vLLM still needs a proxy — see references/other-backends.md.)
• num_ctx ≥ 32768. Claude Code's system prompt is ~18K tokens. The 4096 default breaks it.
• Reasoning models (Qwen3, DeepSeek-R1) think by default and it's slow. You CANNOT disable thinking through the Anthropic /v1/messages path (no think field passes; there's no PARAMETER think in Modelfiles; template edits don't change server-side parse). For Claude Code, either accept the overhead or choose a non-reasoning model. For YOUR OWN agents, call Ollama's native /api/chat with "think": false — that DOES disable it (big speed win).
• count_tokens returns 404 on v0.14+ — harmless; Claude Code falls back to local estimation. A hang (not a 404) means your Ollama is too old — upgrade.
• Prompt cache helps a lot. Turn 2+ reuses the cached ~18K system-prompt prefix, so only the first turn pays full prefill. Any context growth past the cached prefix re-triggers eval.
• Install lean or run out of disk. The full tarball unpacks to ~3 GB (GPU libs); stripping them → ~112 MB. Critical on small VPSs.

When the user's box can't do it

Be direct: it's a great learning rig (the wiring, API, and agent-loop mechanics all work), but not a usable coding agent. Point them to a 14B–30B model on a GPU (even a modest rented one) — same Ollama, same three env vars, the setup transfers 1:1. See references/hardware-sizing.md.

References

• references/opencode-wiring.md — the OpenCode door (OpenAI endpoint, opencode.json), the two-door contrast, gotchas
• references/hardware-sizing.md — model-by-RAM/VRAM table, the two-threshold model, GPU guidance
• references/troubleshooting.md — every failure mode seen and its fix (timeouts, 404s, truncation, malformed tool calls)
• references/other-backends.md — LM Studio / llama.cpp (native Anthropic API, no proxy) and vLLM (needs a proxy)