# Local LLM Agentic Coding > Set up a FREE local LLM (via Ollama) and connect Claude Code or OpenCode to it so a coding agent runs offline with no API costs. Use whenever the user wants to run Claude Code/OpenCode on a local or self-hosted model, use Ollama/Qwen/Llama/Mistral/DeepSeek with a coding agent, get a "free" or "offline" coding agent, avoid Anthropic API costs, point ANTHROPIC_BASE_URL at localhost, or asks "can my machine/VPS run a local LLM for Claude Code". Trigger even on "run Claude Code locally", "local model for coding", "self-hosted Claude Code", or a named local model (qwen3, qwen3-coder, llama3, deepseek-coder, gpt-oss). Handles install, model tuning, wiring for both tools, and an honest hardware reality-check (a capable model AND fast hardware are BOTH required). - Skill: `panaversity/local-llm-agentic-coding` (Agent Skill, multi-file: 3 files) - Install (CLI): `npx skillmds@latest add panaversity/local-llm-agentic-coding` - Raw SKILL.md: https://api.skillmd.com/api/skills/panaversity/local-llm-agentic-coding/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: AI & ML - Author: panaversity (https://skillmd.com/u/panaversity) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/panaversity/local-llm-agentic-coding --- # Local LLM → Claude Code (or OpenCode) Connect a coding agent to a locally-served open model. **One Ollama server, two doors:** Claude Code speaks Ollama's **Anthropic** API (`/v1/messages`, since **Ollama v0.14+**) via three env vars; OpenCode speaks Ollama's **OpenAI-compatible** API (`/v1`) via a `opencode.json` provider block. Either way, **no proxy** is needed (no LiteLLM, no claude-code-router). **The easy button** (offer it first, then open the hood): Ollama ships one-command setup — `ollama launch claude` and `ollama launch opencode` wire and start the tool for you. The scripts below are the *understand-every-step* path, plus the VPS-lean install and the honest verdict the easy button skips. For the OpenCode door specifically, see `references/opencode-wiring.md`. ## Prerequisite: a coding agent must be installed The user needs the target agent already installed — **Claude Code** (`claude`) or **OpenCode** (`opencode`). This skill is normally invoked BY that agent, so the driver is present. But if the user asks to wire a tool that is NOT installed (for example, setting up Claude Code while driving OpenCode), tell them to install it first — Claude Code: , OpenCode: — before wiring. `launch.sh` and `launch-opencode.sh` both guard for this and exit with that pointer if the CLI is missing. ## The one thing to say up front (set expectations honestly) Making this *work* and making it *usable* are different. Two independent thresholds must BOTH be cleared, and cheap hardware usually misses both: 1. **Capability** — the model must emit **valid structured tool calls, turn after turn**. Small models (≤1.7B) malform them (`args` object vs string, skipped tools). Fixed by a **bigger model** (≥14B, ideally a 30B-class MoE), NOT by faster hardware. 2. **Throughput** — the box must chew Claude Code's **~18,000-token system prompt** every turn fast enough to beat its request timeout. CPU-only = minutes/turn. Fixed by a **GPU**, NOT by a smarter model. Say this to the user before installing, then read `references/hardware-sizing.md` and run `scripts/check-hardware.sh` to give them a specific, honest verdict for their box. ## Workflow Run these in order. Each script is idempotent and needs **no root** (user-space install). ### 1. Reality-check the hardware ```bash bash scripts/check-hardware.sh ``` Reports CPU/RAM/GPU/disk and recommends a model tier (or warns it'll only be a learning rig). Details + the sizing table: `references/hardware-sizing.md`. ### 2. Install Ollama ```bash bash scripts/install-ollama.sh ``` **This script is Linux/VPS-only** — it streams the release tarball through `zstd` and **excludes the CUDA/ROCm/MLX GPU libraries**, so a CPU box installs in **~112 MB instead of ~3 GB**. Starts `ollama serve` on `:11434`. (On a GPU box, drop the `--exclude` flags — see the script's header.) **On macOS or Windows**, install the official Ollama app from [ollama.com/download](https://ollama.com/download) (or `brew install ollama`), which handles GPU/Metal automatically — then skip to step 3. Simplest of all on any OS: `ollama launch claude` / `ollama launch opencode` installs, pulls, wires, and launches in one command. ### 3. Pull + tune a model ```bash bash scripts/make-model.sh # e.g. bash scripts/make-model.sh qwen3:14b qwen3-cc ``` Pulls the base model and builds a derived model with a Modelfile that sets **`num_ctx` large enough for Claude Code's ~18K prompt** (default 32768). This is the #1 cause of "local Claude Code is broken" — the Ollama default of 4096 silently truncates the prompt. ### 4. Launch a coding agent against it **Claude Code** (Anthropic door): ```bash bash scripts/launch.sh # e.g. qwen3-cc ``` Exports the env vars and runs `claude --model `. The wiring: ```bash export ANTHROPIC_BASE_URL="http://localhost:11434" # bare host:port; CC appends /v1/messages export ANTHROPIC_AUTH_TOKEN="ollama" # required placeholder, accepted-but-ignored unset ANTHROPIC_API_KEY # canonical (empty string also works). If a real key is # set, CC prefers it and tries to use it as a real key → fails. ``` On slow (CPU) boxes the launcher also raises `API_TIMEOUT_MS` so turns don't 500 mid-prefill. For a **persistent** wiring that applies everywhere (including background agents), put `ANTHROPIC_BASE_URL` / `ANTHROPIC_AUTH_TOKEN` / `API_TIMEOUT_MS` in the `env` block of `~/.claude/settings.json` instead of shell exports. **OpenCode** (OpenAI door): ```bash bash scripts/launch-opencode.sh # writes opencode.json, then launches ``` OpenCode does NOT use the env vars — it needs a `opencode.json` provider block pointed at `http://localhost:11434/v1` (note the required `/v1`). Full details + gotchas: `references/opencode-wiring.md`. ### 5. Verify end-to-end ```bash bash scripts/verify.sh ``` Tests, in order: native `/v1/messages` chat → a real **tool call** → `count_tokens` (a 404 here is FINE on v0.14+, not a hang) → measures prompt-eval tok/s so you can predict per-turn latency. ## Hard-won gotchas (all verified the hard way) - **No proxy needed.** Ollama v0.14+ serves the Anthropic API natively. Do NOT install LiteLLM or claude-code-router for Ollama. (LM Studio ≥0.4.1 and llama.cpp's `llama-server` are ALSO native now; only **vLLM** still needs a proxy — see `references/other-backends.md`.) - **`num_ctx` ≥ 32768.** Claude Code's system prompt is ~18K tokens. The 4096 default breaks it. - **Reasoning models (Qwen3, DeepSeek-R1) think by default and it's slow.** You CANNOT disable thinking through the Anthropic `/v1/messages` path (no `think` field passes; there's no `PARAMETER think` in Modelfiles; template edits don't change server-side parse). For Claude Code, either accept the overhead or choose a non-reasoning model. For YOUR OWN agents, call Ollama's native `/api/chat` with `"think": false` — that DOES disable it (big speed win). - **`count_tokens` returns 404** on v0.14+ — harmless; Claude Code falls back to local estimation. A *hang* (not a 404) means your Ollama is too old — upgrade. - **Prompt cache helps a lot.** Turn 2+ reuses the cached ~18K system-prompt prefix, so only the first turn pays full prefill. Any context growth past the cached prefix re-triggers eval. - **Install lean or run out of disk.** The full tarball unpacks to ~3 GB (GPU libs); stripping them → ~112 MB. Critical on small VPSs. ## When the user's box can't do it Be direct: it's a great **learning rig** (the wiring, API, and agent-loop mechanics all work), but not a usable coding agent. Point them to a **14B–30B model on a GPU** (even a modest rented one) — same Ollama, same three env vars, the setup transfers 1:1. See `references/hardware-sizing.md`. ## References - `references/opencode-wiring.md` — the OpenCode door (OpenAI endpoint, `opencode.json`), the two-door contrast, gotchas - `references/hardware-sizing.md` — model-by-RAM/VRAM table, the two-threshold model, GPU guidance - `references/troubleshooting.md` — every failure mode seen and its fix (timeouts, 404s, truncation, malformed tool calls) - `references/other-backends.md` — LM Studio / llama.cpp (native Anthropic API, no proxy) and vLLM (needs a proxy)