# Local LLM > Use when the user explicitly asks about local LLMs, LM Studio, local model inventory, local model chat, local model benchmarking, model role curation, or configuring private/local model endpoints. Provides public-safe mechanics and model index guidance for locally hosted or trusted-network OpenAI-compatible models without deciding when other domain skills should use a local model. - Skill: `cbusillo/local-llm` (Agent Skill, multi-file: 14 files) - Install (CLI): `npx skillmds@latest add cbusillo/local-llm` - Raw SKILL.md: https://api.skillmd.com/api/skills/cbusillo/local-llm/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: AI & ML - Author: cbusillo (https://skillmd.com/u/cbusillo) - Updated: 2026-09-21 - Page: https://skillmd.com/skills/cbusillo/local-llm --- # Local LLM This skill documents how to inventory, call, benchmark, and curate local or trusted-network LLMs. It does not decide when every other skill should use a local model; domain skills keep their own evidence and prompting rules. ## Configuration Public model curation lives in `references/model-index.yaml`. Machine-specific endpoint choices, trusted LAN settings, installed model IDs, and user overrides live in `.local/local-llm.yaml`. Use repo-root `.local/` for private config. Keeping private files out of skill folders preserves skill portability while still giving each skill a clear private filename. Copy the shape from `references/local-llm.local.example.yaml` when creating private config. Do not commit real hostnames, LAN addresses, tokens, installed-model inventories tied to a private machine, or private benchmark notes. When endpoint URLs, ports, trust scope, or preferred model IDs change, update the private config before data-heavy runs and run a lightweight inventory check. Bump the config `version` only for schema changes, not routine endpoint or model rotation. ## Locality - `localhost`: same machine as the agent, typically `http://127.0.0.1:1234/v1`. - `trusted_lan`: a private LAN or trusted mesh/VPN endpoint. Treat as acceptable for private work when local config says the network is trusted. - `remote_private`: a private endpoint reached over an authenticated tunnel or managed private network. - `cloud`: an internet-hosted provider, even when the API is OpenAI-compatible. The endpoint determines data handling. A model name alone does not prove locality. ## Sensitive Material Localhost and trusted-LAN models are preferred over cloud APIs for private or sensitive local material when the runtime is trusted. The restriction is on durable artifacts, not on local inference itself. When local config marks an endpoint as `locality: localhost` or `locality: trusted_lan` with private/local trust, it may receive original private local inputs for the current task, including unredacted rollout snippets, memory drafts, local config extracts, and repo-specific context. Do not redact away useful local signal just because the input would be unsafe for a public issue or cloud model. Keep these boundaries: - Do not commit raw private prompts, outputs, traces, memory drafts, hostnames, tokens, or account details. - Do not paste private model output into public issues, PR comments, docs, or skill examples. - Check whether LM Studio logging/history is enabled before sending material that should not be retained. - Strip or withhold obvious secrets such as tokens, passwords, API keys, and private keys even for trusted local models unless the user explicitly asks for secret analysis. - Redact, summarize, or avoid sensitive material for `cloud`, unknown, disabled, or untrusted endpoints unless the user explicitly approves that provider and context. ## Model Index The public model index can include curated model IDs, model roles, strengths, weaknesses, minimum token budgets, and last-tested notes. It is allowed to be opinionated, but it should stay public-safe and time-stamped. Private config may add, remove, or override role choices. Prefer roles over hard-coded model names in task-specific scripts when the role has already been curated locally. ## Benchmarking Do not benchmark routinely. Benchmark when adding a model, choosing a model for a new role, investigating odd behavior, after model/runtime updates, or when latency/cold-load behavior matters. Benchmark output is local machine evidence, not public truth. Keep private endpoint details and raw sensitive prompts out of committed results. ## LM Studio Mechanics LM Studio exposes local models through OpenAI-compatible endpoints such as `/v1/models` and `/v1/chat/completions`, plus native lifecycle endpoints under `/api/v1`. Use the bundled scripts as the normal path so endpoint selection, trust/locality, tokens, TTL, context, warm-up, and redacted reporting stay tied to `.local/local-llm.yaml`. Inventory is catalog and runtime evidence, not a universal loaded-state proof. `lm_studio_inventory.py` lists `/v1/models` API-visible models and, for `provider: lm_studio`, attempts the native runtime model endpoint. Treat loaded instance counts as authoritative only when the native runtime response provides them; otherwise use a harmless warm-up chat and the response `served_model` field before sending private prompts. For ordinary local runs, prefer API JIT loading through `lm_studio_chat.py --load-policy jit_chat --ttl --warmup`. The warm-up request uses non-sensitive text first, can absorb cold-load latency, and records the served model before private content is sent. TTL on `/v1/chat/completions` is an LM Studio JIT hint; already-loaded manual or explicit instances may keep their load-time TTL instead of honoring per-request TTL. For deterministic long-context or load-parameter-sensitive runs, use `--load-policy api_explicit` with role `context_length` (or `load.context_length`) or CLI `--context-length`, plus role `load.flash_attention` or CLI `--flash-attention`. Use `--unload-after` when cleanup should be immediate. This uses `/api/v1/models/load` before chat and unloads only by the returned `instance_id`; do not attempt model-name-only unloads. Before warm-up or task content is sent, the helper requires a nonblank returned instance ID without surrounding whitespace. It also verifies typed `status: loaded` plus each requested load setting against the typed `load_config` echoed by LM Studio. Both warm-up and task chat are then bound to that exact instance ID while output keeps the originally requested and served model provenance. Missing, differently typed, or mismatched load evidence fails the run rather than qualifying the requested context. When `--unload-after` is set, later qualification, warm-up, or chat failures still trigger cleanup by that exact returned instance ID. On the tested LM Studio build, native load accepts context and flash-attention options but rejects TTL, so TTL remains a JIT-chat hint rather than an explicit-load cleanup mechanism. Successful JSON output retains provider-supplied usage for the task response without synthesizing absent token values. Warm-up usage, when supplied, remains separate under lifecycle evidence so it is not mistaken for task usage. Failed explicit runs retain the lifecycle evidence gathered before the error, including load verification and requested cleanup results; task usage remains available when a later content or cleanup check fails. The `lms` CLI is an operator fallback for local diagnostics, downloads, or manual recovery when the HTTP lifecycle path is unavailable. For non-local LM Studio instances, CLI commands must target the serving host explicitly, so API helpers are preferred for trusted LAN and remote-private endpoints. Large reasoning models may return no assistant content when `max_tokens` is too low because the budget is consumed by reasoning. Increase `max_tokens` for deep models before declaring them unusable. If a configured local endpoint is unavailable, say so plainly and keep private inputs local. Do not silently fall back to cloud. For private workflows, either retry after the local endpoint is healthy, switch to another configured trusted local/private endpoint, or ask for explicit approval for the exact cloud destination and input scope. ## Local Agent Execution Use `lm_studio_chat.py` for direct local-model inference. When a task needs an agent to use tools, read [local agent execution](references/local-agent.md) and use `local_codex_agent.py` with the selected Codex or Codex Lab host and a verified local model. Reuse a host/model already established by the task; ordinary implementation choices do not require a new user question. Every Code is retired. `local_code_agent.py` is a retirement guard for old callers; do not restore that runtime or rename its executable to make old configuration work. Current hosts use different homes, arguments, and provider contracts. Direct model replies, isolated local agent runs, and the user's ordinary configured host session provide different evidence. ## Related Skills Other skills may reference this one for local model mechanics and model selection. They should keep their own domain-specific evidence rules and remain usable when `local-llm` is absent.