Tool Sp Setup Local Agents

Setup Local Agents

Purpose

Turn "is my local setup any good?" into a measured answer and a short list of fixes. The audit reads the machine and the running server rather than asking the user what they configured, because the common failure is a setting that was never read by the server at all.

When to use this skill

• Setting up a local model for an agent on a specific machine.
• "Is this machine's setup right?" or "why is my local agent so slow?"
• A local agent is silent for minutes, never calls tools, or truncates mid-session.
• Reviewing an existing Ollama configuration that someone set up a while ago.
• Deciding which class of tool this hardware should run.

Scope boundaries

This tool audits and fixes one machine's setup. It owns the check IDs and the remediation order.

• ref-sp-agents-local-setup (hard dependency) owns the rules: the fit arithmetic, the harness classes, the per-runtime and per-client detail, and the model table. Read it for why a check exists; use this tool to run the checks against a real machine.
• tool-sp-setup-agent-repo wires a repository to agent clients. Different subject: that one is about files in a repo, this one is about a host's inference stack.

First step

Run the audit before reading anything else or changing a setting:

python3 <skill-dir>/scripts/audit_local_setup.py

It is read-only, prints one line per check plus the server configuration it saw, and exits non-zero when a check failed. Flags: --class agent|edit-format|fim to audit against a target rather than what the hardware can afford, --json when the output feeds another step, --only H2,S2 to re-run a subset after a fix. uv run <skill-dir>/scripts/audit_local_setup.py works too; the script is stdlib-only with PEP 723 metadata.

Do not re-derive this by hand with systemctl cat, nvidia-smi, and ollama list. That is the slow path the script replaces, and hand-reading misses the variable that is set but undocumented.

Check IDs

Per-check fix procedures are in ./references/remediation.md. Load it when acting on a finding.

Core workflow

1. Audit. Run the script. Note every non-pass check.
2. Settle the harness class first (H2). It changes what every later check means: a context of 32K is a failure for an agent and correct for autocomplete. If the hardware cannot serve the class the user asked for, say so before fixing anything else, and get agreement on the class. The classes and their floors are in ref-sp-agents-local-setup.
3. Read the existing configuration as intent, not as error. A 0.0.0.0 binding may be a deliberate choice to reach the server from a phone. Report it as a finding to confirm, not a mistake to silently revert.
4. Confirm before changing anything that reaches outside the machine or that restarts a service other people may be using. Editing a service unit and restarting the server interrupts whoever is mid-session on it.
5. Fix in order: harness class → server configuration → security → models → client. Each stage assumes the previous one. Procedures in ./references/remediation.md.
6. Verify empirically, not by re-reading the config. See below.
7. Re-run the audit and report the remaining non-pass checks honestly, including any the user declined and why.

Verify empirically

A configuration file is a claim; the loaded model is the evidence. After any change:

ollama ps

PROCESSOR should read 100% GPU. CONTEXT should be the value you set. If CONTEXT still shows the old number, the server never saw the change, which is the single most common outcome of editing Ollama configuration.

Then prove the harness class actually works:

• agent: ask it to read a named file, and confirm a tool is invoked rather than described. Text that narrates a tool call is a failure.
• edit-format: ask for a one-line edit and confirm the edit block applies cleanly.
• fim: type in an editor and confirm a completion appears within a few hundred milliseconds.

Gotchas

• Shell exports do not configure the server. Ollama runs as a background service and reads the service environment. export OLLAMA_CONTEXT_LENGTH=65536 in a terminal changes nothing. This is the most common false fix; the audit reads the service config precisely because of it.
• A silent first turn is prefill, not a hang. Large system prompts take minutes on slow hardware. Raise the client's stream timeout before diagnosing anything as broken.
• Undocumented variables look authoritative. S6 exists because a setting from an older Ollama survives upgrades in the service file and quietly does nothing. Do not assume a variable works because someone set it; check it against the current documented list.
• A model with a tools capability can still fail under load. Tool calling degrades as the system prompt grows. M2 is necessary, not sufficient. Test it.
• ollama launch <client> overwrites that client's configuration. Say so before running it against a client the user configured by hand. --restore is the undo.
• Detecting a client on PATH is not the same as it being wired. C1 reports presence only.
• Do not tune constants to one machine. If a prediction is off, report the gap rather than editing the estimate to match a single observation.

Validation

• Re-run audit_local_setup.py; every check is pass, info, or an explicitly accepted exception.
• ollama ps shows the intended context and 100% GPU for the chosen model.
• The harness class was proven by a real task, not by a configuration read.
• Any check the user declined is reported as declined, not quietly dropped.

References

• Read ./references/remediation.md for the per-check fix procedure, in the order to apply them.
• Read $SKILLS_FOLDER/ref-sp-agents-local-setup/SKILL.md for the rules behind every check: fit arithmetic, harness classes, and the model table.
• Run $SKILLS_FOLDER/ref-sp-agents-local-setup/scripts/local_model_fit.py to size specific models against this machine. Pass --context and --kv-bytes to match the server's real settings, and --config with a model's config.json for an exact rather than estimated cache figure.
• Run ./scripts/audit_local_setup.py for the audit itself; --help lists the flags.