Freellama

FreeLlama

Operate FreeLlama as governed local-model delegation, not as a frontier-model replacement. MCP tools: doctor, models, run_task, ollama_manage, ollama_delete, delegate_research; npx @octocodeai/freellama tools maps them to CLI equivalents.

Flow: PREREQUISITE → DIAGNOSE → DECIDE → EXECUTE → VERIFY

Know who owns execution

FreeLlama is a governed router, not an unrestricted hardware scheduler. A CPU preference cannot move an arbitrary tag: the operator must have assigned that exact model to the CPU Ollama process. One managed call contains one task; only the calling agent knows which tasks are independent.

Prerequisite and diagnose

1. Call doctor. FreeLlama does not install or replace Ollama; if Ollama is missing or unreachable, direct the operator to https://ollama.com/download or ollama serve, then stop—do not recommend or pull a model.
2. Read doctor.machine.memory_bytes, then models{view:"installed"} and models{view:"resident"}. Treat execution.backend as configuration and execution.observation.processor as physical evidence. A mismatch or partial placement warning invalidates feedback. For authenticated serve, set FREELLAMA_AUTH_TOKEN_FILE to the same permission-restricted file used at startup.
3. Check /_freellama/v1/health admission slots. Zero slots means queueing up to the configured wait, not a reservation; a 503 is load shedding, so retry or lower fan-out.

Decide

Prefer deterministic search when an identifier is known; embeddings are for no-keyword similarity. Treat every measured number as one-machine evidence, not a default. Models at or below 12B were unreliable for broad research in the recorded trials.

Before offloading, map the work to completion, coding, code_repair, tools, browser, vision, embedding, or long_context, plus required capabilities and context. Prefer a qualified installed model; never infer inventory from a model family name or this skill's examples.

Execute one flow

A — grounded files

Call delegate_research{question,workspacePath,model?,adapter?,executionPreference?,minPlacementEvidence?,agent?} with one narrow lookup over 1–5 source files. It needs serve: every model turn is a managed coding task with admission and physical-placement receipts. Keep the path inside FREELLAMA_MCP_ALLOWED_ROOTS; Bash is the measured default, Octocode is opt-in for structural/LSP search. Read verification, full citations[], contextManagement, and execution.receipts. Retry isError once, then do the work yourself. Load references/task-delegation.md for the measured boundary and references/context-management.md before tuning compaction.

B — supplied content or vision

Call run_task{task,prompt|messages,images?,model?,executionPreference?,minPlacementEvidence?,keepAlive?,minConfidence?,preview?}. It has no file access. preview:true is free; minConfidence gates quality evidence. minPlacementEvidence:"observed" additionally refuses cold, mixed, or mismatched processor placement; warm once with "configured", inspect the receipt, then require "observed". Images are base64 without a data-URI prefix and require an explicitly trialled vision model. GLM-OCR needs a model-specific repetition guard: options.stop:["```"] handles fence tails; add "\n" only for a one-line transcription contract. Load references/model-selection.md before model choice.

C — embeddings

Batch run_task{task:"embedding",input:[...]} and leave returnEmbeddings:false unless code, not the orchestrator, needs vectors. Use embeddings for grouping, dedup candidates, classification, and similarity—not identifier search. examples/local-rag.sh is the runnable pattern.

D — choose or install a model

Ask for missing workload/modality, quality, latency, context, privacy, download, disk, and memory constraints. Diagnose the host, prefer qualified installed models, then search models{view:"library"} by family and inspect exact pullable tags with models{view:"detail",model}. Present at most two evidence-backed candidates. Ask approval for one exact tag and size before ollama_manage{action:"pull"}; discovery never grants installation permission. Load references/model-selection.md, then references/ollama-config.md for fit.

E — diagnose a failure

Run doctor, resident models, then scripts/check.sh (read-only; exit 0 means healthy). Load references/troubleshooting.md for symptom routing. Never treat a 503, confidence refusal, CPU spill, or proxy/serve 404 as the same failure.

Verify every result

Keep pinnedOverflow:"error"; clipping the system prompt/question requires explicit human acceptance. Accept adaptive placement feedback only when execution.observation.status:"verified"; for keepAlive:"0", also inspect the observe-then-unload execution.lifecycle. Persist model-specific token calibration and bounded placement feedback; use ephemeral modes only for disposable tests. An assigned CPU backend can still execute an MLX model fully on GPU. Never co-resident two large models without memory arithmetic. Never delete ~/.ollama/models files directly; ollama_delete needs exact-tag approval in the current conversation. Operators own endpoints, CPU tag assignments, Ollama settings, pulls, stops, and deletes.

Routes and deterministic helpers

Use current doctor output, installed server behavior, and upstream Ollama docs before changing runtime settings. Read one routed reference at a time; detailed trial transcripts live under assets/evidence/ and should be opened only when auditing a quantitative claim.