prism-shadow Ollama Serving
Ollama runs open-weight models locally with automatic GPU detection and an OpenAI-compatible API on http://localhost:11434.
Before you start
If the user's message only invokes this skill (e.g. "use ollama skill") without a concrete request, ask the user what they want. Do not run any command until the goal is clear.
Ask the user which model to run; if they have no preference, recommend the small default Qwen/Qwen3.5-0.8B (ollama pull qwen3.5:0.8b). The model must fit the machine's RAM/VRAM.
Ollama runs everywhere — macOS, Linux and Windows, on CPUs as well as NVIDIA/AMD GPUs — so engine choice follows the user's preference: Ollama is the simple default, while vLLM targets high-throughput GPU serving. Check the current state first:
ollama --version # is Ollama installed?
ollama ps # is the service already serving models?
If port 11434 is already serving, reuse that instance — never kill an existing Ollama process.
Suggested workflow
- Ask the user which model to run; with no preference, recommend Qwen/Qwen3.5-0.8B (
qwen3.5:0.8b). - Pick the engine the user prefers: Ollama is the default; vLLM covers high-throughput GPU serving.
- Install Ollama if missing, then
ollama pull qwen3.5:0.8b. - Verify with
curl http://localhost:11434/v1/models. - Register the endpoint:
penguin config model add ... --client-type openai-chat --base-url http://localhost:11434/v1— a pulled Ollama model is not visible to Penguin until added. - Confirm the new entry with
penguin config model list.
Install
curl -fsSL https://ollama.com/install.sh | sh # Linux; macOS/Windows use the desktop app
The service then listens on http://localhost:11434.
Pull and run
ollama pull qwen3.5:0.8b # download a model
ollama run qwen3.5:0.8b # interactive chat (pulls first if missing)
ollama list # downloaded models
ollama ps # models loaded in memory
ollama stop qwen3.5:0.8b # unload a model
OpenAI-compatible endpoint
The endpoint is http://localhost:11434/v1; any non-empty API key is accepted (conventionally ollama):
curl http://localhost:11434/v1/models
Context length
The default context window is small, and agent sessions need a large one. Raise it in the server's environment:
OLLAMA_CONTEXT_LENGTH=32768 ollama serve # systemd service: set it via `systemctl edit ollama`
Or bake it into a model variant with a Modelfile:
FROM qwen3.5:0.8b
PARAMETER num_ctx 32768
ollama create qwen3.5-32k -f Modelfile
Register with PenguinHarness
Model configuration is the penguin CLI's job — penguin config model add registers an endpoint and penguin config model list shows what has been registered. A pulled Ollama model is not visible to Penguin until you add it:
penguin config model add --provider custom --client-type openai-chat \
--base-url http://localhost:11434/v1 --model-id qwen3.5:0.8b --api-key ollama
penguin config model list # the new entry should now be listed