Ollama Setup
Overview
Auto-configure Ollama for local LLM deployment, eliminating hosted API costs and enabling offline AI inference. This skill handles system assessment, model selection based on available hardware (RAM, GPU), installation across macOS/Linux/Docker, and integration with Python, Node.js, and REST API clients.
Prerequisites
- macOS 12+, Linux (Ubuntu 20.04+, Fedora 36+), or Docker runtime
- Minimum 8 GB RAM for 7B parameter models; 16 GB for 13B models; 32 GB+ for 70B models
- Optional: NVIDIA GPU with CUDA drivers for accelerated inference (
nvidia-smi to verify)
- Optional: Apple Silicon (M1/M2/M3) for Metal-accelerated inference on macOS
- Disk space: 4-40 GB depending on model size (quantized weights)
- Package manager:
brew (macOS), curl (Linux), or docker (containerized)
Instructions
- Detect the host operating system and available hardware using
uname -s, free -h (Linux) or vm_stat (macOS), and nvidia-smi (if GPU present)
- Select appropriate models based on available RAM:
- 8 GB: llama3.2:7b (4 GB), mistral:7b (4 GB), phi3:14b (8 GB)
- 16 GB: codellama:13b (7 GB), mixtral:8x7b (26 GB quantized)
- 32 GB+: llama3.2:70b (40 GB), codellama:34b (20 GB)
- Install Ollama using the platform-appropriate method:
- macOS:
brew install ollama && brew services start ollama
- Linux:
curl -fsSL https://ollama.com/install.sh | sh && sudo systemctl start ollama
- Docker:
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
- Pull the recommended model:
ollama pull llama3.2
- Verify the installation by listing available models (
ollama list) and running a test prompt (ollama run llama3.2 "Say hello")
- Confirm the REST API is accessible:
curl http://localhost:11434/api/tags
- Configure integration with the target application using the appropriate client library (Python
ollama, Node.js ollama, or raw HTTP)
- Set up GPU acceleration if NVIDIA or Apple Silicon hardware is detected
- Configure model persistence and cache directory if non-default storage location is required
- Validate end-to-end inference latency and throughput for the selected model
See ${CLAUDE_SKILL_DIR}/references/skill-workflow.md for the detailed workflow with code snippets.
Output
- Ollama installation confirmed and running as a system service or Docker container
- Selected model(s) pulled and cached locally with verified inference capability
- REST API endpoint accessible at
http://localhost:11434
- Integration code snippet for the target language (Python, Node.js, or cURL)
- Hardware assessment report: OS, RAM, GPU availability, recommended models
- Performance baseline: tokens per second for the selected model on local hardware
Error Handling
| Error |
Cause |
Solution |
ollama: command not found |
Installation incomplete or PATH not updated |
Re-run install script; restart shell session; verify /usr/local/bin/ollama exists |
| Model pull fails with timeout |
Network connectivity issue or Ollama registry unreachable |
Check internet connection; retry with ollama pull --insecure behind corporate proxy |
| Out of memory during inference |
Model size exceeds available RAM |
Switch to a smaller quantized model (e.g., 7B instead of 13B); close memory-intensive applications |
| GPU not detected |
CUDA drivers missing or incompatible version |
Install CUDA toolkit >= 11.8; verify with nvidia-smi; restart Ollama service after driver install |
| Port 11434 already in use |
Another service occupying the default Ollama port |
Stop conflicting service; or set OLLAMA_HOST=0.0.0.0:11435 environment variable |
See ${CLAUDE_SKILL_DIR}/references/errors.md for additional error scenarios.
Examples
Scenario 1: Developer Workstation Setup -- Install Ollama on a macOS M2 machine with 16 GB RAM. Pull codellama:13b for code generation tasks. Integrate with a Python FastAPI application using the ollama Python package. Expected throughput: 30-50 tokens/second on Apple Silicon.
Scenario 2: Air-Gapped Server Deployment -- Install Ollama on an offline Ubuntu server via pre-downloaded binary. Transfer model weights via USB. Configure as a systemd service with auto-restart. Serve llama3.2:7b via REST API for internal team use.
Scenario 3: Docker-Based CI Pipeline -- Run Ollama in a Docker container as part of a CI/CD pipeline for automated code review. Pull mistral:7b, expose the API on port 11434, and integrate with a Node.js test harness that sends code diffs for analysis.
Resources
1---2name: ollama-setup3description: Configure auto-configure Ollama when user needs local LLM deployment, free AI alternatives, or wants to eliminate hosted API costs. Trigger phrases: "install ollama", "local AI", "free LLM", "self-hosted AI", "replace OpenAI", "no API costs". Use when appropriate context detected. Trigger with relevant phrases based on skill purpose.4license: MIT5---6# Ollama Setup
7
8## Overview
9
10Auto-configure Ollama for local LLM deployment, eliminating hosted API costs and enabling offline AI inference. This skill handles system assessment, model selection based on available hardware (RAM, GPU), installation across macOS/Linux/Docker, and integration with Python, Node.js, and REST API clients.
11
12## Prerequisites
13
14- macOS 12+, Linux (Ubuntu 20.04+, Fedora 36+), or Docker runtime
15- Minimum 8 GB RAM for 7B parameter models; 16 GB for 13B models; 32 GB+ for 70B models
16- Optional: NVIDIA GPU with CUDA drivers for accelerated inference (`nvidia-smi` to verify)
17- Optional: Apple Silicon (M1/M2/M3) for Metal-accelerated inference on macOS
18- Disk space: 4-40 GB depending on model size (quantized weights)
19- Package manager: `brew` (macOS), `curl` (Linux), or `docker` (containerized)
20
21## Instructions
22
231. Detect the host operating system and available hardware using `uname -s`, `free -h` (Linux) or `vm_stat` (macOS), and `nvidia-smi` (if GPU present)
242. Select appropriate models based on available RAM:
25 - **8 GB**: llama3.2:7b (4 GB), mistral:7b (4 GB), phi3:14b (8 GB)
26 - **16 GB**: codellama:13b (7 GB), mixtral:8x7b (26 GB quantized)
27 - **32 GB+**: llama3.2:70b (40 GB), codellama:34b (20 GB)
283. Install Ollama using the platform-appropriate method:
29 - macOS: `brew install ollama && brew services start ollama`
30 - Linux: `curl -fsSL https://ollama.com/install.sh | sh && sudo systemctl start ollama`
31 - Docker: `docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama`
324. Pull the recommended model: `ollama pull llama3.2`
335. Verify the installation by listing available models (`ollama list`) and running a test prompt (`ollama run llama3.2 "Say hello"`)
346. Confirm the REST API is accessible: `curl http://localhost:11434/api/tags`
357. Configure integration with the target application using the appropriate client library (Python `ollama`, Node.js `ollama`, or raw HTTP)
368. Set up GPU acceleration if NVIDIA or Apple Silicon hardware is detected
379. Configure model persistence and cache directory if non-default storage location is required
3810. Validate end-to-end inference latency and throughput for the selected model
39
40See `${CLAUDE_SKILL_DIR}/references/skill-workflow.md` for the detailed workflow with code snippets.
41
42## Output
43
44- Ollama installation confirmed and running as a system service or Docker container
45- Selected model(s) pulled and cached locally with verified inference capability
46- REST API endpoint accessible at `http://localhost:11434`
47- Integration code snippet for the target language (Python, Node.js, or cURL)
48- Hardware assessment report: OS, RAM, GPU availability, recommended models
49- Performance baseline: tokens per second for the selected model on local hardware
50
51## Error Handling
52
53| Error | Cause | Solution |
54|-------|-------|----------|
55| `ollama: command not found` | Installation incomplete or PATH not updated | Re-run install script; restart shell session; verify `/usr/local/bin/ollama` exists |
56| Model pull fails with timeout | Network connectivity issue or Ollama registry unreachable | Check internet connection; retry with `ollama pull --insecure` behind corporate proxy |
57| Out of memory during inference | Model size exceeds available RAM | Switch to a smaller quantized model (e.g., 7B instead of 13B); close memory-intensive applications |
58| GPU not detected | CUDA drivers missing or incompatible version | Install CUDA toolkit >= 11.8; verify with `nvidia-smi`; restart Ollama service after driver install |
59| Port 11434 already in use | Another service occupying the default Ollama port | Stop conflicting service; or set `OLLAMA_HOST=0.0.0.0:11435` environment variable |
60
61See `${CLAUDE_SKILL_DIR}/references/errors.md` for additional error scenarios.
62
63## Examples
64
65**Scenario 1: Developer Workstation Setup** -- Install Ollama on a macOS M2 machine with 16 GB RAM. Pull codellama:13b for code generation tasks. Integrate with a Python FastAPI application using the `ollama` Python package. Expected throughput: 30-50 tokens/second on Apple Silicon.
66
67**Scenario 2: Air-Gapped Server Deployment** -- Install Ollama on an offline Ubuntu server via pre-downloaded binary. Transfer model weights via USB. Configure as a systemd service with auto-restart. Serve llama3.2:7b via REST API for internal team use.
68
69**Scenario 3: Docker-Based CI Pipeline** -- Run Ollama in a Docker container as part of a CI/CD pipeline for automated code review. Pull mistral:7b, expose the API on port 11434, and integrate with a Node.js test harness that sends code diffs for analysis.
70
71## Resources
72
73- [Ollama Official Documentation](https://ollama.com) -- installation, model library, API reference
74- [Ollama Model Library](https://ollama.com/library) -- available models with size and capability details
75- [Ollama Python Client](https://github.com/ollama/ollama-python) -- Python SDK for local inference
76- [Ollama JavaScript Client](https://github.com/ollama/ollama-js) -- Node.js SDK for local inference
77- Hardware sizing guide: RAM requirements by model parameter count and quantization level
micsapp