Ollama Deployment Skill
Overview
Use this skill for deploying and managing Ollama (local LLM runtime) on fleet hosts.
Tools Required
- Bash - Run ollama commands, systemctl
- SSH - Connect to hosts for direct access
- WebFetch - Download Ollama binaries, check releases
- Glob/Grep - Find existing Ollama installations
- Read - Check configs, model availability
- Write/Edit - Create systemd units, configs
Key Principles
1. Architecture Matters
- Raspberry Pis: use
ollama-linux-arm64.tgz - x86_64 hosts: use
ollama-linux-amd64.tgz - Verify with:
file /usr/local/bin/ollama
2. Staged Downloads for Large Files
Ollama binary is ~1.6GB. Download once locally, then distribute:
# Download to controller
curl -fsSL https://github.com/ollama/ollama/releases/download/v0.13.1/ollama-linux-arm64.tgz -o /tmp/ollama.tgz
# Distribute via Ansible copy (parallel, fast LAN)
ansible <hosts> -i inventory -b -m copy -a "src=/tmp/ollama.tgz dest=/tmp/ollama.tgz"
# Extract on hosts
ansible <hosts> -i inventory -b -m shell -a "tar -xzf /tmp/ollama.tgz -C /usr/local"
3. Models Need Separate Pull
After installing Ollama binary, pull models:
ansible <hosts> -i inventory -b -m shell -a "/usr/local/bin/ollama pull kimi-k2.5:cloud"
4. Use Systemd for Service Management
ansible <hosts> -i inventory -b -m systemd -a "name=ollama state=started enabled=yes"
Common Playbooks
Install Ollama
ansible-playbook -i inventory 02-ollama.yml
Check Status
ansible <hosts> -i inventory -m shell -a "/usr/local/bin/ollama list"
ansible <hosts> -i inventory -m systemd -a "name=ollama state=started"
Known Issues
Wrong Architecture
- Problem: juju4.ollama role downloads AMD64 by default
- Fix: Manually specify ARM64 URL in playbook
Model Download Fails
- Problem: Permission errors when running
ollama pullas _ollama user - Fix: Run
ollama pullas ubuntu user or directly without become_user
Service Not Starting
- Check:
journalctl -u ollama -n 50 - Verify binary:
file /usr/local/bin/ollama
Reference Files
$HOME/rust-town/irclaw/ansible/02-ollama.yml
Usage
- Audit first: Check if Ollama already installed
- Stage download: Get binary once, distribute
- Extract & start: Use systemd
- Pull models: Pull required models (kimi-k2.5:cloud)