# Local Models

> Running AI models locally with Ollama and llama.cpp. Free inference, privacy, custom model deployment. Use for cost-free routine tasks or private data processing. Don't use when quality of Opus is needed.

- Skill: `neuron-one/local-models` (Agent Skill)
- Install (CLI): `npx skillmds@latest add neuron-one/local-models`
- Raw SKILL.md: https://api.skillmd.com/api/skills/neuron-one/local-models/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: neuron-one (https://skillmd.com/u/neuron-one)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/neuron-one/local-models

---


# Local Model Deployment

## Ollama Setup
```bash
# Install model
ollama pull qwen2.5:7b

# Run
ollama run qwen2.5:7b

# API endpoint
curl http://localhost:11434/api/generate
```

## llama.cpp (for Claude Code integration)
```bash
# Start server
llama-server --model model.gguf --port 8001

# Point Claude Code to local
ANTHROPIC_BASE_URL=http://localhost:8001
```

## Model Selection
- Routine tasks: Qwen 3B (fast, light)
- Code generation: Qwen 7B or CodeLlama 7B
- Reasoning: Qwen 27B (heavy but capable)

## When to Use Local vs Cloud
- Private data processing -> local
- Cost-free batch operations -> local
- Complex reasoning -> cloud (Opus)
- Critical decisions -> cloud (Opus)