Profile My skills Saved Collections Edit profile Submit a skill

Local Models

Running AI models locally with Ollama and llama.cpp. Free inference, privacy, custom model deployment. Use for cost-free routine tasks or private data processing. Don't use when quality of Opus is needed.

neuron-one d1d914a 931 B Updated
AI & ML

File contents

Local Model Deployment

Ollama Setup

# Install model
ollama pull qwen2.5:7b

# Run
ollama run qwen2.5:7b

# API endpoint
curl http://localhost:11434/api/generate

llama.cpp (for Claude Code integration)

# Start server
llama-server --model model.gguf --port 8001

# Point Claude Code to local
ANTHROPIC_BASE_URL=http://localhost:8001

Model Selection

  • Routine tasks: Qwen 3B (fast, light)
  • Code generation: Qwen 7B or CodeLlama 7B
  • Reasoning: Qwen 27B (heavy but capable)

When to Use Local vs Cloud

  • Private data processing -> local
  • Cost-free batch operations -> local
  • Complex reasoning -> cloud (Opus)
  • Critical decisions -> cloud (Opus)

neuron-one/godmode/tree/main/skills/multimodel/local-models commit d1d914a17d

Browse category

Frequently asked questions

npx skillmds@latest add neuron-one/local-models