Ollama

title: "Ollama" description: "Run large language models locally with a simple REST API and CLI. Supports Llama, Mistral, Gemma, Qwen, DeepSeek, and 100+ models. Zero-config setup, model library, multimodal support, and OpenAI-compatible API. Best for local development, offline inference, and privacy-sensitive workloads." skillName: "ollama" skillVersion: "1.0.0" skillAuthor: "Orchestra Research" skillLicense: "MIT" skillTags: ["Inference", "Ollama", "Local LLMs", "REST API", "OpenAI Compatible", "Privacy", "Offline", "Llama", "Mistral", "Gemma"] skillDeps: ["ollama", "httpx", "openai"]

Ollama — Run LLMs Locally

The simplest way to run open-source LLMs on your own hardware.

When to use Ollama

Use Ollama when:

• Local development without cloud API costs
• Privacy-sensitive workloads (data never leaves your machine)
• Offline inference required
• Rapid prototyping with 100+ available models
• OpenAI API drop-in replacement needed

Metrics:

• 100,000+ GitHub stars
• 100+ supported models (Llama 3, Mistral, Gemma, Qwen, DeepSeek, Phi)
• Zero-config setup — single binary

Quick start

curl -fsSL https://ollama.com/install.sh | sh
ollama pull llama3.2
ollama run llama3.2

Python SDK

import ollama

response = ollama.chat(
    model='llama3.2',
    messages=[{'role': 'user', 'content': 'What is GRPO training?'}]
)
print(response['message']['content'])

# Streaming
for chunk in ollama.generate(model='llama3.2', prompt='Explain LoRA', stream=True):
    print(chunk['response'], end='', flush=True)

OpenAI-compatible drop-in

from openai import OpenAI
client = OpenAI(base_url='http://localhost:11434/v1', api_key='ollama')
response = client.chat.completions.create(
    model='llama3.2',
    messages=[{'role': 'user', 'content': 'Explain attention mechanisms'}]
)

Embeddings for RAG

import ollama
response = ollama.embeddings(model='nomic-embed-text', prompt='Flash Attention paper')
embedding = response['embedding']  # 768-dim vector

Custom Modelfile

FROM llama3.2
SYSTEM "You are an expert AI researcher. Provide precise, citation-backed answers."
PARAMETER temperature 0.2
PARAMETER num_ctx 8192

ollama create research-assistant -f Modelfile
ollama run research-assistant

Model library

REST API

curl http://localhost:11434/api/generate -d '{"model":"llama3.2","prompt":"Explain FSDP","stream":false}'

References

• Ollama GitHub
• Model Library
• API Reference