mkurman Overview
Ollama runs LLMs locally with a single command. Supports Llama 3, Mistral, Gemma, Qwen 2.5, DeepSeek, Phi, and 100+ models with GPU acceleration (CUDA/Metal), OpenAI-compatible API, and custom Modelfiles for configuration.
Installation
# macOS / Linux
curl -fsSL https://ollama.com/install.sh | sh
Basic Usage
ollama pull llama3.1:8b
ollama run llama3.1:8b "Explain quantum computing"
Python API
import openai
client = openai.OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")
resp = client.chat.completions.create(
model="llama3.1:8b",
messages=[{"role": "user", "content": "What is ML?"}],
)
print(resp.choices[0].message.content)
Custom Modelfile
FROM llama3.1:8b
PARAMETER temperature 0.3
SYSTEM "You are a medical coding assistant."
ollama create my-coder -f Modelfile