Ollama Patterns

Ollama — Local LLM

Setup & Pull Models

# Install
curl -fsSL https://ollama.ai/install.sh -o /tmp/ollama-install.sh
# Inspect first: head -40 /tmp/ollama-install.sh — then run if safe:
sh /tmp/ollama-install.sh

# Pull models
ollama pull llama3.2             # 3B — fast, good for dev
ollama pull llama3.1:8b          # 8B — balanced quality
ollama pull mistral-nemo         # 12B — strong reasoning
ollama pull deepseek-r1:8b       # reasoning model
ollama pull nomic-embed-text     # embeddings
ollama pull mxbai-embed-large    # better embeddings

# List, remove
ollama list
ollama rm llama3.2
ollama show llama3.2             # model info, parameters

Python Client

import ollama

# Simple chat
response = ollama.chat(
    model="llama3.2",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain quantum entanglement briefly."},
    ],
)
print(response["message"]["content"])

# Streaming
for chunk in ollama.chat(
    model="llama3.2",
    messages=[{"role": "user", "content": "Write a haiku"}],
    stream=True,
):
    print(chunk["message"]["content"], end="", flush=True)

# Generate (no chat format)
response = ollama.generate(
    model="llama3.2",
    prompt="Complete this: The sky is",
    options={"temperature": 0.8, "top_p": 0.9, "num_predict": 100},
)
print(response["response"])

Embeddings

import ollama

# Single embedding
result = ollama.embeddings(
    model="nomic-embed-text",
    prompt="Machine learning is fascinating",
)
vector = result["embedding"]   # List[float], dim=768

# Batch embeddings (loop — no native batch API)
texts = ["text1", "text2", "text3"]
embeddings = [
    ollama.embeddings(model="nomic-embed-text", prompt=t)["embedding"]
    for t in texts
]

# With async client
import asyncio

async def embed_batch(texts: list[str]) -> list[list[float]]:
    client = ollama.AsyncClient()
    tasks = [client.embeddings(model="nomic-embed-text", prompt=t) for t in texts]
    results = await asyncio.gather(*tasks)
    return [r["embedding"] for r in results]

Async Client

import asyncio
import ollama

async def main():
    client = ollama.AsyncClient()

    # Async chat
    response = await client.chat(
        model="llama3.2",
        messages=[{"role": "user", "content": "Hello"}],
    )
    print(response["message"]["content"])

    # Async streaming
    async for chunk in await client.chat(
        model="llama3.2",
        messages=[{"role": "user", "content": "Tell me a story"}],
        stream=True,
    ):
        print(chunk["message"]["content"], end="", flush=True)

asyncio.run(main())

Structured Output (JSON mode)

import ollama
import json

response = ollama.chat(
    model="llama3.2",
    messages=[{
        "role": "user",
        "content": "Extract: name, age, profession from: 'John is a 30-year-old engineer'",
    }],
    format="json",                  # forces JSON output
)
data = json.loads(response["message"]["content"])
print(data["name"], data["age"])    # "John", 30

# With Pydantic schema
from pydantic import BaseModel

class Person(BaseModel):
    name: str
    age: int
    profession: str

response = ollama.chat(
    model="llama3.2",
    messages=[{"role": "user", "content": "Extract from: 'Alice is a 25-year-old designer'"}],
    format=Person.model_json_schema(),   # enforce schema
)
person = Person.model_validate_json(response["message"]["content"])

REST API (curl / direct)

# Generate
curl http://localhost:11434/api/generate -d '{
  "model": "llama3.2",
  "prompt": "Why is the sky blue?",
  "stream": false
}'

# Chat
curl http://localhost:11434/api/chat -d '{
  "model": "llama3.2",
  "messages": [{"role": "user", "content": "Hello"}],
  "stream": false
}'

# Embeddings
curl http://localhost:11434/api/embeddings -d '{
  "model": "nomic-embed-text",
  "prompt": "Here is an article about llamas"
}'

OpenAI-Compatible API

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="ollama",               # required but ignored
)

response = client.chat.completions.create(
    model="llama3.2",
    messages=[{"role": "user", "content": "Say hello"}],
)
print(response.choices[0].message.content)

# Works with any OpenAI-compatible library

Modelfile — Custom Models

# Modelfile
FROM llama3.2

SYSTEM """You are a Python expert. Always include type hints and docstrings."""

PARAMETER temperature 0.3
PARAMETER top_p 0.9
PARAMETER num_predict 2048

ollama create python-expert -f Modelfile
ollama run python-expert "Write a binary search function"

Anti-Fake-Pass Checks

• Ollama server must be running: ollama serve or daemon auto-start on macOS/Linux
• stream=False in REST API — without it, response is streaming NDJSON, not single JSON
• format="json" encourages but doesn't guarantee valid JSON — use outlines for guaranteed schema
• Embeddings are model-specific dimension — nomic-embed-text=768, mxbai-embed-large=1024
• num_predict=-1 means unlimited tokens — set a limit to avoid runaway generation
• Async client returns coroutines — must await every call inside async def
• OpenAI-compatible API at /v1 — base_url must include /v1 path