Langchain Ollama

LangChain Ollama Skill

Expert assistance for langchain-ollama: run local LLMs via Ollama with full LangChain integration — chat, completions, embeddings, tool calling, and structured output.

Install:

pip install -U langchain-ollama
# Pull a model: ollama pull llama3.1
# Linux: start server with `ollama serve`  (Mac: runs automatically)

Reference: references/api.md (500 KB — full API reference).

When to Use This Skill

Activate when:

• Using ChatOllama — chat completions with local models, including streaming and multi-turn
• Enabling reasoning/thinking mode — setting reasoning=True on supported models (DeepSeek-R1, etc.)
• Tool calling with local models — binding tools to ChatOllama for function/tool use
• Structured output — using .with_structured_output() for JSON/Pydantic output
• Raw text completions — using OllamaLLM for non-chat completion tasks
• Generating embeddings — using OllamaEmbeddings for RAG or similarity search
• Connecting to a remote Ollama server — setting base_url to a non-localhost instance
• Controlling generation params — temperature, num_predict, top_k, top_p, seed

Quick Reference

ChatOllama — invoke and stream

from langchain_ollama import ChatOllama

model = ChatOllama(
    model="llama3.1",
    temperature=0.8,
    num_predict=256,
    # base_url="http://remote-server:11434",  # default: localhost:11434
    # validate_model_on_init=True,            # check model exists on startup
)

# Invoke
messages = [
    ("system", "You are a helpful translator. Translate the user sentence to French."),
    ("human", "I love programming."),
]
response = model.invoke(messages)
print(response.content)

# Stream
for chunk in model.stream("Explain recursion in one paragraph."):
    print(chunk.content, end="", flush=True)

Reasoning / thinking mode (DeepSeek-R1, QwQ, etc.)

from langchain_ollama import ChatOllama

model = ChatOllama(
    model="deepseek-r1:7b",
    reasoning=True,   # separates reasoning from final answer
    # reasoning=False  → suppress thinking entirely
    # reasoning=None   → default; <think> tags appear in content
)

response = model.invoke("What is 17 * 23?")
print(response.content)                                      # final answer only
print(response.additional_kwargs.get("reasoning_content"))  # reasoning trace

Tool calling

from langchain_ollama import ChatOllama
from langchain_core.tools import tool

@tool
def get_weather(city: str) -> str:
    """Get the current weather for a city."""
    return f"The weather in {city} is sunny and 22°C."

model = ChatOllama(model="llama3.1")
model_with_tools = model.bind_tools([get_weather])

response = model_with_tools.invoke("What's the weather in Paris?")
print(response.tool_calls)
# [{'name': 'get_weather', 'args': {'city': 'Paris'}, 'id': '...'}]

Structured output (JSON / Pydantic)

from langchain_ollama import ChatOllama
from pydantic import BaseModel, Field

class Translation(BaseModel):
    original: str = Field(description="The original text")
    translated: str = Field(description="The translated text")
    language: str = Field(description="Target language")

model = ChatOllama(model="llama3.1")
structured = model.with_structured_output(Translation)

result = structured.invoke("Translate 'Hello world' to Spanish")
print(result.translated)   # "Hola mundo"

OllamaLLM — raw text completions

from langchain_ollama import OllamaLLM

llm = OllamaLLM(
    model="llama3.1",
    temperature=0.7,
    num_predict=256,
    top_k=40,
    top_p=0.9,
    seed=42,              # reproducible output
    format="json",        # force JSON output format
    keep_alive="5m",      # how long model stays loaded (default "5m")
)

response = llm.invoke("The capital of France is")
print(response)

# Stream raw text
for chunk in llm.stream("Write a haiku about code:"):
    print(chunk, end="", flush=True)

OllamaEmbeddings — generate embeddings for RAG

from langchain_ollama import OllamaEmbeddings
from langchain_core.vectorstores import InMemoryVectorStore

embed = OllamaEmbeddings(model="nomic-embed-text")

# Embed a single query
query_vec = embed.embed_query("What is LangChain?")

# Embed a batch of documents
doc_vecs = embed.embed_documents([
    "LangChain is a framework for LLM applications.",
    "Ollama runs LLMs locally.",
])

# Use in a vector store
vectorstore = InMemoryVectorStore(embed)
vectorstore.add_texts(["LangChain is a framework.", "Ollama runs locally."])
results = vectorstore.similarity_search("What is LangChain?", k=1)

Connect to remote Ollama server

from langchain_ollama import ChatOllama, OllamaEmbeddings

chat = ChatOllama(
    model="llama3.1",
    base_url="http://192.168.1.100:11434",
)

embed = OllamaEmbeddings(
    model="nomic-embed-text",
    base_url="http://192.168.1.100:11434",
)

API Reference

ChatOllama key parameters

OllamaLLM key parameters

OllamaEmbeddings key parameters

Common Ollama CLI commands

ollama pull llama3.1              # download a chat model
ollama pull nomic-embed-text      # download an embedding model
ollama pull deepseek-r1:7b        # download a reasoning model
ollama list                       # list downloaded models
ollama serve                      # start server (Linux/WSL)
ollama ps                         # show running models
ollama rm llama3.1                # remove a model

Reference Files

Source: https://reference.langchain.com/python/langchain-ollama
Models: https://ollama.com/library