Configuring Models

Configuring Models in DataGenFlow

DataGenFlow uses LiteLLM under the hood. Models configured via Settings UI or REST API.

Model Config Fields

LLM: name, provider (openai|anthropic|gemini|ollama), model_name, endpoint, api_key Embedding: same + dimensions (0 = provider default)

Critical: model_name is the raw model name WITHOUT provider prefix. LiteLLM adds prefixes automatically.

Provider Setup

OpenAI

# LLM
curl -X POST http://localhost:8000/api/llm-models -H 'Content-Type: application/json' -d '{
  "name": "default", "provider": "openai",
  "model_name": "gpt-4o-mini", "api_key": "sk-..."
}'

# Embedding
curl -X POST http://localhost:8000/api/embedding-models -H 'Content-Type: application/json' -d '{
  "name": "default", "provider": "openai",
  "model_name": "text-embedding-3-small", "api_key": "sk-...", "dimensions": 1536
}'

• No endpoint needed. LiteLLM passes model name as-is.
• LLMs: gpt-4o, gpt-4o-mini
• Embeddings: text-embedding-3-small (1536d), text-embedding-3-large (3072d)

Anthropic

# LLM only — Anthropic has NO embedding models
curl -X POST http://localhost:8000/api/llm-models -H 'Content-Type: application/json' -d '{
  "name": "default", "provider": "anthropic",
  "model_name": "claude-sonnet-4-20250514", "api_key": "sk-ant-..."
}'

• No endpoint needed. LiteLLM adds anthropic/ prefix.
• LLMs: claude-sonnet-4-20250514, claude-opus-4-20250514
• No embedding models — use OpenAI or Ollama for embeddings.

Gemini

# LLM
curl -X POST http://localhost:8000/api/llm-models -H 'Content-Type: application/json' -d '{
  "name": "default", "provider": "gemini",
  "model_name": "gemini-2.0-flash", "api_key": "AIza..."
}'

# Embedding
curl -X POST http://localhost:8000/api/embedding-models -H 'Content-Type: application/json' -d '{
  "name": "default", "provider": "gemini",
  "model_name": "text-embedding-004", "api_key": "AIza...", "dimensions": 768
}'

• No endpoint needed. API key from Google AI Studio. LiteLLM adds gemini/ prefix.
• LLMs: gemini-2.0-flash, gemini-1.5-pro

Ollama

# LLM
curl -X POST http://localhost:8000/api/llm-models -H 'Content-Type: application/json' -d '{
  "name": "default", "provider": "ollama",
  "model_name": "llama3.2", "endpoint": "http://localhost:11434"
}'

# Embedding
curl -X POST http://localhost:8000/api/embedding-models -H 'Content-Type: application/json' -d '{
  "name": "default", "provider": "ollama",
  "model_name": "nomic-embed-text", "endpoint": "http://localhost:11434", "dimensions": 768
}'

• Endpoint required. No API key.
• LiteLLM adds ollama/ prefix, strips /v1/* from endpoint.
• Pull models first: ollama pull llama3.2
• LLMs: llama3.2, mistral, codellama
• Embeddings: nomic-embed-text, mxbai-embed-large

Testing Connections

# test LLM (sends "Say hello", max_tokens=10, timeout=10s)
curl -X POST http://localhost:8000/api/llm-models/test -H 'Content-Type: application/json' -d '{
  "name": "test", "provider": "openai", "model_name": "gpt-4o-mini", "api_key": "sk-..."
}'

# test embedding (sends "test" string)
curl -X POST http://localhost:8000/api/embedding-models/test -H 'Content-Type: application/json' -d '{
  "name": "test", "provider": "openai", "model_name": "text-embedding-3-small", "api_key": "sk-...", "dimensions": 1536
}'

Via UI: Settings page → click Test button next to any model.

Model Resolution Order

When a block doesn't specify a model:

1. Named model (block config model: "my-model")
2. Model named "default"
3. First model in DB
4. .env fallback: LLM_ENDPOINT + LLM_MODEL + LLM_API_KEY (LLM only)

Recommendation: always create a model named "default".

Minimum Setup

• 1 LLM model — required for any pipeline
• 1 embedding model — recommended (needed by DuplicateRemover block)

Common Mistakes

Troubleshooting

API Reference

Related Skills

• implementing-datagenflow-blocks — LLM/embedding integration patterns in blocks
• creating-pipeline-templates — templates that use configured models

Converted and distributed by TomeVault — claim your Tome and manage your conversions.