Open Webui Embeddings

Open WebUI RAG — embedding + reranking models and backends

Target: operators choosing and wiring open-weights embedding and reranker models into Open WebUI. Almost every failure mode here degrades quality silently — right HTTP status, right vector dimension, worse answers — so this skill is built to catch them at config-time rather than by noticing that RAG "feels worse".

Two layers, kept separate:

• Open WebUI's side is backend-agnostic. RAG_EMBEDDING_ENGINE=openai, the /v1/embeddings payload, prefix handling, batching and concurrency are identical whether you serve with TEI, vLLM, OpenRouter, or OpenAI itself. That layer is this file plus references/prefix-models.md.
• The serving layer differs per backend and lives in references/backends.md (+ the TEI-specific cliffs in references/gotchas.md §2-§6, §9).

Verified against v0.11.0 source (2026-07-29). The embed and rerank code paths are unchanged from 0.10.2 — 0.11.0's retrieval churn landed almost entirely in retrieval/web/ (web search), not the embedding core — so every wire shape and env default below still holds. One adjacent 0.11.0 change does affect ingestion verification: see §Verifying ingestion below.

Start here

Three findings worth knowing before you choose anything:

1. A weak reranker is worse than no reranker. bge-reranker-v2-m3 — this skill's former worked example — scores below the un-reranked baseline on MTEB-R and MMTEB-R. Prefer Qwen3-Reranker-0.6B at identical size and cost.
2. BGE-M3 is the only prefix-free model in common use. Every migration off it inherits query/document prefix configuration, and getting that wrong costs quality with no error.
3. Read MTEB's zero-shot column before its scores. The current #1 sits at 78% zero-shot — up to 22% of the benchmark may be in its training data.

Siblings in the open-webui plugin. Setting these values through the REST API rather than the settings UI — and the knowledge/RAG endpoints that ingest documents — is the open-webui-api skill. Running more than one Open WebUI replica, where RAG requests and their WebSocket streams must survive hitting a different pod, is open-webui-valkey-websocket.

The architecture in 30 seconds

Open WebUI terminates two contracts. Anything that speaks both works:

Open WebUI ──embed──→  POST {RAG_OPENAI_API_BASE_URL}/embeddings   OpenAI shape
           └─rerank──→ POST {RAG_EXTERNAL_RERANKER_URL}            Cohere shape, strict

The one hard constraint: TEI rerank cannot be wired directly. Its shape is {query, texts} → [{index, score}]; Open WebUI demands {model, query, documents, top_n} → {results: [{index, relevance_score}]}. Everything else is a preference. Details in references/backends.md.

Wire shapes (exact)

Embed — Open WebUI code path

backend/open_webui/retrieval/utils.py:862 (generate_openai_batch_embeddings, v0.11.0):

POST {RAG_OPENAI_API_BASE_URL}/embeddings        ← URL is auto-appended
Authorization: Bearer {RAG_OPENAI_API_KEY}
Content-Type: application/json

{"input": ["text1", "text2", ...], "model": "{RAG_EMBEDDING_MODEL}"}

Response parsed as data["data"][i]["embedding"] (OpenAI shape).

Async fan-out (get_embedding_function at utils.py:1090, batching + asyncio.gather at utils.py:1139-1156, v0.11.0): chunks bundled into batches of RAG_EMBEDDING_BATCH_SIZE (default 1); all batches dispatched concurrently via asyncio.gather, wrapped in an asyncio.Semaphore only when RAG_EMBEDDING_CONCURRENT_REQUESTS is non-zero (default 0 = unlimited). A 100-chunk file at default config fires 100 concurrent single-chunk requests.

Rerank — Open WebUI code path

backend/open_webui/retrieval/models/external.py:13 (ExternalReranker, predict at line 26; v0.11.0, unchanged since 0.10.2):

POST {RAG_EXTERNAL_RERANKER_URL}                 ← URL is exact, NOT appended
Authorization: Bearer {RAG_EXTERNAL_RERANKER_API_KEY}
Content-Type: application/json

{"model": "{RAG_RERANKING_MODEL}", "query": "...",
 "documents": ["doc1", "doc2", ...], "top_n": <len(documents)>}

Response parsed: data["results"] sorted by index, extracts relevance_score. Cohere shape, strict.

Failure handling: requests.post() exception or non-2xx → predict() returns None → retrieval silently downgrades to un-reranked hybrid order. No user-visible error in Open WebUI. Always alert on rerank-side 4xx in TEI/LiteLLM logs.

Open WebUI environment variables

Triage table

Verifying ingestion (changed in 0.11.0)

The obvious way to confirm a knowledge base actually extracted text — list its files and look at the content — stopped working in 0.11.0. Knowledges.get_file_metadatas_by_id became a column-only SELECT File.id, File.hash, File.meta, File.created_at, File.updated_at (models/knowledge.py:684-694), deliberately excluding File.data, which is where the extracted text lives. Its own docstring says so.

Consequence: GET /api/v1/knowledge/{id} and the KB detail view now return file entries with no extracted content, so an empty-looking listing no longer distinguishes "extraction failed" from "extraction succeeded, field not selected". To verify extraction actually produced text, fetch the individual file (GET /api/v1/files/{id}) rather than reading the collection listing. Knowledge list items gained a file_count in exchange.

This is a pure API-shape change — embedding and retrieval quality are unaffected. It matters only for ingestion-verification scripts and health checks.

Reference index

• references/model-selection.md — which embedder and which reranker. MTEB scores with the zero-shot contamination column, a FLOPs/MFU throughput model (no leaderboard publishes speed), the reranker table including the below-baseline finding, OpenRouter's free tier with real prices, and a complete copy-paste Open WebUI config block per model. Load first when choosing or replacing a model.
• references/prefix-models.md — symmetric vs asymmetric embedders, exact prefix strings per model, the PREFIX_FIELD_NAME mode switch, post-transformer module stacks, and a numerical check that prefixes are really applied. Load whenever the embedder is not BGE-M3.
• references/backends.md — TEI, vLLM, OpenRouter and LiteLLM as peers: which to pick, serving commands, vLLM registry gaps and pooling fallbacks, OpenRouter's API quirks, and how to verify a backend swap numerically.
• references/gotchas.md — nine gotchas with HTTP error strings, root causes, and fixes. §2-§6 and §9 are TEI-specific; §1 is LiteLLM; §7-§8 are backend-agnostic. Load when the triage table points here.
• references/end-to-end-config.md — one fully worked TEI + LiteLLM + Open WebUI deployment. Historical worked example (BGE-M3 + BGE-Reranker-v2-m3): correct as a TEI wiring reference, but see model-selection.md before adopting those two models.
• references/performance.md — cross-engine numerical-identity check + the measured throughput baseline that calibrates the estimates in model-selection.md.
• references/sources.md — authoritative source files and PR/issue URLs. Load to verify a claim or run freshen mode.