Mixedbread Search Agent

Mixedbread Search Agent

Toast-1 is Mixedbread's search model: a deep search and lookup agent trained end-to-end to gather evidence and answer from it — as a ranked evidence list, ranked evidence plus an answer, or a plain-text answer. It is served over two OpenAI-compatible endpoints, Chat Completions and Responses, and takes two kinds of tools:

• Hosted store tools — search_corpus, grep, filter_chunks, inspect_metadata, get_chunks, list_stores. Opt-in per request; the server runs the whole search loop over your Mixedbread Stores and the request returns with the answer. Use the API reference for their wire contract; the open-source toast-harness illustrates retrieval and orchestration patterns.
• Function tools — ordinary function schemas your application executes and feeds back, so the corpus can live anywhere: Stores, a vector DB, Elasticsearch, SQL, or the filesystem.

With no hosted retrieval or server-side context editing, a request performs one model generation. Custom function calls return to your application for execution.

Docs: https://www.mixedbread.com/docs/agent/chat-completions and https://www.mixedbread.com/docs/agent/responses Harness guide: https://www.mixedbread.com/docs/agent/build-your-own-harness API reference: https://www.mixedbread.com/api-reference/endpoints/chat/create-chat-completion and https://www.mixedbread.com/api-reference/endpoints/responses/create-response Model card: https://www.mixedbread.com/docs/agent/models Agent-readable docs: https://www.mixedbread.com/docs/llms.txt

Choose a path

Both endpoints take the same tools and extensions. The Responses API is the primary surface for hosted tools and stored conversations; Chat Completions is the message-based surface. Either endpoint can support a custom harness. This skill is written against Chat Completions; Responses API maps the differences.

Authentication

Get an API key at https://platform.mixedbread.com/platform?next=api-keys, then:

export MXBAI_API_KEY=mxb_xxxxx

Pass the key explicitly. The OpenAI SDKs default to OPENAI_API_KEY and know nothing about MXBAI_API_KEY; without an explicit api_key/apiKey the client either sends the wrong key or raises a missing-credentials error at construction.

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.mixedbread.com/v1",
    api_key=os.environ["MXBAI_API_KEY"],
)

import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'https://api.mixedbread.com/v1',
  apiKey: process.env.MXBAI_API_KEY!,
});

The endpoints are POST https://api.mixedbread.com/v1/chat/completions and POST https://api.mixedbread.com/v1/responses, so the base URL keeps the /v1. A scope-restricted key needs the Completions scope. Any OpenAI-compatible client works if it allows a custom base URL and an escape hatch for extension fields.

Making a request

completion = client.chat.completions.create(
    model="toast-1",
    messages=[{"role": "user", "content": "Which contract governs the 2019 agreement?"}],
    tools=TOOLS,            # hosted tool entries, function schemas, or both
    temperature=0.7,
    top_p=0.95,
    store=False,            # completions are stored by default
)

message = completion.choices[0].message
if completion.choices[0].finish_reason == "length":
    ...                     # incomplete: recover or report it explicitly
elif message.tool_calls:
    ...                     # execute your functions and continue
elif (message.content or "").strip():
    print(message.content)

Mixedbread extension fields go in extra_body in Python and inline with a cast in Node (the SDK forwards them unchanged):

Unknown fields are ignored; chat_template_kwargs is not a parameter.

Continuation examples are in Stored conversations.

Reading the response

The Python SDK parks extension fields in model_extra:

hosted = (completion.model_extra or {}).get("hosted_tool_calls") or []
edits = ((completion.model_extra or {}).get("context_management") or {}).get("applied_edits") or []

Hosted store tools

Declare hosted tools only when you want server-side retrieval over Mixedbread Stores. They can be combined with custom functions under different names. Read hosted-tools.md for store scope, tool-entry fields, returned evidence, citations, and a hybrid terminal recipe.

Function tools

Tools are ordinary function schemas. What to declare and how to describe them is in tool-contracts.md; backing them with Mixedbread Stores is in mixedbread-tools.md.

finish_reason="tool_calls" means the model is waiting on you. Send exactly one tool message per tool_call_id — including for calls you rejected or that failed; an assistant tool call with no matching tool message makes the next request invalid. Continuing with previous_completion_id, the tool messages are the whole next request; resending the history yourself, append the assistant turn first.

Prefer JSON results for clear, consistent tool output; the envelope is your choice. Catch errors at the executor boundary and return useful failure information so the model can recover while the protocol remains valid.

These serial snippets illustrate the message protocol. For a search harness, execute independent retrieval calls concurrently; the harness skill's Python example shows bounded parallel execution.

messages.append(message.model_dump(exclude_none=True))
for call in message.tool_calls:
    try:
        result = IMPLEMENTATIONS[call.function.name](**json.loads(call.function.arguments or "{}"))
    except Exception as exc:                  # never let a tool raise past the executor
        result = {"error": str(exc), "retryable": True}
    messages.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(result)})

messages.push(message);
for (const call of message.tool_calls ?? []) {
  let result;
  try {
    result = await run(call);
  } catch (error) {
    result = { error: error instanceof Error ? error.message : String(error), retryable: true };
  }
  messages.push({ role: 'tool', tool_call_id: call.id, content: JSON.stringify(result) });
}

Terminal modes

The public harness uses ranked evidence, evidence plus an answer, or plain text. These are useful starting points; custom terminal names and payloads are supported. See the optional shapes and validation guidance in tool-contracts.md.

Offer a terminal during exploration if you want structured output. Forcing that function by name can recover a prose ending or enforce a final turn at your chosen cap. The hybrid recipe covers this after hosted retrieval. Check finish_reason and validate results before accepting them.

Context management

We recommend server-side pruning even when you build the entire retrieval loop yourself:

extra_body={"context_management": {"edits": [{"type": "prune_context"}]}}

Declare it on each request. The API gives the model a pruning tool over both hosted and custom function results; pruning counts against max_tool_calls. The public harness addresses chunks, while the API uses response and span references internally to prune general tool output. You do not need to reproduce either implementation or manage those internal references.

context_management.applied_edits reports applied pruning and overflow-recovery truncation. Stored conversation content is retained; edits affect the model's context and carry forward through stored continuation. With a full-history resend, previously pruned content returns unless you also edit that history. A stateless loop can still request server-side pruning for each request.

Without declared context editing, an oversized input fails with 422 context_length_exceeded_error. Clip incoming payloads and leave headroom even when pruning is enabled. If you choose to edit history yourself, resend the edited history without a continuation ID. Custom context policies and framework compaction are also options; account for evidence identity and restoration.

Stored conversations

first = client.chat.completions.create(model="toast-1", messages=messages, tools=TOOLS, store=True)

second = client.chat.completions.create(
    model="toast-1",
    messages=[{"role": "user", "content": "And when does it expire?"}],   # only the new suffix
    tools=TOOLS, store=True,
    extra_body={"previous_completion_id": first.id},
)

const second = await client.chat.completions.create({
  model: 'toast-1',
  messages: [{ role: 'user', content: 'And when does it expire?' }],
  tools, store: true,
  previous_completion_id: first.id,
  context_management: { edits: [{ type: 'prune_context' }] },
} as never); // Mixedbread extensions go inline in Node, not in extra_body.

Streaming

stream = client.chat.completions.create(
    model="toast-1", messages=messages, tools=TOOLS, stream=True, store=False,
)
for chunk in stream:
    for call in (chunk.model_extra or {}).get("hosted_tool_calls") or []:   # once in_progress, once finished
        show_progress(call["type"], call["status"], call.get("error"))
    for choice in chunk.choices:            # hosted-progress chunks arrive with choices == []
        if choice.delta.content:
            show_answer(choice.delta.content)

Streamed function calls arrive as delta.tool_calls fragments and must be accumulated by index before execution. context_management arrives on the final usage chunk. Check the final finish reason before treating streamed text as a complete answer.

Responses API

The same model, tools, and extensions over POST /v1/responses, and the primary documentation surface for the hosted tools:

response = client.responses.create(
    model="toast-1",
    input="Which suppliers had recalls in 2023?",
    tools=[{"type": "search_corpus", "store_identifiers": ["my-store"]}],
    include=["search_corpus_call.results"],
    extra_body={"context_management": {"edits": [{"type": "prune_context"}]}},
)
if response.status != "completed":
    raise RuntimeError(f"Response did not complete: {response.status}")
print(response.output_text)

Streaming emits the standard semantic events plus response.output_item.added/done for each hosted call; a terminal response.completed or response.incomplete carries the full response. Hosted items are an extension: they claim an output_index in the shared item order but land in hosted_tool_calls, not output, so do not index output by output_index.

Limits

References

• hosted-tools.md — read when declaring hosted store tools: per-tool fields and defaults, call item and result shapes, include, streaming, citations, the hybrid recipe, complete Python and TypeScript examples.
• tool-contracts.md — optional backend-agnostic guidance for descriptions, evidence envelopes, errors, and terminal formats.
• mixedbread-tools.md — read when backing function tools with Mixedbread Stores yourself: the call behind each primitive, filters, chunk identity, gotchas.

Troubleshooting