Modelmix

ModelMix Library Skill

Overview

ModelMix is a Node.js library providing a unified fluent API to interact with multiple AI LLM providers. It handles automatic fallback between models, round-robin load balancing, structured JSON output, streaming, MCP tool integration, custom local tools, rate limiting, and token tracking.

Use this skill when:

• Integrating one or more AI models into a Node.js project
• Chaining models with automatic fallback or round-robin
• Extracting structured JSON from LLMs
• Adding MCP tools or custom tools to models
• Streaming responses from any provider
• Working with templates and file-based prompts
• Tracking token usage and costs

Do NOT use for:

• Python or non-Node.js projects
• Direct HTTP calls to LLM APIs (use ModelMix instead)

Quick Reference

• Installation
• Creating an instance
• Attaching models
• Unified effort
• Get a text response
• Get structured JSON
• Stream a response
• Extract a code block
• Get raw response (tokens, thinking, tool calls)
• Access full response with lastRaw
• Add images
• EJS templates
• Round-robin load balancing
• MCP integration
• Custom local tools
• Rate limiting
• Conversation history
• Debug mode
• Free-tier models
• Multi-provider routing

Installation

npm install modelmix

Core Concepts

Import

import { ModelMix } from 'modelmix';

Creating an Instance

const model = ModelMix.new();

const model = ModelMix.new({
    options: { max_tokens: 4096, temperature: 0.7 },
    config: {
        system: "You are a helpful assistant.",
        max_history: 5,   // -1 = unlimited, 0 = none (default), N = keep last N
        debug: 0,          // 0=silent, 1=minimal, 2=summary, 3=full, 4=verbose
        roundRobin: false, // false=fallback, true=rotate models
        effort: 50         // unified 0..100, or -1 adaptive
    }
});

Attaching Models

Chain shorthand methods to attach providers. First model is primary; others are fallbacks (or rotated if roundRobin: true):

const model = ModelMix.new()
    .sonnet46()        // primary
    .gpt52()           // fallback 1
    .gemini38flash()   // fallback 2
    .addText("Hello!")

If sonnet46 fails, it automatically tries gpt52, then gemini38flash.

The equivalent chain() form accepts public shortcut names directly in the same order. Append @effort for a per-model unified effort override (-1 or 0–100). Without the suffix, the entry inherits config.effort, or keeps the provider default when no chain effort is configured:

const model = ModelMix.new()
    .chain('sonnet46', 'gpt52@20', 'gemini38flash@-1')
    .addText('Hello!');

Instance plugins

Register middleware with .use({ name, execute }). Plugins are scoped to the instance, run in registration order, and receive the rendered provider-neutral request. They may edit context.request, call next(), or return a complete ModelMix result. context.signal is the current execution's AbortSignal and is inherited by context.invoke() children.

model.use({
    name: 'metrics',
    async execute(context, next) {
        const result = await next();
        return { ...result, executionId: context.execution.executionId };
    }
});

context.invoke() starts a child without conversation history. Its plugins policy is 'inherit', 'none', { include: [...] }, or { exclude: [...] }. Passing model routes the child through another ModelMix worker chain while preserving execution-tree metadata. Child invocations may pass assign with either system or systemFile; systemFile uses the ordinary ModelMix EJS renderer and relative includes.

The optional @modelmix/rlm package is a separate workspace/npm package for recursive processing of large structured inputs. Pass Markdown through documents: { name: { format: 'markdown', content } }, register named ModelMix worker chains, and provide every runtime limit explicitly. A worker uses either model: anotherModelMixInstance or useParent: true. Its planner sees content-free variable size/shape metadata, while document values and generated orchestration code stay inside an isolated-vm sandbox. RLM planner prompts are Markdown files rendered with the normal child assign plus systemFile path.

Unified effort

Provider-agnostic reasoning intensity. Not an options field — use config.effort or .effort(n).

ModelMix.new({ config: { effort: 40 } }).sonnet46().addText('Plan this refactor').message();
ModelMix.new().deepseekV4Flash({ config: { effort: 100 } }).addText('...').message();
ModelMix.new().effort(-1).minimaxM3().addText('Quick question').message();

// Native provider fields win when already set
ModelMix.new({ config: { effort: 80 } })
  .gpt52({ options: { reasoning_effort: 'none' } }) // stays none

* GPT-6 Astra maps 0–39 / 40–59 / 60–79 / 80–99 / 100 to low / medium / high / xhigh / max. GPT-5.6 maps 100 to max; 80–99 remains xhigh. Qwen 3.8 27B and Flash map 0–39 / 40–79 / 80–100 to low / medium / xhigh; Qwen 3.8 Flash is the managed production version based on Flash-Next. GLM 5.3 and GLM 5.3 Flash require reasoning and map those bands to low / high / max. Gemini bands: 0–24 / 25–49 / 50–74 / 75–100. Gemini 3.8 Flash and 3.7 Flash support only low / medium / high, so the first two bands clamp to low; -1 keeps their native medium default. DeepSeek ↑ = thinking on; off = thinking disabled. MiniMax off/adaptive = thinking.disabled / thinking.type=adaptive. Gemini 2.5 maps 0–100 to thinkingBudget. Anthropic: adaptive + output_config.effort on Claude 5 / Fable / Opus 4.6+ / Sonnet 4.6+; Sonnet 4.5 / Haiku 4.5 use thinking.type=enabled + budget_tokens. Grok 4.6 maps 0–39 / 40–59 / 60–79 / 80–100 to low / medium / high / xhigh; without effort it uses native high. -1 = adaptive/dynamic when available, else no-op. Levels clamp per model. Former *think() methods are removed — use .effort(n).<model>(). Kimi: kimiK25() / kimiK26(). Grok 4.20: .grok420() non-reasoning; .effort(20+|-1).grok420() selects reasoning.

Available Model Shorthands

OpenAI

Use ModerationMix.new().openai() with .raw() to classify text and images through OpenAI's Moderations endpoint. Read the results from raw.moderation. ModerationMix accepts moderation providers as ordered fallbacks, rejects generative providers, and does not generate text or support streaming.

gpt6astra() gpt56sol() gpt56terra() gpt56luna() gpt55() gpt55pro() gpt54() gpt54mini() gpt54nano() gpt54pro() gpt53codex() gpt53chat() gpt52() gpt51() gpt5() gpt5mini() gpt5nano() gptRealtime() gptRealtimeMini() gptOss()

Every textual GPT-5 and GPT-6 shortcut registers only the official OpenAI model by default. Pass mix: { openrouter: true } to ModelMix.new() or to an individual shortcut to append its openai/* OpenRouter route as a fallback. gpt53chat() uses gpt-5.3-chat-latest officially and openai/gpt-5.3-chat through OpenRouter. Both API keys are required when that fallback is enabled. Realtime shortcuts remain official-only.

Anthropic

fable51() fable50() opus50() opus48() opus47() opus46() sonnet5() sonnet46() sonnet45() haiku45()

Use .effort(n) (or config.effort) to enable Anthropic thinking — e.g. .effort(100).opus50(). fable5() and opus5() remain available as compatibility aliases.

fable51() registers claude-fable-5-1 through Anthropic by default. Pass mix: { openrouter: true } to append anthropic/claude-fable-5.1 as its fallback.

Google

gemini31pro() gemini38flash() gemini37flash() gemini36flash() gemini35flash() gemini35flashLite() gemini31flashLite()

Grok

grok46() grok45() grok43() grok420multiAgent() grok420()

Perplexity

sonar() sonarPro()

Together

museGlimmer30b() qwen36plus() qwen37plus() GLM52() kimiK25() kimiK26() kimiK27Code() kimiK3() minimaxM27() minimaxM3() gptOss()

NVIDIA

museGlimmer30b() gptOss() minimaxM27()

Moonshot

kimiK3() — requires MOONSHOT_API_KEY; Fireworks, OpenRouter, and Together are available through mix.

MiniMax

minimaxM27() minimaxM3()

Fireworks

museGlimmer30b() gptOss() qwen36plus() (private/on-demand only) qwen37plus() qwen38max() deepseekV4Flash() deepseekV4Pro() kimiK26() kimiK27Code() kimiK3() minimaxM27() minimaxM3() GLM52()

Cerebras

GLM46()

OpenRouter

museGlimmer30b() museSpark12() museSpark12c() museSpark13() museSpark13c() gptOss() qwen35397b() qwen36plus() qwen37plus() qwen3827b() qwen38flash() hermes470b() hermes4405b() qwen38max() kimiK27Code() kimiK3() minimaxM27() minimaxM3() GLM45() GLM52() GLM53() GLM53Flash()

Muse Spark: the c suffix selects Contributor, where prompts and outputs may be used to improve Meta products. Without c, the standard tier is selected. Use museSpark12c() for the former museSpark12() Contributor behavior.

Multi-provider (auto-fallback across free/paid tiers)

hermes3() kimiK25()

Local

lmstudio() — for LM Studio local models

Each method accepts optional { options, config } to override per-model settings.

Common Tasks

Get a text response

const answer = await ModelMix.new()
    .gpt5mini()
    .addText("What is the capital of France?")
    .message();

Get structured JSON

const result = await ModelMix.new()
    .gpt5mini()
    .addText("Name and capital of 3 South American countries.")
    .json(
        { countries: [{ name: "", capital: "" }] },
        { countries: [{ name: "country name", capital: "in uppercase" }] },
        { addNote: true }
    );

json() signature: json(schemaExample, schemaDescription?, { addSchema, addExample, addNote }?)

Enhanced descriptors

Descriptions can be strings or descriptor objects with metadata:

const result = await model.json(
    { name: 'martin', age: 22, sex: 'Male' },
    {
        name: { description: 'Name of the actor', required: false },
        age: 'Age of the actor',
        sex: { description: 'Gender', enum: ['Male', 'Female', null] }
    }
);

Descriptor properties:

Nested object descriptions

Pass a plain object as the description value to annotate fields inside a nested object:

model.json(
    { user: { name: 'Alice', age: 30 } },
    { user: { name: 'Full name', age: 'Age in years' } }
);

To mark the object itself as optional, use a descriptor (only description/required/nullable keys) — it applies to the parent, not the children:

model.json(
    { user: { name: 'Alice', age: 30 } },
    { user: { description: 'User details', required: false } }
);

Array item descriptions

Wrap descriptions in an array to annotate items of an array field:

model.json(
    { countries: [{ name: 'France', capital: 'Paris' }] },
    { countries: [{ name: 'Country name', capital: 'Capital in uppercase' }] }
);

Automatic type and format detection

Schema types are inferred from example values: integer (whole numbers), number (floats), boolean, null, string, and special formats:

• 'user@example.com' → { type: 'string', format: 'email' }
• '1990-01-01' → { type: 'string', format: 'date' }
• '14:30' / '09:15:45' → { type: 'string', format: 'time' }

Array auto-wrap

Top-level arrays are auto-wrapped as { out: [...] } for better LLM compatibility, and unwrapped on return:

const result = await model.json([{ name: 'martin' }]);
// result is an array: [{ name: "Martin" }, { name: "Carlos" }, ...]

Stream a response

await ModelMix.new()
    .gpt5mini()
    .addText("Tell me a story.")
    .stream(({ delta, message }) => {
        process.stdout.write(delta);
    });

Extract a code block

const code = await ModelMix.new()
    .gpt5mini()
    .addText("Write a hello world function in JavaScript.")
    .block();
// Returns only the content inside the first code block

block() accepts { addSystemExtra } (default true) — adds system instructions that tell the model to wrap output in a code block.

Get raw response

const raw = await ModelMix.new()
    .effort(100)
    .sonnet45()
    .addText("Solve this step by step: 2+2*3")
    .raw();
// raw.message, raw.think, raw.tokens, raw.toolCalls, raw.response

Cancel one execution

Pass an AbortSignal directly to the terminal method:

const controller = new AbortController();
const pending = model.message(controller.signal);
controller.abort(new Error('Request cancelled'));
await pending;

The other forms are raw(signal), stream(callback, signal), block(options, signal), and json(example, description, jsonOptions, signal). For execute(), use the top-level { signal } field. Never place signal in config or options; ModelMix rejects those reusable/payload-bearing locations. Plugins read context.signal, child invocations inherit it, and local tool callbacks receive it as their second argument.

Access full response with lastRaw

After calling message(), json(), block(), or stream(), use lastRaw to access the complete response:

const model = ModelMix.new().gpt5mini().addText("Hello!");
const text = await model.message();
console.log(model.lastRaw.tokens);
// {
//   input: 1200, output: 50, thinking: 0, total: 1250,
//   cached: 1024, cacheWrite: 0, uncachedInput: 176,
//   cacheWrite5m: 0, cacheWrite1h: 0,
//   cacheHitRate: 0.8533, cacheSavings: 0.00018432,
//   cacheWritePremium: 0, breakEvenHits: 0,
//   cost: 0.00011568,
//   costBreakdown: {
//     uncachedInput: 0.0000352, cachedInput: 0.00002048,
//     cacheWrite: 0, cacheWrite5m: 0, cacheWrite1h: 0,
//     output: 0.00006, total: 0.00011568
//   },
//   speed: 38
// }
console.log(model.lastRaw.think);    // reasoning content (if available)
console.log(model.lastRaw.response); // raw API response

GPT-5.6 explicit prompt caching

const model = ModelMix.new()
  .gpt56luna({
    options: {
      prompt_cache_key: 'stable-prefix-v1',
      prompt_cache_options: { mode: 'explicit', ttl: '30m' }
    }
  })
  .haiku45({
    options: {
      cache_control: { type: 'ephemeral', ttl: '1h' }
    }
  })
  .addText(longStableInstructions, {
    cache: { breakpoint: true }
  })
  .addText('Handle this variable request.');

const result = await model.raw();

cache: { breakpoint: true } is provider-neutral: GPT-5.6 receives prompt_cache_breakpoint, Anthropic receives cache_control, and unsupported providers omit it. Keep native request policies inside each model shorthand so they do not leak across fallbacks. Anthropic usage separates cacheWrite5m and cacheWrite1h; cacheWrite stays as their compatible aggregate.

GPT-5.6 replaces prompt_cache_retention with prompt_cache_options.ttl. Explicit breakpoints also work on image methods and Responses-native input_text, input_image, and input_file blocks. Prompts need at least 1,024 tokens to be cached. Requests over 272K input tokens use 2× input and 1.5× output prices for the complete request; ModelMix applies these multipliers to cost, costBreakdown, and cache economics.

Add images

const model = ModelMix.new().sonnet45();
model.addImage('./photo.jpg');                          // from file
model.addImageFromUrl('https://example.com/img.png');   // from URL
model.addImageFromBuffer(imageBuffer);                  // from Buffer
model.addText('Describe this image.');
const description = await model.message();

All image methods accept an optional second argument { role } (default "user").

EJS templates

const model = ModelMix.new().gpt5mini();
model.setSystemFromFile('./prompts/system.md');
model.addTextFromFile('./prompts/task.md');
model.assign({
    role: 'data analyst',
    language: 'Spanish'
});
console.log(await model.message());

Templates use standard EJS syntax. Use <%- value %> for raw prompt content and <%= value %> only when XML escaping is intentional. Missing variables and files throw. Templates may contain JavaScript, so the template source must be developer-controlled; untrusted content belongs only in assign() data.

Use assignKey(key, value) for one value and assign({ ... }) for several values.

Start with a static include. Paths are resolved relative to the containing template:

<%- include('shared/rules.md') %>

Use a variable when the included file must be selected dynamically:

Analyze the following source:

<%- include(sourceFile) %>

model.assign({ sourceFile: '../src/utils.js' });

Included files are EJS source, so the path and file must be developer-controlled. Untrusted runtime content belongs in ordinary assign() values, not include paths. To expose a rendered file as a data key, call assignKeyFromFile(key, filePath); it uses EJS include, supports includes relative to that file, and renders once per request. For recursive data, a template may include itself with an explicit stopping condition:

<%- node.text %>
<% if (node.children?.length && depth < maxDepth) { %>
<% for (const child of node.children) { %>
<%- include('tree.ejs', { node: child, depth: depth + 1, maxDepth }) %>
<% } %>
<% } %>

Initialize it with assign({ node, depth: 0, maxDepth: 10 }). Values supplied through assign() remain data and are never interpreted recursively as EJS.

Use ModelMix choice directives for random prompt variants:

<% choice %>
<% option 20 %>
Use emojis.
<% option 40 %>
Use few emojis.
<% option 40 %>
Do not use emojis.
<% /choice %>

Omit all weights for equal probabilities. Otherwise every option needs a positive relative weight; weights do not need to total 100. Keep directives on their own lines. Nested choices and choices inside includes are supported. A request keeps its selections through retries, provider fallbacks, and tool continuations.

Round-robin load balancing

const pool = ModelMix.new({ config: { roundRobin: true } })
    .gpt5mini()
    .sonnet45()
    .gemini38flash();

const r1 = await pool.new().addText("Request 1").message();
const r2 = await pool.new().addText("Request 2").message();

MCP integration

const model = ModelMix.new({ config: { max_history: 10 } }).gpt5nano();
model.setSystem('You are an assistant. Today is ' + new Date().toISOString());
await model.addMCP('@modelcontextprotocol/server-brave-search');
model.addText('Use Internet: What is the latest news about AI?');
console.log(await model.message());

Requires BRAVE_API_KEY in .env for Brave Search MCP.

Custom local tools

const model = ModelMix.new({ config: { max_history: 10 } }).gpt5mini();

model.addTool({
    name: "get_weather",
    description: "Get weather for a city",
    inputSchema: {
        type: "object",
        properties: { city: { type: "string" } },
        required: ["city"]
    }
}, async ({ city }) => {
    return `The weather in ${city} is sunny, 25C`;
});

model.addText("What's the weather in Tokyo?");
console.log(await model.message());

Register multiple tools at once:

model.addTools([
    { tool: { name: "tool_a", description: "...", inputSchema: {...} }, callback: async (args) => {...} },
    { tool: { name: "tool_b", description: "...", inputSchema: {...} }, callback: async (args) => {...} }
]);

Manage tools: model.removeTool("tool_a") and model.listTools() → { local, mcp }.

Rate limiting

const model = ModelMix.new({
    config: {
        bottleneck: {
            maxConcurrent: 4,
            minTime: 1000
        }
    }
}).gpt5mini();

Conversation history

const chat = ModelMix.new({ config: { max_history: 10 } }).gpt5mini();
chat.addText("My name is Martin.");
await chat.message();
chat.addText("What's my name?");
const reply = await chat.message();  // "Martin"

max_history: 0 = no history (default), N = keep last N exchanges, -1 = unlimited.

Debug mode

const model = ModelMix.new({
    config: { debug: 2 }  // 0=silent, 1=minimal, 2=summary, 3=full, 4=verbose
}).gpt5mini();

For full debug output, also set: DEBUG=ModelMix* node script.js

Free-tier models

const model = ModelMix.new()
    .gptOss()
    .kimiK25()
    .addText("What is the capital of France?");
console.log(await model.message());

These use providers with free quotas (Groq, Cerebras, and Together). OpenRouter is disabled because its GPT-OSS 120B route is no longer free. If one runs out of quota, ModelMix falls back to the next.

Multi-provider routing

Some model shorthands register the same model across multiple providers for maximum resilience. Control which providers are enabled via the mix parameter:

const model = ModelMix.new({
    mix: {
        anthropic: true,    // fable51() default: true
        openrouter: false,  // fallback default: false
        cerebras: true,      // default: true
        groq: true,          // default: true
        together: false,     // default: false
        lambda: false,       // default: false
        minimax: false,      // default: false
        fireworks: false     // default: false
    }
}).kimiK25();

Agent Usage Rules

• Check package.json for modelmix before running npm install.
• Use ModelMix.new() static factory (not new ModelMix()).
• Store API keys in .env and load with dotenv/config or process.loadEnvFile(). Never hardcode keys.
• Chain models for resilience: primary model first, fallbacks after.
• When using MCP tools or addTool(), set max_history to at least 3 — tool call/response pairs consume history slots.
• Use .json() for structured output instead of parsing text manually. Use descriptor objects { description, required, enum, default, nullable } for richer schema control.
• Use .message() for simple text, .raw() when you need tokens/thinking/toolCalls.
• For Anthropic thinking, use unified effort (-1 or 0–100) via config.effort or .effort(n) — e.g. .effort(100).opus50(). Never put effort in options. Native fields win if already set.
• Templates use EJS syntax in both system prompts and user messages; prefer <%- key %> for raw prompt data.
• The library uses CommonJS internally but supports ESM import via { ModelMix }.
• GPT-5+ models automatically use max_completion_tokens instead of max_tokens.
• o-series models (o3, o4mini) automatically strip max_tokens and temperature since those APIs don't support them.
• Anthropic Opus 4.7+ / Claude 5 family automatically strip temperature, top_p, and top_k (API rejects them).
• addText(), addImage(), addImageFromUrl(), and addImageFromBuffer() accept { role, cache?: { breakpoint: true } } as the second argument (default role: "user"). Adapters translate or omit the neutral marker by provider.

API Quick Reference

Available Provider Classes

ModerationMix MixModeration MixOpenAI MixOpenAIResponses MixOpenAIModeration MixAnthropic MixGoogle MixPerplexity MixGroq MixTogether MixGrok MixOpenRouter MixOllama MixLMStudio MixCustom MixCerebras MixFireworks MixKimi MixMiniMax MixLambda

Troubleshooting

Model fails with "API key not found" The provider's API key env var is not set. Add it to .env and ensure it loads before ModelMix runs. Each provider looks for its standard env var (e.g. OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI_API_KEY).

Tool calls not working Set max_history to at least 3. Tool call/response pairs are stored in history and the model needs to see them to complete the conversation loop.

JSON response parsing fails Add { addNote: true } to the json() options — this injects instructions about JSON escaping that prevent common parsing errors. For complex schemas, also try { addExample: true }.

Model returns empty or truncated response Increase max_tokens in options. Default is 8192 but some tasks need more. For GPT-5+ models, max_completion_tokens is used automatically.

Rate limit errors Configure Bottleneck: config: { bottleneck: { maxConcurrent: 2, minTime: 2000 } }. This throttles requests to stay within provider limits.

MCP server fails to connect Ensure the MCP package is installed (npm install @modelcontextprotocol/server-brave-search) and required env vars are set. Call addMCP() with await — it's async.

References

• GitHub Repository