Grok API

Build or refine a Grok API agent: $ARGUMENTS

$ARGUMENTS can include:

• What to build (e.g. function calling agent, 2M context RAG, streaming chatbot, multi-agent pipeline)
• A file path to existing code to refine
• A capability to add (e.g. add function calling, add structured output, maximize context, add reasoning)
• A model preference (e.g. --model grok-4.20, --model grok-4-1-fast, --model grok-4.20-multi-agent)
• A mode preference (e.g. --mode reasoning, --mode non-reasoning)
• Empty — scaffold a starter agent with best-practice defaults

Authoritative Sources

Always fetch fresh docs before generating or advising on code. These are the canonical xAI sources:

Note: fetch https://docs.x.ai/docs first to discover current guide URLs — xAI's docs structure evolves frequently.

Fetch the model overview before writing any code. xAI is releasing models frequently and pricing/IDs change.

Environment Setup

# Reuse the OpenAI SDK — Grok is fully OpenAI-compatible
pnpm add openai

# .env.local
XAI_API_KEY=xai-...

// src/lib/ai/grok.ts
import OpenAI from 'openai';

export const grok = new OpenAI({
  apiKey: process.env.XAI_API_KEY,
  baseURL: 'https://api.x.ai/v1',
});

Current model IDs (verify against live docs before using):

Key facts:

• #1 LMArena Text Arena (1483 Elo) — grok-4-1-fast-reasoning
• All models: 2M token context window (largest available)
• OpenAI-compatible: swap baseURL, reuse OpenAI SDK — no migration cost
• Cached input: 90% off (from $2.00 → $0.20 on grok-4.20, from $0.20 → $0.05 on grok-4-1-fast)
• Batch API available at 50% off (confirmed on models page)
• logprobs not supported on grok-4.20 models

Phase 1 — Load Context

1. Fetch the model overview to confirm current model IDs and pricing
2. Read any existing code files specified in $ARGUMENTS
3. Read _docs/ai-intelligence/ for latest snapshot if available
4. Determine model tier:
   • grok-4-1-fast → high-volume tasks, chatbots, RAG, content generation at commodity price
   • grok-4.20 → complex reasoning, multi-step agents, high-accuracy tasks
   • grok-4.20-multi-agent → orchestrator role in multi-agent systems (only dedicated multi-agent model on market)
5. Determine reasoning mode:
   • reasoning → better accuracy, slower, slightly more tokens
   • non-reasoning → fastest output, use for latency-sensitive tasks

Phase 2 — Core Patterns

Pattern 1: Basic Completion

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: process.env.XAI_API_KEY,
  baseURL: 'https://api.x.ai/v1',
});

const response = await client.chat.completions.create({
  model: 'grok-4-1-fast-reasoning',
  messages: [
    { role: 'system', content: 'You are a helpful assistant.' },
    { role: 'user', content: 'Hello!' },
  ],
});

console.log(response.choices[0].message.content);
console.log('Tokens:', response.usage);

Pattern 2: Function Calling

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: process.env.XAI_API_KEY,
  baseURL: 'https://api.x.ai/v1',
});

const tools: OpenAI.ChatCompletionTool[] = [
  {
    type: 'function',
    function: {
      name: 'search_knowledge_base',
      description: 'Search the internal knowledge base for relevant information.',
      parameters: {
        type: 'object',
        properties: {
          query: {
            type: 'string',
            description: 'The search query',
          },
          top_k: {
            type: 'number',
            description: 'Number of results to return (default: 5)',
          },
        },
        required: ['query'],
        additionalProperties: false,
      },
      strict: true,
    },
  },
];

async function runWithTools(userMessage: string) {
  const messages: OpenAI.ChatCompletionMessageParam[] = [
    { role: 'user', content: userMessage },
  ];

  while (true) {
    const response = await client.chat.completions.create({
      model: 'grok-4-1-fast-reasoning',
      tools,
      tool_choice: 'auto',
      messages,
    });

    const choice = response.choices[0];
    messages.push(choice.message);

    if (choice.finish_reason === 'stop') {
      return choice.message.content;
    }

    if (choice.finish_reason !== 'tool_calls') break;

    // Execute all tool calls in parallel
    const toolResults = await Promise.all(
      (choice.message.tool_calls ?? []).map(async (toolCall) => {
        const args = JSON.parse(toolCall.function.arguments);
        const result = await executeTool(toolCall.function.name, args);
        return {
          role: 'tool' as const,
          tool_call_id: toolCall.id,
          content: JSON.stringify(result),
        };
      })
    );

    messages.push(...toolResults);
  }
}

async function executeTool(
  name: string,
  args: Record<string, unknown>,
): Promise<unknown> {
  if (name === 'search_knowledge_base') {
    // Replace with real search implementation
    return { results: [], query: args.query };
  }
  throw new Error(`Unknown tool: ${name}`);
}

Pattern 3: Structured Output (JSON Schema)

import { z } from 'zod';
import { zodResponseFormat } from 'openai/helpers/zod';

const ArticleSchema = z.object({
  title: z.string(),
  summary: z.string(),
  topics: z.array(z.string()),
  sentiment: z.enum(['positive', 'neutral', 'negative']),
});

const response = await client.beta.chat.completions.parse({
  model: 'grok-4-1-fast-non-reasoning',
  messages: [
    { role: 'system', content: 'Extract article metadata accurately.' },
    { role: 'user', content: articleText },
  ],
  response_format: zodResponseFormat(ArticleSchema, 'article'),
});

const article = response.choices[0].message.parsed;
// Typed as z.infer<typeof ArticleSchema>

Manual JSON schema (without Zod):

const response = await client.chat.completions.create({
  model: 'grok-4-1-fast-non-reasoning',
  messages: [...],
  response_format: {
    type: 'json_schema',
    json_schema: {
      name: 'extraction',
      strict: true,
      schema: {
        type: 'object',
        properties: {
          title: { type: 'string' },
          topics: { type: 'array', items: { type: 'string' } },
        },
        required: ['title', 'topics'],
        additionalProperties: false,
      },
    },
  },
});

const data = JSON.parse(response.choices[0].message.content ?? '{}');

Pattern 4: Streaming

const stream = await client.chat.completions.stream({
  model: 'grok-4-1-fast-reasoning',
  messages: [{ role: 'user', content: 'Write a detailed analysis...' }],
});

for await (const chunk of stream) {
  const delta = chunk.choices[0]?.delta?.content;
  if (delta) process.stdout.write(delta);
}

const finalCompletion = await stream.finalChatCompletion();
console.log('Total tokens:', finalCompletion.usage?.total_tokens);

Next.js streaming (App Router):

// app/api/chat/route.ts
import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: process.env.XAI_API_KEY,
  baseURL: 'https://api.x.ai/v1',
});

export async function POST(req: Request) {
  const { messages } = await req.json();

  const stream = await client.chat.completions.stream({
    model: 'grok-4-1-fast-reasoning',
    messages,
  });

  return new Response(stream.toReadableStream(), {
    headers: { 'Content-Type': 'text/event-stream' },
  });
}

Pattern 5: 2M Context — Direct Document Injection

Grok's 2M context makes traditional RAG chunking optional for most document sets.

import fs from 'fs';

// Load entire document corpus directly into context
function loadDocuments(paths: string[]): string {
  return paths
    .map(p => {
      const content = fs.readFileSync(p, 'utf-8');
      return `<document path="${p}">\n${content}\n</document>`;
    })
    .join('\n\n');
}

const documents = loadDocuments([
  './docs/guide.md',
  './docs/api-reference.md',
  './docs/changelog.md',
  // Add more — up to ~2M tokens total
]);

const response = await client.chat.completions.create({
  model: 'grok-4-1-fast-reasoning',
  messages: [
    {
      role: 'system',
      content: `You have access to the following documentation:\n\n${documents}`,
    },
    { role: 'user', content: 'How do I configure authentication?' },
  ],
});

When to use direct injection vs RAG:

• Direct injection: document set < 1.5M tokens, need cross-document reasoning, simpler architecture
• RAG: document set > 1.5M tokens, need citation precision, updates frequently

Pattern 6: Prompt Caching (90% off cached tokens)

Cache reused system prompts and context to dramatically reduce costs on repeated requests.

// Grok supports prompt caching — cached tokens cost $0.05/M vs $0.20/M on grok-4-1-fast
// Cache is automatic when the same prefix is reused across requests

// Pattern: put the large, stable content at the START of the system message
// Grok caches the longest common prefix automatically
const largeSystemPrompt = `
You are an expert assistant with deep knowledge of the following documentation:

${LARGE_DOCUMENT_CONTEXT}  // This prefix will be cached after first request

Always cite specific sections when answering questions.
`;

// First request — cache miss, full price
const response1 = await client.chat.completions.create({
  model: 'grok-4-1-fast-reasoning',
  messages: [
    { role: 'system', content: largeSystemPrompt },
    { role: 'user', content: 'Question 1' },
  ],
});

// Subsequent requests — cache hit on the shared prefix, 90% cheaper
const response2 = await client.chat.completions.create({
  model: 'grok-4-1-fast-reasoning',
  messages: [
    { role: 'system', content: largeSystemPrompt },  // Same prefix → cache hit
    { role: 'user', content: 'Question 2' },
  ],
});

// Check cache usage
console.log('Cache hit tokens:', response2.usage?.prompt_tokens_details?.cached_tokens);

Pattern 7: Multi-Turn Conversation

class GrokConversation {
  private messages: OpenAI.ChatCompletionMessageParam[] = [];

  constructor(
    private client: OpenAI,
    private systemPrompt: string,
    private model = 'grok-4-1-fast-reasoning',
  ) {}

  async send(userContent: string): Promise<string> {
    this.messages.push({ role: 'user', content: userContent });

    const response = await this.client.chat.completions.create({
      model: this.model,
      messages: [
        { role: 'system', content: this.systemPrompt },
        ...this.messages,
      ],
    });

    const assistantMessage = response.choices[0].message;
    this.messages.push(assistantMessage);
    return assistantMessage.content ?? '';
  }

  reset() {
    this.messages = [];
  }
}

Phase 3 — Agent Patterns

Autonomous Tool-Loop Agent

async function runGrokAgent(
  task: string,
  tools: OpenAI.ChatCompletionTool[],
  toolHandlers: Record<string, (args: Record<string, unknown>) => Promise<unknown>>,
  model = 'grok-4.20-0309-reasoning',
  maxTurns = 15,
): Promise<string> {
  const messages: OpenAI.ChatCompletionMessageParam[] = [
    { role: 'user', content: task },
  ];

  for (let turn = 0; turn < maxTurns; turn++) {
    const response = await client.chat.completions.create({
      model,
      tools,
      tool_choice: 'auto',
      messages,
    });

    const choice = response.choices[0];
    messages.push(choice.message);

    if (choice.finish_reason === 'stop') {
      return choice.message.content ?? '';
    }

    if (choice.finish_reason !== 'tool_calls') break;

    const toolResults = await Promise.all(
      (choice.message.tool_calls ?? []).map(async (tc) => {
        const handler = toolHandlers[tc.function.name];
        if (!handler) throw new Error(`No handler for: ${tc.function.name}`);
        const output = await handler(JSON.parse(tc.function.arguments));
        return {
          role: 'tool' as const,
          tool_call_id: tc.id,
          content: JSON.stringify(output),
        };
      })
    );

    messages.push(...toolResults);
  }

  throw new Error('Agent exceeded max turns');
}

Multi-Agent Orchestration (grok-4.20-multi-agent)

The grok-4.20-multi-agent model is the only dedicated multi-agent model variant on the market. Use it as the orchestrator.

const orchestratorTools: OpenAI.ChatCompletionTool[] = [
  {
    type: 'function',
    function: {
      name: 'run_researcher',
      description: 'Delegate a research task to the researcher subagent.',
      parameters: {
        type: 'object',
        properties: {
          task: { type: 'string', description: 'The research task to perform' },
        },
        required: ['task'],
        additionalProperties: false,
      },
      strict: true,
    },
  },
  {
    type: 'function',
    function: {
      name: 'run_writer',
      description: 'Delegate a writing task to the writer subagent, with research context.',
      parameters: {
        type: 'object',
        properties: {
          task: { type: 'string' },
          context: { type: 'string', description: 'Research findings to incorporate' },
        },
        required: ['task', 'context'],
        additionalProperties: false,
      },
      strict: true,
    },
  },
];

const orchestratorHandlers = {
  run_researcher: async ({ task }: { task: string }) =>
    runGrokAgent(task, researcherTools, researcherHandlers, 'grok-4-1-fast-reasoning'),
  run_writer: async ({ task, context }: { task: string; context: string }) =>
    runGrokAgent(
      `${task}\n\nResearch context:\n${context}`,
      writerTools,
      writerHandlers,
      'grok-4-1-fast-reasoning',
    ),
};

// Use the multi-agent model as orchestrator
const result = await runGrokAgent(
  'Research recent AI developments and write a 1000-word summary.',
  orchestratorTools,
  orchestratorHandlers,
  'grok-4.20-multi-agent-0309',  // dedicated orchestrator model
);

RAG with 2M Context (Simplified Architecture)

// With 2M context, you can skip chunking for most use cases
class GrokRAG {
  private client: OpenAI;
  private documents: Map<string, string> = new Map();

  constructor() {
    this.client = new OpenAI({
      apiKey: process.env.XAI_API_KEY,
      baseURL: 'https://api.x.ai/v1',
    });
  }

  addDocument(name: string, content: string) {
    this.documents.set(name, content);
  }

  buildContext(): string {
    return Array.from(this.documents.entries())
      .map(([name, content]) => `<document name="${name}">\n${content}\n</document>`)
      .join('\n\n');
  }

  async query(question: string): Promise<string> {
    const context = this.buildContext();
    // Estimate tokens: ~0.75 tokens per word, or use tiktoken
    const estimatedTokens = context.split(' ').length * 0.75;
    if (estimatedTokens > 1_800_000) {
      throw new Error('Document set exceeds safe 2M context limit. Implement chunking.');
    }

    const response = await this.client.chat.completions.create({
      model: 'grok-4-1-fast-reasoning',
      messages: [
        {
          role: 'system',
          content: `You are a helpful assistant. Answer questions based on the provided documents.\n\n${context}`,
        },
        { role: 'user', content: question },
      ],
    });

    return response.choices[0].message.content ?? '';
  }
}

Phase 4 — Migrating from OpenAI

Since Grok uses the OpenAI-compatible API, migration is a 2-line change:

// BEFORE (OpenAI)
const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
const model = 'gpt-5.4-mini';

// AFTER (Grok)
const client = new OpenAI({
  apiKey: process.env.XAI_API_KEY,
  baseURL: 'https://api.x.ai/v1',
});
const model = 'grok-4-1-fast-reasoning';

Known differences from OpenAI:

• logprobs not supported on grok-4.20 models
• No Batch API (use real-time only)
• No DALL-E or image generation in the text API (separate grok-imagine-image endpoint)
• No Assistants API / Threads API
• No fine-tuning (as of last research — verify at docs.x.ai)

Phase 5 — Error Handling & Rate Limits

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: process.env.XAI_API_KEY,
  baseURL: 'https://api.x.ai/v1',
});

async function callWithRetry<T>(
  fn: () => Promise<T>,
  maxRetries = 3,
): Promise<T> {
  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    try {
      return await fn();
    } catch (error) {
      if (error instanceof OpenAI.RateLimitError) {
        const delay = Math.pow(2, attempt) * 1000;
        console.warn(`Rate limited. Retrying in ${delay}ms...`);
        await new Promise(r => setTimeout(r, delay));
        continue;
      }
      if (error instanceof OpenAI.APIError && error.status >= 500 && attempt < maxRetries) {
        await new Promise(r => setTimeout(r, Math.pow(2, attempt) * 1000));
        continue;
      }
      throw error;
    }
  }
  throw new Error('Max retries exceeded');
}

Phase 6 — Image Generation (Separate API)

Grok has a dedicated image generation API separate from the text models.

// Image generation — NOT OpenAI-compatible, use fetch directly
const response = await fetch('https://api.x.ai/v1/images/generations', {
  method: 'POST',
  headers: {
    'Authorization': `Bearer ${process.env.XAI_API_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    model: 'grok-imagine-image',    // or 'grok-imagine-image-pro'
    prompt: 'A futuristic city at sunset',
    n: 1,
  }),
});

const data = await response.json();
console.log(data.data[0].url);

Image pricing: grok-imagine-image = $0.02/image (300 RPM), grok-imagine-image-pro = $0.07/image (30 RPM).

Phase 7 — Cost Reference (verify at docs.x.ai)

No Batch API — all requests real-time.

Cost formula:

cost = (input_tokens / 1_000_000 × input_price)
     + (output_tokens / 1_000_000 × output_price)
     - (cached_tokens / 1_000_000 × (input_price - cached_price))

Strategic note: grok-4-1-fast at $0.20/$0.50 per 1M is the most disruptive value in the market as of Q1 2026. At 93% cheaper than Claude Sonnet / GPT-5.4-mini on input, any task currently on a mid-tier model should be evaluated on grok-4-1-fast first.

Output

When building or scaffolding:

1. Fetch the model overview URL to confirm current model IDs
2. Read any existing code files before modifying
3. Default to grok-4-1-fast-reasoning for most tasks — only upgrade to grok-4.20 for complex multi-step reasoning
4. Use grok-4.20-multi-agent-0309 when building multi-agent orchestrators
5. Always use OpenAI SDK with baseURL override — no new SDK needed
6. Include caching pattern when the system prompt is large and reused
7. Note any OpenAI features that are NOT supported (logprobs, Batch API, Assistants API)
8. Write files to their appropriate path (e.g. src/lib/ai/, src/agents/, src/app/api/)

Always state which model ID you used, reasoning vs non-reasoning mode, and why.