Groq Migration Deep Dive

Groq Migration Deep Dive

Current State

!npm list groq-sdk openai @anthropic-ai/sdk 2>/dev/null | grep -E "groq|openai|anthropic" || echo 'No LLM SDKs found'

Overview

Migrate to Groq from OpenAI, Anthropic, or other LLM providers. Groq's OpenAI-compatible API makes migration straightforward — the primary changes are a different SDK import, different model IDs, and different response metadata. The reward is 10-50x faster inference.

The safe path is a provider-abstraction layer plus feature-flagged traffic shifting: route a small canary to Groq, benchmark quality and speed, ramp to 100%, and keep a one-flag rollback the whole way.

Migration Complexity

Prerequisites

• A Groq API key (GROQ_API_KEY) from console.groq.com.
• groq-sdk installed: npm install groq-sdk.
• A feature-flag mechanism (LaunchDarkly, env var, config service) exposing a groq_migration_pct value for gradual traffic shifting.
• The existing provider's key still available (OPENAI_API_KEY or equivalent) so you can run both providers side-by-side during the cutover.
• Node >=18 if you use the performance.now() benchmark helper.

Instructions

Steps 1-2 below are the essential skeleton — the two changes every migration needs. Steps 3-7 (the provider abstraction, traffic shifting, scanner, benchmark, and full compatibility matrix) are moved verbatim into the reference files linked under Examples so this file stays scannable.

Step 1: OpenAI to Groq Migration

The minimal change: swap the SDK import, client, and model ID. The response shape is identical, so downstream code (result.choices[0].message.content) is untouched.

// BEFORE: OpenAI
import OpenAI from "openai";
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
const result = await openai.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "Hello" }],
});

// AFTER: Groq (minimal changes)
import Groq from "groq-sdk";
const groq = new Groq({ apiKey: process.env.GROQ_API_KEY });
const result = await groq.chat.completions.create({
  model: "llama-3.3-70b-versatile",  // or "llama-3.1-8b-instant"
  messages: [{ role: "user", content: "Hello" }],
});

// Same response shape: result.choices[0].message.content

Step 2: Model ID Mapping

Centralize the OpenAI/Anthropic → Groq translation so a single map drives the whole codebase and unknown models fall back to a safe default.

// OpenAI → Groq model equivalents
const MODEL_MAP: Record<string, string> = {
  // OpenAI → Groq (quality equivalent)
  "gpt-4o":        "llama-3.3-70b-versatile",
  "gpt-4o-mini":   "llama-3.1-8b-instant",
  "gpt-4-turbo":   "llama-3.3-70b-versatile",
  "gpt-3.5-turbo": "llama-3.1-8b-instant",

  // Anthropic → Groq (approximate)
  "claude-3-5-sonnet": "llama-3.3-70b-versatile",
  "claude-3-haiku":    "llama-3.1-8b-instant",
};

function migrateModelId(model: string): string {
  return MODEL_MAP[model] || "llama-3.3-70b-versatile";
}

Steps 3-7: Zero-Downtime Rollout

Once Steps 1-2 compile, wrap both providers in a common interface and shift traffic gradually. The full code lives in the references:

• Step 3 — Provider abstraction layer and Step 4 — feature-flag traffic shifting: references/implementation.md.
• Step 5 — automated migration scanner (sizes the migration before you start) and the rollback plan: references/implementation.md.
• Step 6 — comparison benchmark and Step 7 — the OpenAI↔Groq compatibility matrix: references/examples.md.

Output

Running this skill's workflow produces:

• A migration assessment printout from the scanner (Step 5): OpenAI import count, the distinct gpt-* model IDs in use, any OpenAI-only features (embeddings/images/fine-tuning) that block a clean cutover, and the number of API-key references to update.
• A provider-agnostic LLMProvider layer with GroqProvider and OpenAIProvider implementations both live behind one getProvider() call.
• A benchmark table per prompt: Groq vs OpenAI latency in ms, token counts, and the measured speedup factor.
• A groq_migration_pct feature flag driving the canary → 100% ramp, with a one-flag rollback to 0%.

Error Handling

Examples

• Full provider abstraction + traffic shifting + scanner + rollback: references/implementation.md — the complete Step 3-5 code plus the rollback procedure.
• Benchmark harness + compatibility matrix: references/examples.md — the side-by-side quality/speed benchmark and the full OpenAI↔Groq feature table.

Resources

• Groq Quickstart
• Groq Models
• Groq API Reference
• groq-sdk npm

Next Steps

For ongoing SDK version upgrades between Groq model generations, see the companion groq-upgrade-migration skill in this pack.