Groq Multi Env Setup

Groq Multi-Environment Setup

Overview

Configure Groq across environments with the right balance of cost, speed, and capability per tier. Groq's key differentiator is inference speed (100-300 tokens/second), but rate limits differ dramatically by plan: free tier is 30 RPM / 14,400 RPD for llama-3.1-70b, while paid tier removes most limits.

Prerequisites

• Groq API key(s) per environment from console.groq.com
• Environment variable management (.env.local, GitHub Secrets, or cloud secret manager)
• Understanding of Groq's model tiers and rate limits

Environment Strategy

Instructions

Step 1: Configuration Structure

config/
  groq/
    base.ts           # Shared Groq client setup
    development.ts    # Dev: fast small models, verbose logging
    staging.ts        # Staging: production models, test rate limits
    production.ts     # Prod: hardened retry, error handling
    index.ts          # Environment resolver

Step 2: Base Configuration with Groq SDK

// config/groq/base.ts
import Groq from "groq-sdk";

export const BASE_GROQ_CONFIG = {
  maxRetries: 3,
  timeout: 30000,  # 30000: 30 seconds in ms
};

Step 3: Environment-Specific Configs

// config/groq/development.ts
export const devConfig = {
  ...BASE_GROQ_CONFIG,
  apiKey: process.env.GROQ_API_KEY,
  model: "llama-3.1-8b-instant",      // fastest, cheapest for dev iteration
  maxTokens: 1024,  # 1024: 1 KB
  temperature: 0.7,
  logRequests: true,                   // verbose logging in dev
};

// config/groq/staging.ts
export const stagingConfig = {
  ...BASE_GROQ_CONFIG,
  apiKey: process.env.GROQ_API_KEY_STAGING,
  model: "llama-3.1-70b-versatile",   // match production model
  maxTokens: 4096,  # 4096: 4 KB
  temperature: 0.3,
  logRequests: false,
};

// config/groq/production.ts
export const productionConfig = {
  ...BASE_GROQ_CONFIG,
  apiKey: process.env.GROQ_API_KEY_PROD,
  model: "llama-3.1-70b-versatile",   // or llama-3.3-70b-specdec for faster
  maxTokens: 4096,  # 4 KB
  temperature: 0.3,
  maxRetries: 5,                       // more retries for production reliability
  logRequests: false,
};

Step 4: Environment Resolver with Groq Client

// config/groq/index.ts
import Groq from "groq-sdk";

type Env = "development" | "staging" | "production";

function detectEnvironment(): Env {
  const env = process.env.NODE_ENV || "development";
  if (env === "production") return "production";
  if (env === "staging") return "staging";
  return "development";
}

let _client: Groq | null = null;

export function getGroqClient(): Groq {
  if (_client) return _client;

  const env = detectEnvironment();
  const configs = { development: devConfig, staging: stagingConfig, production: productionConfig };
  const config = configs[env];

  if (!config.apiKey) {
    throw new Error(`GROQ_API_KEY not configured for ${env} environment`);
  }

  _client = new Groq({
    apiKey: config.apiKey,
    maxRetries: config.maxRetries,
    timeout: config.timeout,
  });

  return _client;
}

export function getModelConfig() {
  const env = detectEnvironment();
  const configs = { development: devConfig, staging: stagingConfig, production: productionConfig };
  return configs[env];
}

Step 5: Usage with Rate Limit Handling

// lib/groq-service.ts
import { getGroqClient, getModelConfig } from "../config/groq";

export async function complete(prompt: string): Promise<string> {
  const groq = getGroqClient();
  const { model, maxTokens, temperature } = getModelConfig();

  try {
    const completion = await groq.chat.completions.create({
      model,
      messages: [{ role: "user", content: prompt }],
      max_tokens: maxTokens,
      temperature,
    });
    return completion.choices[0].message.content || "";
  } catch (err: any) {
    if (err.status === 429) {  # HTTP 429 Too Many Requests
      const retryAfter = parseInt(err.headers?.["retry-after"] || "10");
      console.warn(`Groq rate limited. Retry after ${retryAfter}s`);
      throw new Error(`Rate limited on model ${model}. Retry after ${retryAfter}s`);
    }
    throw err;
  }
}

Error Handling

Examples

Check Which Config Is Active

import { getModelConfig } from "./config/groq";

const cfg = getModelConfig();
console.log(`Model: ${cfg.model}, max_tokens: ${cfg.maxTokens}`);

Test Rate Limits Per Environment

set -euo pipefail
# Quick check: what's my current rate limit status?
curl -s "https://api.groq.com/openai/v1/models" \
  -H "Authorization: Bearer $GROQ_API_KEY" | jq '.data[].id'

Resources

• Groq API Documentation
• Groq Models Reference
• Groq Rate Limits by Tier

Next Steps

For deployment configuration, see groq-deploy-integration.

Output

• Configuration files or code changes applied to the project
• Validation report confirming correct implementation
• Summary of changes made and their rationale