Groq

Groq API Skill

Overview

Groq delivers real-time AI inference at hundreds of tokens per second using Language Processing Units (LPUs). Ideal for conversational agents, real-time voice loops, and streaming applications.

Installation

npm install groq-sdk
pip install groq

Authentication & Setup

import Groq from 'groq-sdk';

const groq = new Groq({
  apiKey: process.env.GROQ_API_KEY,
});

Core API Operations

1. High-Speed Chat Completion Streaming

const stream = await groq.chat.completions.create({
  model: 'llama-3.3-70b-versatile',
  messages: [
    { role: 'system', content: 'You are a responsive assistant.' },
    { role: 'user', content: 'Generate a short poem about distributed systems.' },
  ],
  stream: true,
  temperature: 0.5,
  max_tokens: 1024,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content || '');
}

2. JSON Mode Structured Generation

const response = await groq.chat.completions.create({
  model: 'llama-3.3-70b-versatile',
  messages: [
    { role: 'system', content: 'You must respond with valid JSON.' },
    { role: 'user', content: 'Extract skills: "Node.js, PostgreSQL, Redis"' },
  ],
  response_format: { type: 'json_object' },
});

const data = JSON.parse(response.choices[0].message.content || '{}');

3. Audio Transcription (Whisper LPU)

import fs from 'fs';

const transcription = await groq.audio.transcriptions.create({
  file: fs.createReadStream('audio.mp3'),
  model: 'whisper-large-v3-turbo',
  response_format: 'verbose_json',
});

console.log(transcription.text);

AI Pitfalls & Anti-Hallucination Guidelines

• Model Deprecations: Groq retires preview checkpoints frequently. Use standard aliases such as llama-3.3-70b-versatile or mixtral-8x7b-32768.
• Rate Limits on Free Tier: The free tier has strict RPM/TPM limits. Implement retry queues with exponential backoff on 429 status codes.

Production Verification Checklist

• Streaming connection handled with client abort signals
• Audio files stay under the 25MB request limit
• Fallback models configured in case of provider rate-limiting

Last Verified: 2026-07-03