1667 Groq Integration 0475efe8

Groq Integration - LLM CLI Skill

Overview

The LLM CLI Skill has been enhanced with support for Groq's fast Llama models. This enables you to use free, high-speed inference with Meta's Llama models through the Groq platform.

What Was Added

1. Provider Detection

• Added GROQ_API_KEY environment variable detection
• Groq automatically detected when API key is set
• Setup suggestions include Groq instructions

2. Model Registry

Added three Groq Llama models:

3. Provider Aliases

• groq → Groq provider
• llama → Groq provider (shorthand)

4. Model Aliases

• groq-llama-3.3 → groq-llama-3.3-70b
• groq-llama → groq-llama-3.3-70b
• llama-3.3-70b → groq-llama-3.3-70b

Setup & Usage

1. Get API Key (Free)

Visit: https://console.groq.com/keys

No credit card required!

2. Export API Key

export GROQ_API_KEY='gsk_...'

Or use llm CLI to store it:

llm keys set groq

3. Verify Setup

llm models | grep groq

You should see available Groq models listed.

4. Use the Skill

Direct llm CLI (Recommended)

llm -m groq-llama-3.3-70b "Your prompt here"

With LLM Skill

/llm "Your prompt" --model groq-llama-3.3-70b

By Provider

/llm "Your prompt" --model groq
# Will show menu of available Groq models

By Alias

/llm "Your prompt" --model llama
# Uses groq-llama-3.3-70b by default

Example: IFS Analysis

The skill was tested with:

export GROQ_API_KEY='gsk_your_api_key_here'
llm -m groq-llama-3.3-70b "Give me 10 key critical ideas about IFS"

Result: Successfully generated 10 comprehensive points about Internal Family Systems therapy in ~15 seconds.

Get your own free API key at: https://console.groq.com/keys

Key Benefits

✅ Free Inference

• No credit card required
• Generous free tier
• Great for prototyping and testing

✅ Lightning Fast

• Sub-second response times for most queries
• 70B model running at incredible speeds
• Perfect for interactive use

✅ Quality Models

• Meta's Llama 3.3 70B (state-of-the-art open model)
• Llama 3.1 8B (lightweight but capable)
• Both instruction-tuned for chat

✅ Simple Integration

• Works seamlessly with llm CLI
• No code changes needed
• Automatic model detection

Files Modified

1. models.py

   • Added 3 Groq Llama models
   • Added groq and llama provider aliases

2. providers.py

   • Added GROQ_API_KEY detection
   • Updated setup suggestions

Performance Characteristics

Llama 3.3 70B (groq-llama-3.3-70b)

• Speed: ⚡⚡⚡ Lightning fast
• Quality: ⭐⭐⭐⭐⭐ Excellent
• Cost: 🆓 Free
• Use Case: Best all-around choice

Llama 3.1 8B (groq-llama-3.1-8b)

• Speed: ⚡⚡⚡⚡⚡ Blazing fast
• Quality: ⭐⭐⭐⭐ Very good
• Cost: 🆓 Free
• Use Case: Maximum speed, lightweight tasks

Comparison with Other Providers

Usage Examples

Quick Test

llm -m groq-llama-3.3-70b "Hello, what's your name?"

File Processing

cat document.txt | llm -m groq-llama-3.3-70b "Summarize this"

Code Analysis

llm -m groq-llama-3.3-70b "Review this code for bugs" < main.py

Complex Tasks

llm -m groq-llama-3.3-70b "Explain quantum computing in 3 paragraphs"

Interactive Chat

llm -m groq-llama-3.3-70b --conversation
# Chat until you exit

API Key Security

• API key stored in environment variable only (not in config files)
• Never committed to git
• Can be revoked anytime from Groq console
• Consider creating separate key for different use cases

Troubleshooting

API Key Not Recognized

# Verify key is set
echo $GROQ_API_KEY

# Should show your key (first 10 chars)
echo ${GROQ_API_KEY:0:10}

# If empty, set it again
export GROQ_API_KEY='gsk_...'

# And reload if in ~/.bashrc or ~/.zshrc
source ~/.zshrc

Model Not Found

# List all Groq models
llm models | grep groq

# Should show groq models available

Rate Limiting

Groq's free tier has rate limits. If you hit them:

• Wait a few minutes
• Use Llama 3.1 8B (lighter weight)
• Create another free account (if needed)

Advanced Usage

Use with Skill Selection Menu

/llm "Your prompt"
# When asked for provider, select "groq"
# Choose preferred model from menu

Set as Default

Edit ~/.claude/llm-skill-config.json:

{
  "last_model": "groq-llama-3.3-70b",
  "last_provider": "groq"
}

Then just use:

llm "Your prompt"
# Uses Groq by default

Next Steps

1. Get API Key: https://console.groq.com/keys
2. Set Environment: export GROQ_API_KEY='gsk_...'
3. Verify: llm models | grep groq
4. Start Using: llm -m groq-llama-3.3-70b "prompt"

Resources

• Groq Console: https://console.groq.com
• Groq Docs: https://console.groq.com/docs
• LLM CLI Docs: https://llm.datasette.io
• Llama Models: https://www.llama.com

Summary

Groq integration enables you to:

• ✅ Access powerful Llama models for FREE
• ✅ Get lightning-fast inference speeds
• ✅ Use with the LLM CLI skill seamlessly
• ✅ No code changes needed
• ✅ Works alongside OpenAI, Anthropic, Google

Recommended: Use groq-llama-3.3-70b as your default for best balance of speed and quality!

Status: ✅ Production Ready Last Updated: November 3, 2025 Version: 1.0.0