Fireworks AI

Fireworks AI Skill

Fast, cost-effective access to 100+ open-source models with OpenAI-compatible APIs, LoRA fine-tuning, and advanced deployment options.

When to Use This Skill

Quick Reference

Chat Completion (OpenAI SDK)

from openai import OpenAI

client = OpenAI(
    base_url="https://api.fireworks.ai/inference/v1",
    api_key="<YOUR_FIREWORKS_API_KEY>",
)

chat_completion = client.chat.completions.create(
    model="accounts/fireworks/models/llama-v3p1-8b-instruct",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Say this is a test"},
    ],
)
print(chat_completion.choices[0].message.content)

Chat Completion (curl)

curl --request POST \
     --url https://api.fireworks.ai/inference/v1/chat/completions \
     --header "accept: application/json" \
     --header "authorization: Bearer $FIREWORKS_API_KEY" \
     --header "content-type: application/json" \
     --data '{
       "model": "accounts/fireworks/models/llama-v3p1-8b-instruct",
       "messages": [{"role": "user", "content": "Hello!"}]
     }'

Supervised Fine-Tuning Job

firectl supervised-fine-tuning-job create \
  --base-model accounts/fireworks/models/llama-v3p1-8b-instruct \
  --dataset my-training-dataset \
  --output-model my-fine-tuned-model \
  --epochs 3 \
  --learning-rate 1e-4 \
  --lora-rank 8

Create Dataset for Fine-Tuning

from fireworks.client import Dataset

dataset = Dataset.from_file(
    "path/to/training_data.jsonl",
    name="my-training-dataset"
)
# Dataset is now available on Fireworks for fine-tuning

Monitor Training Progress

while not job.is_completed:
    job.raise_if_bad_state()
    print(f"Training state: {job.state}")
    time.sleep(10)
    job = job.get()

print(f"Training completed! New model: {job.output_model}")

Deploy Fine-Tuned Model (Multi-LoRA)

from fireworks import LLM

base_model = LLM(
    model="accounts/fireworks/models/llama-v3p2-3b-instruct",
    deployment_type="on-demand",
    id="shared-base-deployment",
    enable_addons=True
)

Generate Embeddings

from openai import OpenAI

client = OpenAI(
    base_url="https://api.fireworks.ai/inference/v1",
    api_key="<YOUR_FIREWORKS_API_KEY>",
)

response = client.embeddings.create(
    model="fireworks/qwen3-embedding-8b",
    input="Your text to embed"
)
embeddings = response.data[0].embedding

Export Billing Metrics

firectl billing export-metrics \
  --start-time "2025-01-01" \
  --end-time "2025-01-31" \
  --filename january_metrics.csv

Create Deployment

firectl deployment create accounts/fireworks/models/deepseek-v3 \
  --deployment-shape throughput

Key Concepts

Fine-Tuning Methods

Decision Tree:

• Have 1000+ labeled examples? → SFT
• Task is verifiable but lacks golden outputs? → RFT
• Want to align with preferences? → DPO

LoRA (Low-Rank Adaptation)

Fireworks uses LoRA for efficient fine-tuning:

• Faster & cheaper - Train in hours, not days
• Easy to deploy - Instant deployment on Fireworks
• Flexible - Run multiple LoRAs on single base deployment

Deployment Types

Agent Tracing (RFT)

For reinforcement fine-tuning with agents:

1. Use model_base_url from trainer (points to tracing.fireworks.ai)
2. Attach FireworksTracingHttpHandler for structured logging
3. Log Status.rollout_finished() or Status.rollout_error() on completion
4. Trainer joins traces + logs via rollout_id

API Compatibility

Fireworks is OpenAI-compatible. Key differences:

Set context_length_exceeded_behavior: "error" for OpenAI-like behavior.

firectl CLI Quick Reference

# Authentication
firectl login

# Account operations
firectl account list

# Dataset operations
firectl dataset download <dataset-id>
firectl dataset list

# Fine-tuning jobs
firectl supervised-fine-tuning-job create --help
firectl supervised-fine-tuning-job list
firectl dpo-job resume <job-id>

# Deployments
firectl deployment create <model> --deployment-shape <shape>
firectl deployment scale <deployment-id> --replicas <n>

# Evaluators
firectl evaluator-revision get <evaluator-id>

# Billing
firectl billing export-metrics

Available Models (Highlights)

Text Models:

• DeepSeek V3, DeepSeek R1
• Llama 3.1/3.2/3.3 (8B, 70B, 405B)
• Qwen 2.5 family
• Kimi K2

Embedding Models:

• fireworks/qwen3-embedding-8b (serverless)
• fireworks/qwen3-embedding-4b
• nomic-ai/nomic-embed-text-v1.5

Reranking Models:

• fireworks/qwen3-reranker-8b (serverless)

Image Models:

• FLUX Kontext Pro/Max
• SDXL ControlNet

Browse all: https://fireworks.ai/models

Reference Files

Navigation tips:

• Search for specific CLI commands: firectl <command>
• API endpoints follow pattern: /v1/accounts/{account_id}/<resource>
• Fine-tuning docs under #fine-tuning-* sections
• Deployment docs under #deployment-* sections

Working with This Skill

For Beginners

1. Start with Chat Completion example above
2. Get API key from https://app.fireworks.ai
3. Use OpenAI SDK (familiar interface)
4. Try serverless models first (no deployment needed)

For Fine-Tuning

1. Prepare JSONL dataset with messages format
2. Upload with Dataset.from_file() or firectl
3. Choose fine-tuning method (SFT/RFT/DPO)
4. Monitor with firectl supervised-fine-tuning-job list
5. Deploy LoRA or merge into base model

For Production

1. Consider on-demand deployments for consistent performance
2. Enable prompt caching for repeated prefixes
3. Use batch inference for offline processing
4. Monitor usage via billing export or dashboard
5. Set up service accounts for CI/CD

Common Patterns

Streaming with Usage Stats

for chunk in client.chat.completions.create(stream=True, ...):
    if chunk.usage:  # Available in final chunk
        print(f"Tokens: {chunk.usage.total_tokens}")

Variable-Length Embeddings

response = client.embeddings.create(
    model="fireworks/qwen3-embedding-8b",
    input="Your text",
    dimensions=128  # Reduce from default for faster similarity
)

Reranking Documents

# Using /rerank endpoint
response = client.post("/rerank", json={
    "model": "fireworks/qwen3-reranker-8b",
    "query": "search query",
    "documents": ["doc1", "doc2", "doc3"]
})

Resources

• Model Library: https://fireworks.ai/models
• Playground: https://app.fireworks.ai/playground
• Usage Dashboard: https://app.fireworks.ai/account/usage
• API Reference: https://docs.fireworks.ai/api-reference
• firectl Docs: https://docs.fireworks.ai/tools-sdks/firectl

Notes

• Generated from official Fireworks AI documentation (410 pages)
• OpenAI SDK examples work directly with Fireworks
• Model names use accounts/fireworks/models/<model-name> format
• Fine-tuning uses LoRA by default (set --lora-rank 0 for full parameter)