Dspy Gepa Optimizer

GEPA Optimizer

🎯 Trigger Conditions

Use when asked about GEPA (Generative Evaluation-Preserving Optimization), advanced DSPy optimization, or when standard optimizers are insufficient.

📚 Prerequisites

• dspy package installed
• Training data with gold labels
• Clear evaluation objectives

🛠️ GEPA API

1. Basic GEPA Usage

from dspy import GEPA

# Create GEPA optimizer
gepa = GEPA(
    metric=your_metric,
    max_budget=100,
    temperature=0.7,
    seed=42
)

# Optimize program
optimized = gepa.optimize(
    program=your_program,
    trainset=train_data,
    max_rounds=10
)

2. GEPA Configuration

gepa = GEPA(
    metric=your_metric,
    # Budget control
    max_budget=100,
    max_rounds=10,
    # Sampling control
    temperature=0.7,
    top_p=0.9,
    # Optimization strategy
    strategy="generate-and-rerank",
    # Caching
    use_cache=True,
    cache_path="./gepa_cache"
)

3. Custom GEPA Strategies

from dspy.optimizers.gepa import GenerateAndRerank, EvolveAndSelect

# Generate and rerank
strategy = GenerateAndRerank(
    n_candidates=10,
    reranker=your_reranker
)

# Evolve and select
strategy = EvolveAndSelect(
    population_size=20,
    mutation_rate=0.3,
    crossover_rate=0.4
)

gepa = GEPA(metric=your_metric, strategy=strategy)

4. GEPA with Fine-Tuning

from dspy import GEPA, BootstrapFinetune

# Combine GEPA with fine-tuning
gepa = GEPA(metric=your_metric)
finetune = BootstrapFinetune(
    metric=your_metric,
    model_path="your-model-path"
)

# Two-stage optimization
optimized = gepa.optimize(program, train_data)
final = finetune.optimize(optimized, train_data)

⚠️ Pitfalls

• Compute cost: GEPA can be expensive
• Overfitting: Monitor dev set performance
• Cache management: Clear stale cache regularly
• Strategy selection: Choose strategy based on data size

📖 References

• GEPA Documentation
• GEPA GitHub