Optimization Prompt Tuning

Prompt Optimization

🎯 Trigger Conditions

Use when asked about prompt engineering, prompt optimization, or improving DSPy signature quality.

📚 Prerequisites

• dspy package installed
• Evaluation metric defined
• Prompt variants to test

🛠️ Prompt Optimization Techniques

1. Bootstrap Few-Shot

from dspy import BootstrapFewShot

# Create base program
program = dspy.ChainOfThought("question -> reasoning -> answer")

# Prepare training data
train_data = [
    dspy.Example(
        question="What is X?",
        reasoning="Step 1: ...",
        answer="Y"
    ).with_inputs("question")
]

# Optimize with few-shot examples
optimizer = BootstrapFewShot(
    metric=your_metric,
    max_demos=5
)

optimized = optimizer.optimize(program, train_data)

2. Prompt Templating

class OptimizedPrompt(dspy.Module):
    def __init__(self):
        self.prompt = dspy.Prompt(
            """Answer the following question with reasoning:
            
            Question: {question}
            
            Reasoning:
            {reasoning}
            
            Answer: {answer}""",
            examples=[
                {
                    "question": "Example question",
                    "reasoning": "Example reasoning",
                    "answer": "Example answer"
                }
            ]
        )
    
    def forward(self, question):
        return self.prompt(question=question)

3. Prompt Search

from dspy import PromptSearch

# Create prompt variants
prompts = [
    "Simple prompt format",
    "Detailed prompt format",
    "Structured prompt format"
]

# Search for best prompt
searcher = PromptSearch(
    prompts=prompts,
    metric=your_metric,
    train_data=train_data
)

best_prompt = searcher.search()

4. Automatic Prompt Optimization

from dspy import MIPRO

# Configure MIPRO optimizer
optimizer = MIPRO(
    metric=your_metric,
    num_trials=10,
    minibatch=[0.2, 0.4, 0.6],
    shard_size=1
)

# Optimize program
optimized = optimizer.optimize(
    program=base_program,
    trainset=train_data,
    valset=dev_data
)

⚠️ Pitfalls

• Overfitting: Optimized prompts may not generalize
• Complexity: Complex prompts increase latency
• Maintainability: Hard-to-read prompts are hard to maintain
• Bias: Prompts can introduce or amplify bias

📖 References

• DSPy Prompt Optimization
• Prompt Engineering Guide