# Optimization Embedding Tuning

> Embedding optimization techniques for DSPy

- Skill: `j33bs/optimization-embedding-tuning` (Agent Skill)
- Install (CLI): `npx skillmds@latest add j33bs/optimization-embedding-tuning`
- Raw SKILL.md: https://api.skillmd.com/api/skills/j33bs/optimization-embedding-tuning/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: j33bs (https://skillmd.com/u/j33bs)
- Updated: 2026-09-22
- Page: https://skillmd.com/skills/j33bs/optimization-embedding-tuning

---


# Embedding Optimization

## 🎯 Trigger Conditions
Use when asked about embedding optimization, vector search tuning, or improving retrieval quality in DSPy.

## 📚 Prerequisites
- `dspy` package installed
- Embedding model available
- Training data with relevance labels

## 🛠️ Embedding Optimization Techniques

### 1. Embedding Selection
```python
import dspy

# Different embedding models
embeddings = {
    "small": dspy.OllamaEmbedding(model="nomic-embed"),
    "medium": dspy.OpenAIEmbedding(model="text-embedding-3-small"),
    "large": dspy.OpenAIEmbedding(model="text-embedding-3-large")
}

# Choose based on needs
retriever = dspy.Retrieve(
    k=5,
    embedding_model=embeddings["medium"]
)
```

### 2. Embedding Fine-Tuning
```python
from sentence_transformers import SentenceTransformer, InputExample, losses

# Prepare training data
train_examples = [
    InputExample(texts=["query", "relevant passage"]),
    InputExample(texts=["query", "irrelevant passage"])
]

# Create model
model = SentenceTransformer("all-MiniLM-L6-v2")

# Define loss
train_loss = losses.CosineSimilarityLoss(model)

# Fine-tune
model.fit(
    train_objectives=[(train_dataloader, train_loss)],
    epochs=3,
    warmup_steps=100
)
```

### 3. Embedding Cache Optimization
```python
import dspy

# Configure embedding cache
dspy.settings.configure(
    rm=dspy.LMCache(
        lm=dspy.LM("openai/gpt-3.5-turbo"),
        cache_path="./embedding_cache"
    )
)

# Use cached embeddings
retriever = dspy.Retrieve(
    k=5,
    embedding_model=dspy.OllamaEmbedding(model="nomic-embed")
)
```

### 4. Hybrid Search Optimization
```python
class HybridSearch(dspy.Module):
    def __init__(self):
        self.vector_retriever = dspy.Retrieve(k=5)
        self.keyword_retriever = dspy.BM25Retriever(k=5)
        self.reranker = dspy.Reranker(top_n=3)
    
    def forward(self, query):
        # Hybrid retrieval
        vector_results = self.vector_retriever(query).passages
        keyword_results = self.keyword_retriever(query).passages
        
        # Combine results
        combined = self.combine_results(vector_results, keyword_results)
        
        # Rerank
        return self.reranker(query, combined)
    
    def combine_results(self, vector_results, keyword_results):
        # Reciprocal rank fusion
        scores = {}
        for i, passage in enumerate(vector_results):
            scores[passage] = 1 / (i + 1)
        for i, passage in enumerate(keyword_results):
            scores[passage] = scores.get(passage, 0) + 1 / (i + 1)
        
        return sorted(scores.keys(), key=lambda x: scores[x], reverse=True)
```

## ⚠️ Pitfalls
- **Dimensionality**: Higher dimensions increase compute
- **Bias**: Embeddings can encode dataset bias
- **Update frequency**: Embeddings need regular updates
- **Storage**: Large embedding collections require storage

## 📖 References
- [Embedding Optimization](https://dspy-docs.vercel.app/docs/deep-dive/retrieval/embeddings)
- [Sentence Transformers](https://www.sbert.net/)