# Vector Search Indexing

> Use when implementing vector search indexing algorithms.

- Skill: `loopyluci/vector-search-indexing` (Agent Skill)
- Install (CLI): `npx skillmds@latest add loopyluci/vector-search-indexing`
- Raw SKILL.md: https://api.skillmd.com/api/skills/loopyluci/vector-search-indexing/raw
- Safety review: pending (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- License: MIT
- Author: LoopyLuci (https://skillmd.com/u/loopyluci)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/loopyluci/vector-search-indexing

---


# Vector Search and Indexing

Implementing vector search indexing algorithms — from HNSW and IVF through product quantization, filtering, hybrid search, and distributed vector databases.

## When to Use

- Building semantic search with embedding vectors
- Implementing approximate nearest neighbor (ANN) search
- Scaling vector search to millions/billions of vectors
- Hybrid search combining vector + keyword (BM25)

## Indexing Algorithms

```python
INDEXING_ALGORITHMS = {
    'flat': 'Brute force — exact, O(n*d), good for <10K vectors',
    'ivf': 'Inverted File Index — k-means clustering, coarse quantizer, O(log n)',
    'hnsw': 'Hierarchical Navigable Small World — graph-based, best recall/speed tradeoff',
    'pq': 'Product Quantization — compresses vectors, reduces memory 4-8x',
    'ivf_pq': 'IVF + PQ — combination for billion-scale search',
}

class VectorIndexBuilder:
    """Build and query vector indexes."""
    def __init__(self, dimension: int, index_type: str = 'hnsw'):
        import faiss
        self.dim = dimension
        
        if index_type == 'flat': self.index = faiss.IndexFlatL2(dimension)
        elif index_type == 'hnsw':
            self.index = faiss.IndexHNSWFlat(dimension, 32)  # 32 neighbors
        elif index_type == 'ivf':
            self.index = faiss.IndexIVFFlat(faiss.IndexFlatL2(dimension), dimension, 100)
            self.index.train = lambda x: None
    
    def add(self, vectors): self.index.add(vectors)
    def search(self, query, k: int = 10): return self.index.search(query, k)
```

## Verification Checklist

- [ ] Index type matches dataset size (Flat <10K, IVF <1M, HNSW <100M, IVF+PQ >100M)
- [ ] Recall benchmarked against brute force (target: >95% recall@10)
- [ ] Index build time and memory usage measured
- [ ] Filtering support (metadata pre-filter or post-filter)
- [ ] Hybrid search: vector + keyword (BM25) fusion
- [ ] Distributed indexing (sharding) for billion-scale
- [ ] Index update strategy (incremental add, delete)