Milvus

Milvus 2.4+

Architecture at a Glance

Milvus 2.x is cloud-native by design:

Clients (pymilvus, REST, gRPC)
     |
  Proxy nodes   ---- etcd (metadata, service discovery)
     |
  Coord nodes   ---- Pulsar / Kafka (message stream for writes)
     |
  Query / Data / Index nodes
     |
  MinIO / S3 (persistent object storage)

Key consequences:

• Horizontal scale by adding query/data/index nodes.
• Writes go to a message stream, not directly to storage — eventually consistent.
• State lives in etcd + object storage; nodes are stateless.

For small deployments, milvus-standalone packs everything into one container. Don't run standalone in production > a few million vectors.

Zilliz Cloud (Managed Milvus)

Use Zilliz Cloud if you don't want to run the dependencies yourself. Tiers:

• Serverless — pay per read/write; cold starts on first query.
• Dedicated — pinned compute, predictable latency.
• Cloud Enterprise — multi-region, private networking, BYOK.

The pymilvus client works identically against OSS Milvus and Zilliz Cloud.

pymilvus v2 Connection

# pip install pymilvus==2.4.*
from pymilvus import MilvusClient, DataType, Function, FunctionType

client = MilvusClient(
    uri="https://in03-xxxx.api.gcp-us-west1.zillizcloud.com",
    token=os.environ["ZILLIZ_TOKEN"],  # or user:password for OSS
)

For OSS local: uri="http://localhost:19530", no token.

Collection with Dynamic Schema

Dynamic schema lets you push arbitrary JSON fields without pre-declaring them — like MongoDB but for vectors.

schema = client.create_schema(auto_id=False, enable_dynamic_field=True)
schema.add_field("doc_id", DataType.VARCHAR, is_primary=True, max_length=64)
schema.add_field("vector", DataType.FLOAT_VECTOR, dim=1024)
schema.add_field("tenant_id", DataType.VARCHAR, max_length=32)
schema.add_field("created_at", DataType.INT64)
# dynamic fields (not declared) are stored in the `$meta` JSON blob

index_params = client.prepare_index_params()
index_params.add_index(field_name="vector", index_type="HNSW",
                       metric_type="COSINE", params={"M": 16, "efConstruction": 200})
index_params.add_index(field_name="tenant_id", index_type="INVERTED")

client.create_collection(
    collection_name="docs",
    schema=schema,
    index_params=index_params,
    consistency_level="Bounded",   # Strong | Bounded | Eventually | Session
)

Partitions for Scale

Partitions are physical subdivisions within a collection, loaded/unloaded independently. Use them for:

• Hot/cold data (recent vs archive)
• Multi-tenancy at huge tenant counts
• Per-topic isolation

client.create_partition(collection_name="docs", partition_name="2025-Q2")
client.insert(collection_name="docs", partition_name="2025-Q2",
              data=[{"doc_id": "d1", "vector": v, "tenant_id": "acme", "created_at": ts}])

client.search(
    collection_name="docs",
    partition_names=["2025-Q2"],
    data=[q_vec],
    limit=10,
)

partition_key (declared on the schema) auto-routes rows to partitions based on a field — cleanest for multi-tenant workloads.

schema.add_field("tenant_id", DataType.VARCHAR, max_length=32, is_partition_key=True)

Index Types

Defaults: HNSW for general use; DiskANN when vector count * bytes exceeds available RAM.

index_params.add_index(
    field_name="vector",
    index_type="HNSW",
    metric_type="COSINE",
    params={"M": 16, "efConstruction": 200},
)

# DiskANN
index_params.add_index(
    field_name="vector",
    index_type="DISKANN",
    metric_type="COSINE",
    params={"search_list": 100},
)

Scalar Field Indexing

Milvus 2.4+ supports rich scalar indexes, not just the primary key:

index_params.add_index("tenant_id", index_type="INVERTED")      # exact match
index_params.add_index("created_at", index_type="STL_SORT")     # range queries
index_params.add_index("tags", index_type="INVERTED")           # array field

Without scalar indexes, filter expressions force a full scan before ANN.

Hybrid Search (BM25 + Vector) via Multi-Vector

Milvus 2.5+ introduces full-text (BM25) search as a first-class capability via function fields.

schema = client.create_schema(auto_id=True, enable_dynamic_field=False)
schema.add_field("id", DataType.INT64, is_primary=True)
schema.add_field("text", DataType.VARCHAR, max_length=8192, enable_analyzer=True)
schema.add_field("dense", DataType.FLOAT_VECTOR, dim=1024)
schema.add_field("sparse", DataType.SPARSE_FLOAT_VECTOR)

schema.add_function(Function(
    name="bm25_fn",
    input_field_names=["text"],
    output_field_names=["sparse"],
    function_type=FunctionType.BM25,
))

ix = client.prepare_index_params()
ix.add_index("dense", index_type="HNSW", metric_type="COSINE",
             params={"M": 16, "efConstruction": 200})
ix.add_index("sparse", index_type="SPARSE_INVERTED_INDEX", metric_type="BM25")

client.create_collection("hybrid", schema=schema, index_params=ix)

# Hybrid query
from pymilvus import AnnSearchRequest, RRFRanker
req_dense = AnnSearchRequest(data=[q_vec], anns_field="dense",
                             param={"metric_type": "COSINE"}, limit=50)
req_sparse = AnnSearchRequest(data=["oauth token refresh"], anns_field="sparse",
                              param={"metric_type": "BM25"}, limit=50)
results = client.hybrid_search(
    collection_name="hybrid",
    reqs=[req_dense, req_sparse],
    ranker=RRFRanker(60),
    limit=10,
)

Upserts

Milvus 2.3+ supports true upserts (delete-then-insert atomic):

client.upsert(
    collection_name="docs",
    data=[{"doc_id": "d1", "vector": new_vec, "tenant_id": "acme", "created_at": ts}],
)

Without upsert, you delete by primary key and insert. The stream-based write model means deletes are logical until compaction — do not rely on immediate disappearance.

Time Travel (point-in-time queries)

# Query as of a timestamp (micro-second precision)
client.search(
    collection_name="docs",
    data=[q_vec],
    limit=10,
    travel_timestamp=1712000000000,
)

Useful for reproducible eval runs and debugging. Retention is bounded by dataCoord.gc.dropTolerance (default 3 hours); extend for longer time-travel windows.

Consistency Levels

Default: Bounded. Use Strong only when your RAG pipeline must reflect a just-ingested document immediately.

Load / Release for Memory Management

Collections must be loaded into memory to be searchable. Release when not in use to free RAM.

client.load_collection("docs")
# search, insert, etc.
client.release_collection("docs")

For partitioned collections, load only the hot partitions.

Sizing Estimation

RAM per vector (approximate):

• FLOAT_VECTOR 1024-d: 4 KB
• HNSW overhead: +20-40%
• IVF_FLAT overhead: +5-10% + centroids
• DiskANN: mostly on disk, ~0.1 KB/vector in RAM

For 100M 1024-d vectors with HNSW: ~500 GB RAM in pure vector terms — consider DiskANN, IVF_PQ, or GPU indexes.

Anti-Patterns

Production Checklist

• Distributed deployment (or Zilliz) — not standalone
• Scalar indexes on all filtered fields
• Consistency level chosen per use case
• Partition key on tenant/time field if multi-tenant
• Index type matches RAM budget (HNSW / DiskANN / IVF_PQ / GPU)
• Load / release strategy for cold partitions
• Batch insert 1000-10000 rows per request
• Compaction schedule reviewed (deletes linger otherwise)
• etcd and MinIO backed up
• Monitoring: query node CPU, data node storage, compaction lag
• Upgrade path tested (2.3 -> 2.4 -> 2.5)