Vector Index Operations

Index types and trade-offs

Algorithm	Recall	Build cost	Query latency	Updates
HNSW	High	Slow build	Fast	Incremental OK
IVF	Tunable	Faster build	Fast with tuning	Needs retrain clusters
Flat (brute)	Exact	None	Slow at scale	Easy

Most production vector DBs default to HNSW — know the knobs: M, efConstruction, efSearch (higher = better recall, more RAM/latency).

Hybrid search at scale

Pure vector search misses exact keyword matches (SKUs, error codes, function names). Hybrid combines:

• Dense retrieval (embeddings)
• Sparse retrieval (BM25/inverted index)

Merge via RRF (reciprocal rank fusion) or learned reranker. Weaviate, Elasticsearch, Pinecone support hybrid — use it for code and support KB.

Operational concerns

Index rebuild

Plan blue/green: new index → shadow traffic comparison → swap alias → delete old.

Recall regression

Automated eval: fixed query set, measure recall@10 and MRR after any index change.

Memory and sharding

Billion vectors → shard by tenant or hash, replicate for read QPS. Monitor p99 query latency per shard — hot tenants need isolation.

Metadata filters

Pre-filter by team_id, doc_type before vector search — reduces search space and enforces ACLs.

Query path latency budget

Embed query (20ms) → ANN search (30ms) → Fetch chunks (20ms) → Rerank (50ms) → Pack context (10ms)

Reranking top-100 with a cross-encoder is quality-positive, latency-negative — cap candidates.

Embedding drift

Corpus embeddings age when:

• Embedding model updated
• Domain shift (new product vocabulary)
• Chunking policy changed

Detect via: retrieval hit rate drop, downstream eval score drop, user "wrong source" feedback.

Disaster recovery

• Rebuild index from object store + embedding cache
• RPO/RTO defined — vector index is derived data, source of truth is raw docs + embed pipeline state

Interview framing

"Vector DB went slow after 10× growth":

1. Check shard balance and hot partitions
2. Review HNSW params vs recall requirements
3. Add metadata pre-filters
4. Consider separate indexes per high-QPS tenant

Senior signal: Say you'd maintain a recall eval harness tied to deploy gates — same discipline as load tests for traditional services.

Link forward

Reliability & Safety addresses what happens when retrieval returns nothing or stale chunks — degradation must be designed, not accidental.