Vector Databases in Production RAG: Comparing Qdrant, Milvus, Weaviate, and pgvector Architecture, HNSW vs DiskANN, Filter Pipelines, Quantization, and Serving Economics

In production Retrieval-Augmented Generation (RAG) systems, vector retrieval is rarely a simple nearest-neighbor lookup across unconstrained float32 embeddings. Real-world enterprise workloads require strict multi-tenant metadata filtering, low p99 latency under concurrent load, high write throughput for continuous ingestion, and strict memory budgeting.

While synthetic benchmarks frequently test unconstrained recall on static datasets, production teams encounter severe operational divergence across storage engines. An index topology that performs efficiently on 100,000 documents can exhaust system memory or collapse query recall when scaled to tens of millions of records with strict filter constraints.

Choosing between purpose-built vector search engines like Qdrant, Milvus, and Weaviate, or extending relational infrastructure via pgvector and pgvectorscale, requires evaluating underlying graph traversal algorithms, quantization methods, filter execution pipelines, and serving economics.

Core Architectural Paradigms

Vector storage engines fall into four distinct architectural patterns, each trading operational simplicity against horizontal scaling efficiency.

Qdrant: Single-Process Rust Engine

Qdrant is implemented in Rust and focuses on single-node resource efficiency and predictable tail latency. Its core design relies on memory-mapped files (mmap) for vector payloads and graph structures, enabling datasets to exceed physical RAM limits by leveraging the OS page cache.

Qdrant avoids generic index wrappers by embedding metadata directly into its vector storage format. The search planner dynamically inspects query filters and selects between payload index scans and graph navigation based on calculated filter cardinality.

Milvus: Disaggregated Cloud-Native Microservices

Milvus separates compute, coordinator, and storage layers into distinct Kubernetes microservices. Its architecture comprises stateless Proxies, Coordinators (RootCoord, QueryCoord, DataCoord), worker nodes (QueryNode, DataNode, IndexNode), a Write-Ahead Log (WAL), and an object storage persistence tier (S3 or MinIO).

Milvus executes vector searches via Knowhere, an internal acceleration engine that wraps core C++ similarity search libraries including Faiss, HNSWLib, and Microsoft DiskANN. For massive deployments exceeding hundreds of millions of vectors, Milvus scales out compute nodes independently of storage nodes, supporting tiered storage policies that keep hot segments in memory, warm segments on NVMe SSDs, and cold segments in object storage.

Weaviate: Modular LSM Multi-Tenant Architecture

Weaviate is written in Go with a C++ and Rust vector execution core. It uses a custom Log-Structured Merge-tree (LSM) storage engine for object persistence, coupled with dynamic vector caching.

Weaviate treats multi-tenancy as a first-class architectural primitive, allowing collections to shard data per tenant with independent HNSW graphs. Inactive tenant shards can be dynamically offloaded to disk or object storage to free active RAM, and reloaded on demand when queried. Weaviate natively integrates sparse keyword search (BM25) and dense embeddings within a unified inverted index and graph framework.

pgvector and pgvectorscale: Relational Co-Location

The pgvector extension introduces native vector, halfvec (float16), sparsevec, and bit data types directly into PostgreSQL. This design provides full ACID guarantees, point-in-time recovery, and zero-ETL joins between relational tables and high-dimensional embeddings.

Standard pgvector implements IVFFlat and HNSW indexing within PostgreSQL's shared buffer architecture. To overcome PostgreSQL memory constraints on large datasets, the companion extension pgvectorscale introduces StreamingDiskANN, a disk-resident graph index adapted from Microsoft's DiskANN algorithm, along with Statistical Binary Quantization (SBQ).

Vector Indexing and Filtering Architecture

Indexing Topologies: HNSW vs DiskANN vs Inverted Lists

The choice of approximate nearest neighbor (ANN) index determines memory consumption, construction speed, and query latency.

Hierarchical Navigable Small World (HNSW)

HNSW builds a multi-layer graph where upper layers contain long-range skip connections and lower layers contain dense, short-range connections. Search begins at top layers to quickly navigate toward the query neighborhood, progressively descending to layer zero for fine-grained traversal.

Key HNSW configuration parameters include:

M: The maximum number of bidirectional links per node (typically 16 to 64). Higher values improve recall and connectivity at the cost of larger memory consumption and slower index build times.
ef_construction: The size of the dynamic candidate list during index construction (typically 64 to 512). Higher values yield better graph quality at the cost of quadratic construction time.
ef_search: The size of the dynamic candidate list during query execution. Increasing ef_search improves recall during retrieval while linearly increasing latency.

HNSW delivers sub-10ms query latencies with high recall (often exceeding 98%). However, storing uncompressed 1536-dimensional float32 vectors in HNSW requires 60 to 70 GB of RAM per 10 million vectors, making raw in-memory graph hosting expensive at scale.

DiskANN and StreamingDiskANN

DiskANN (developed by Microsoft Research) uses a single-layer Vamana graph combined with compressed vector representations. The full precision vectors and graph adjacency lists reside on NVMe SSDs, while compressed representations (typically quantized via Product Quantization or Binary Quantization) are cached in RAM.

During query execution, the search algorithm uses in-memory quantized vectors to navigate the graph and identify candidate nodes, streaming full precision vectors from disk in parallel asynchronous I/O batches for final distance rescoring.

In pgvectorscale, StreamingDiskANN brings this architecture to PostgreSQL, enabling systems to serve tens of millions of vectors from NVMe SSDs with 95% to 99% recall while reducing RAM requirements by 70% to 80% compared to pure in-memory HNSW.

Inverted File Index (IVF)

IVF partitions vector space into Voronoi cells using k-means clustering. At query time, the system compares the query vector against centroid vectors, searching only the vectors assigned to the closest nprobe centroids.

While IVF builds indexes significantly faster and consumes less memory than HNSW, it suffers from lower recall when dataset density varies or when strict metadata filters exclude candidates within the probed centroids.

Quantization Mechanics and Memory Footprint

Quantization compresses high-dimensional vectors to reduce memory footprint and accelerate distance computations through hardware SIMD instructions.

Scalar Quantization (SQ / int8)

Scalar Quantization maps continuous 32-bit floating-point numbers into 8-bit integers (int8) by computing minimum and maximum values across each vector dimension:

val_int8 = round((val_fp32 - min_dim) / (max_dim - min_dim) * 255)

Scalar quantization reduces vector memory footprint by 4x (e.g., from 6.14 KB to 1.54 KB per 1536-dimensional vector) with negligible recall degradation (typically under 1% drop). Distance calculations benefit from AVX-512 and ARM NEON integer dot-product instructions.

Binary Quantization (BQ) and Statistical Binary Quantization (SBQ)

Binary Quantization compresses each dimension into a single bit (1 if the value is positive, 0 if negative or below threshold). This achieves a 32x memory reduction and transforms distance computations into single-cycle bitwise XOR and POPCNT (population count) CPU instructions:

Distance = POPCNT(vector_a XOR vector_b)

Standard BQ requires embeddings to be symmetrically distributed around zero (such as OpenAI text-embedding-3 or Cohere v3 models). For models where dimensions exhibit non-zero means, standard BQ loses critical variance.

Timescale's Statistical Binary Quantization (SBQ) addresses this by calculating per-dimension mean and variance statistics across the dataset, establishing dynamic thresholds per dimension. In production retrieval, systems perform initial coarse candidate gathering using 1-bit Hamming distance, followed by an oversampled rescore step against uncompressed or 8-bit cached vectors.

Product Quantization (PQ)

Product Quantization divides a high-dimensional vector of dimension D into M sub-vectors of dimension D/M. For each subspace, k-means clustering identifies K centroids (typically 256, fitting into an 8-bit index). A 1536-dimensional vector split into 192 sub-vectors of 8 dimensions is represented as 192 bytes.

PQ enables high compression ratios (up to 32x to 64x) but introduces higher quantization distortion than SQ, requiring dedicated calibration training on representative dataset samples.

Metadata Filtering Execution: Pre, Post, and Single-Stage Graph Traversal

Production RAG queries almost always include filter conditions (e.g., tenant_id = 'acme', status = 'published', created_at >= '2026-01-01'). The execution order between vector similarity search and metadata filtering determines whether queries succeed or fail.

The Pitfalls of Pre-Filtering and Post-Filtering

Post-Filtering: The engine executes standard top-k ANN search across the entire graph, then discards vectors that do not match the metadata filter. If the filter matches only 1% of the dataset (high selectivity), top-k search may return zero valid results, causing catastrophic recall collapse.
Pre-Filtering: The engine first evaluates metadata filters to produce an ID match list, then runs a brute-force exact k-NN scan over the matching subset. While this guarantees 100% recall, latency degrades linearly with the number of matching documents, becoming unusable when filter matches exceed 50,000 items.

Single-Stage Filtered Graph Traversal

Modern vector engines implement single-stage filtered graph traversal to eliminate the tradeoffs of pre- and post-filtering:

Qdrant (Filterable HNSW and ACORN): Qdrant constructs payload indexes (inverted lists, numeric ranges, geo-indexes) alongside the vector graph. During HNSW graph generation, if payload indexes exist, Qdrant adds extra navigation links connecting points that share common payload attributes. During search, the query planner evaluates filter cardinality: for high-selectivity filters, it uses payload inverted lists; for medium-selectivity filters, it traverses the HNSW graph while restricting candidate exploration to valid payload nodes via bitset masks.
Weaviate (Inverted Index Bitset Masking): Weaviate resolves metadata filter expressions into a Roaring Bitset using its LSM inverted index prior to graph traversal. As the HNSW search navigates neighbor connections, it checks candidate IDs against the bitset in memory, evaluating distance only for matching objects without fragmenting the traversal path.
Milvus (Partition Keys and Iterative Filtering): Milvus allows defining Partition Keys to route entities physically into isolated data segments. For scalar expressions across segments, Milvus executes iterative graph exploration, dynamically expanding the search radius when filtered nodes are encountered.

Hybrid Search and Rank Fusion

Dense semantic search often struggles with exact keyword lookups, part numbers, SKU codes, and rare technical acronyms. Hybrid search bridges this gap by combining dense vector retrieval with sparse lexical retrieval (BM25 or SPLADE).

Fusion Algorithms: RRF vs Relative Score Fusion

To combine candidate rankings from dense and sparse retrievers, engines use two primary fusion strategies:

Reciprocal Rank Fusion (RRF)

RRF operates solely on ordinal rank positions rather than raw similarity scores, ensuring robustness against differing score distributions:

RRF_Score(d) = Sum( 1 / (k + rank_i(d)) )

Where k is a constant (typically 60) that prevents top-ranked candidates from dominating the aggregated score.

Relative Score Fusion (RSF)

Starting in Weaviate v1.24, Relative Score Fusion normalizes raw scores from each retriever into a [0, 1] range before computing a weighted sum using an alpha parameter:

Hybrid_Score(d) = (alpha * Normalized_Dense(d)) + ((1 - alpha) * Normalized_BM25(d))

alpha = 1.0: Pure dense vector search.
alpha = 0.0: Pure BM25 keyword search.
alpha = 0.75: Production standard setting, favoring semantic context while retaining exact term matches.

Qdrant supports server-side RRF and dense-sparse fusion directly via its unified Query API, enabling multi-vector late interaction models (such as ColBERT) alongside standard dense and sparse representations.

Serving Economics and Production Selection Matrix

When selecting a vector database for production, infrastructure decisions hinge on dataset scale, operational complexity, and latency requirements.

Decision Matrix by Workload

Under 2 Million Vectors with Existing PostgreSQL: Standard pgvector with HNSW and halfvec (float16) is the simplest operational choice. It eliminates data duplication, avoids maintaining an external sync pipeline, and leverages existing backup, replication, and security infrastructure.
5 Million to 50 Million Vectors in PostgreSQL: pgvectorscale with StreamingDiskANN and Statistical Binary Quantization allows scaling on a single PostgreSQL instance without incurring high RAM costs, maintaining p50 latencies under 15ms.
10 Million to 100 Million Vectors (High QPS, Complex Filtering): Qdrant on dedicated NVMe instances provides superior single-node query throughput, predictable tail latency, and advanced payload-aware graph filtering with scalar or binary quantization.
100 Million to Billions of Vectors (Enterprise Scale): Milvus with disaggregated compute/storage and tiered storage policies allows horizontal scaling of QueryNodes across Kubernetes, offloading historical data to object storage while maintaining low-latency active segments.
Multi-Tenant SaaS with Dynamic Tenant Lifecycle: Weaviate provides native tenant isolation with dynamic offloading of inactive tenant shards, combined with built-in hybrid search and vectorizer modules.

Evaluating vector infrastructure requires benchmarking with realistic metadata filters, representative payload distributions, and quantization configurations matching production query volumes.