Vector Databases and Semantic Search Engines in Production: Comparing Qdrant, Milvus, Weaviate, and pgvector Architecture, Filtered Index Traversal, Quantization Memory Footprints, and Serving Economics

In production retrieval-augmented generation (RAG) and agentic workflows, semantic search is rarely a simple vector similarity calculation. Production vector search requires sub-10ms query latency under heavy concurrency, high-throughput ingestion, zero downtime during index rebuilds, precise metadata filtering, and strict memory budgeting.

Teams deploying vector search face a fundamental infrastructure choice between dedicated vector databases (such as Qdrant, Milvus, and Weaviate) and relational extensions like pgvector. While relational extensions eliminate data synchronization pipelines by keeping vectors alongside transactional records, purpose-built engines implement specialized memory layouts, SIMD-accelerated distance kernels, and filter-aware graph navigation algorithms.

This analysis examines the architectural foundations, Approximate Nearest Neighbor (ANN) indexing mechanics, filtered search execution paths, memory quantization profiles, and operational economics of Qdrant, Milvus, Weaviate, and pgvector.

1. Architectural Paradigms: Monolithic, Cloud-Native, and Relational

The four engines reflect distinct engineering philosophies regarding storage layouts, process isolation, and state management.

Qdrant: Rust-Native Segment Architecture

Qdrant is implemented in Rust and structured around an actor-based concurrency model. Data in Qdrant is organized into collections, which are partitioned into independent segments. Each segment contains its own write-ahead log (WAL), vector storage, payload (metadata) store, and index structures.

Segments operate under copy-on-write semantics. As vectors and payloads arrive, they are written to append-only mutable segments. Background optimizer threads periodically merge small segments, purge deleted records, and build immutable HNSW graphs. Vectors and payloads can be stored in RAM, memory-mapped from disk via mmap, or stored directly in on-disk RocksDB and custom appendable files. Clustering uses the Raft consensus protocol for metadata coordination and hash-ring sharding.

Milvus: Disaggregated Cloud-Native Architecture

Milvus adopts a fully disaggregated, microservice-based architecture designed for distributed enterprise scale. Its core execution layer is separated into stateless compute nodes and shared storage:

• Coordinator Nodes: RootCoord, QueryCoord, DataCoord, and IndexCoord manage cluster topology, collection schemas, and task scheduling using etcd.
• Worker Nodes: QueryNodes handle real-time search queries; DataNodes consume ingestion logs and persist segment data; IndexNodes construct ANN indices asynchronously.
• Log Broker & Object Storage: Ingestion streams through message brokers (Apache Kafka or Apache Pulsar), while persistent segment data and indices reside in object storage (MinIO or Amazon S3).

Vector computation inside Milvus is executed by Knowhere, an independent C++ vector engine that encapsulates algorithms from FAISS, HNSWLib, DiskANN, and ScaNN with custom SIMD kernels.

Weaviate: Hybrid Modular Schema Engine

Weaviate is written in Go with performance-critical vector operations implemented in C/C++. Data is structured around typed classes with explicit property schemas. Storage is handled via a Log-Structured Merge (LSM) tree architecture that manages object payloads, inverted indices (for BM25 text search and scalar filtering), and custom HNSW vector graphs.

Weaviate provides native multi-tenant sharding. Each tenant within a collection maintains an isolated LSM store and vector index, allowing idle tenant shards to be dynamically offloaded to cold object storage and reloaded into memory on demand.

pgvector: Relational Extension

pgvector is an open-source C extension for PostgreSQL that integrates vector data types, distance operators (<-> for Euclidean, <#> for negative inner product, and <=> for cosine distance), and custom access methods directly into the PostgreSQL storage engine.

Vectors are stored as standard column values inside PostgreSQL heap pages (typically 8KB pages). Vector queries leverage PostgreSQL query planning, buffer pool management (shared_buffers), write-ahead logging, and Multi-Version Concurrency Control (MVCC). Because it runs inside the Postgres process, vector operations compete directly with standard transactional queries for CPU cycles, shared memory buffers, and disk I/O.

2. Indexing Topologies and Quantization Mechanics

Exact k-Nearest Neighbors (k-NN) brute-force scanning requires $O(N \cdot d)$ floating-point distance calculations per query, which becomes computationally prohibitive beyond 100,000 vectors. Production vector engines rely on Approximate Nearest Neighbor (ANN) index structures.

Hierarchical Navigable Small World (HNSW)

HNSW is the dominant graph-based ANN algorithm across all four engines. It constructs a multi-layer graph where lower layers contain dense connections among all nodes, and upper layers contain sparse, long-range skip links.

Search begins at the top layer with greedy routing, locating the local entry point, and steps down layer by layer. Performance and recall are governed by three primary hyperparameters:

• $M$: The maximum number of bidirectional connections per node (typically 16 to 64). Higher $M$ values improve recall on complex high-dimensional manifolds at the cost of index build time and memory usage ($O(M \cdot d)$ per node).
• $efConstruction$: The beam search size used during index generation. Higher values construct higher-quality graph connectivity.
• $efSearch$: The size of the dynamic candidate priority queue evaluated during runtime queries. Increasing $efSearch$ trades query latency for higher recall without rebuilding the index.

Inverted File (IVFFlat)

IVFFlat partitions the vector space into $k$ Voronoi cells using k-means clustering. During query execution, the search vector calculates distances to cluster centroids and scans only the vectors contained within the top $n_{probe}$ nearest cells. IVFFlat provides rapid index construction and lower RAM usage than HNSW, but suffers steeper recall degradation and linear latency increases as $n_{probe}$ expands.

Quantization and Memory Compression

Uncompressed float32 embeddings require 4 bytes per dimension (6,144 bytes per 1,536-dimensional vector). A collection of 10 million vectors requires over 61 GB of RAM just for raw vector vectors, excluding graph connectivity overhead. Production engines implement several quantization techniques to reduce this footprint:

1. Scalar Quantization (SQ8 / SQ4): Maps continuous 32-bit floats into uniform 8-bit or 4-bit integers based on dimension-wise minimum and maximum bounds. SQ8 reduces vector memory consumption by 75% (1.536 KB per 1,536-dim vector) while preserving over 98% to 99% of original recall. Qdrant and Milvus support full SIMD integer arithmetic for SQ-compressed vectors.
2. Product Quantization (PQ): Decomposes a $d$-dimensional vector into $m$ low-dimensional subspaces, assigns each subspace to a centroid index from a trained codebook, and represents the vector as an array of byte-indices. A 1,536-dim vector can be compressed to 64 or 128 bytes (a 95%+ memory reduction). Query execution uses precomputed Asymmetric Distance Computation (ADC) lookup tables.
3. Binary Quantization (BQ / 1-Bit): Thresholds each floating-point dimension to a single bit (0 or 1). Vector distance reduces to Hamming distance computed via hardware XOR and POPCNT instructions. BQ yields a 32x memory reduction and orders-of-magnitude faster distance scans. It is particularly effective for high-dimensional models trained with spherical cosine loss (such as Cohere embed-v3 or OpenAI text-embedding-3-large).
4. Half-Precision (halfvec) in pgvector: pgvector supports 16-bit float storage (halfvec), cutting index and table storage footprints by 50% without meaningful loss of recall or embedding fidelity.

3. The Production Bottleneck: Filtered Vector Search Architecture

Real-world AI applications rarely query the entire database; they search within specific user IDs, tenant scopes, document types, or date ranges. How an engine combines scalar filtering with vector graph navigation is the single largest determinant of production latency.

According to research on Filtered Approximate Nearest Neighbor Search, filtering methods fall into three distinct architectural categories:

Naive Post-Filtering

The vector engine executes an unconstrained HNSW graph search to locate the top $K \cdot multiplier$ nearest neighbors, then discards candidates that do not match the scalar filter. When filter selectivity is high (e.g., 99% of documents belong to other users), the top candidates returned by the graph search are almost entirely filtered out, causing recall to drop toward zero.

Brute-Force Pre-Filtering

The database executes the scalar filter first to generate a list of matching internal IDs, then performs an exact brute-force scan across only the matching vectors. This approach works efficiently when filter selectivity is tiny (e.g., matching fewer than 1,000 vectors), but degrades when the filter matches hundreds of thousands of candidates.

Single-Pass Filter-Aware Graph Traversal

Modern dedicated vector engines integrate metadata masks directly into the HNSW graph traversal step:

• Qdrant (Filterable HNSW and Adaptive Planner): Qdrant constructs payload indexes (hash maps for keywords, B-Trees for numeric ranges, inverted indexes for text). During indexing, Qdrant builds additional graph edges to ensure subgraphs matching specific payload values remain navigable. During execution, Qdrant's query planner estimates filter selectivity: if selectivity is tight, it iterates payload IDs; if selectivity is moderate to wide, it executes single-pass HNSW traversal checking payload bitsets on every visited node without losing graph connectivity.
• Milvus / Knowhere (Dual-Pool Graph Navigation): Standard graph traversal algorithms can have their candidate queues saturated by invalid nodes when filters eliminate 90%+ of the dataset. Milvus's Knowhere engine addresses this with a dual-pool priority queue (NeighborSetDoublePopList). Filtered-out nodes continue to serve as structural navigation waypoints through the graph, while only valid nodes matching the filter bitset are eligible to enter the top-k result pool.
• Weaviate (Inverted Index Bitmaps): Weaviate resolves scalar and text filters against its LSM-based inverted index, producing a compressed Roaring Bitmap of valid document IDs. The HNSW traversal kernel checks this bitmap during node evaluation, skipping non-matching candidates while traversing neighboring graph edges.
• pgvector (Postgres Planner Dynamics): pgvector relies on PostgreSQL's query optimizer. The planner chooses between an Index Scan (using iterative HNSW scanning that inspects extra candidate neighbors when filtered out), a Bitmap Index Scan combined with table heap filters, or a Sequential Scan. Under restrictive filters, pgvector may fall back to sequential heap scans unless query hints or composite index structures are explicitly configured.

4. Performance Benchmarks and Resource Footprint Analysis

Standardized evaluations across 1 million 1,536-dimensional vectors running on equivalent compute instances (8 vCPU, 32GB RAM, NVMe storage) demonstrate clear performance and resource trade-offs.

Query Latency Under Load (p50 / p95 / p99)

• Qdrant: p50: 4ms | p95: 8ms | p99: 25ms. Rust-native memory management and SIMD optimizations deliver consistent sub-10ms response times at high query concurrency (800+ QPS per single node).
• Milvus: p50: 6ms | p95: 12ms | p99: 35ms. Disaggregated QueryNodes process batch queries with high parallel throughput (1,200+ QPS), with optional GPU-accelerated indexing via Knowhere.
• Weaviate: p50: 12ms | p95: 22ms | p99: 50ms. Efficient hybrid search execution (combining BM25 inverted text scoring and vector cosine similarity via Reciprocal Rank Fusion) with balanced single-node latency.
• pgvector: p50: 35ms | p95: 65ms | p99: 140ms. Query latency is dependent on shared_buffers cache sizing. If the HNSW index exceeds available PostgreSQL buffer cache, disk page thrashing increases p95 latency noticeably.

Memory Footprint per 1 Million Vectors (1,536 Dimensions)

• pgvector (Uncompressed float32 HNSW, $M=16$): ~4.8 GB to 6.2 GB (includes heap table storage, HNSW graph edges, and WAL).
• pgvector (halfvec fp16 HNSW): ~2.8 GB to 3.4 GB.
• Qdrant (Raw float32 HNSW): ~6.5 GB.
• Qdrant (Scalar Quantization SQ8 + on-disk vectors): ~1.4 GB to 1.8 GB RAM.
• Weaviate (Product Quantization PQ): ~1.6 GB to 2.1 GB RAM.
• Milvus (Knowhere SQ8 / DiskANN): ~2.0 GB to 2.5 GB RAM.

5. Architectural Comparison and Selection Guide

Selecting the appropriate vector search infrastructure depends on dataset scale, operational complexity tolerance, update frequency, and multi-tenancy requirements.

Comparison Summary

• Qdrant
• Language / Core: Rust
• Architecture: Single binary or distributed Raft cluster; segment-based storage
• Primary Index Types: HNSW, Flat
• Quantization: SQ8, SQ4, PQ, Binary Quantization (BQ)
• Filtering Strategy: Adaptive single-pass filterable HNSW + payload indices
• Best For: Standalone production RAG, high-concurrency microservices, sub-10ms latency budgets (1M to 50M vectors).
• Milvus
• Language / Core: Go coordinator + C++ Knowhere engine
• Architecture: Disaggregated distributed microservices (Kafka/Pulsar + etcd + S3/MinIO)
• Primary Index Types: HNSW, IVFFlat, DiskANN, ScaNN, GPU indices
• Quantization: SQ8, PQ, BQ, FP16
• Filtering Strategy: Dual-pool graph navigation (NeighborSetDoublePopList) + bitset masking
• Best For: Enterprise datasets (50M to 1B+ vectors), distributed data lake integration, high write ingestion pipelines.
• Weaviate
• Language / Core: Go + C/C++ modules
• Architecture: Modular class-based LSM tree engine; multi-tenant sharding
• Primary Index Types: HNSW, Flat, Dynamic
• Quantization: PQ, SQ, BQ
• Filtering Strategy: Inverted index Roaring Bitmaps integrated into graph traversal
• Best For: Native hybrid search (BM25 + Dense RRF), multi-tenant SaaS applications with dynamic tenant offloading.
• pgvector
• Language / Core: C (PostgreSQL Extension)
• Architecture: Relational storage inside PostgreSQL; shared buffer cache
• Primary Index Types: HNSW, IVFFlat
• Quantization: halfvec (fp16), sparsevec, bit indexing
• Filtering Strategy: PostgreSQL cost-based optimizer + iterative HNSW scanning
• Best For: Existing PostgreSQL environments, transactional ACID requirements, vector collections under 5M records without extreme QPS requirements.

6. Engineering Recommendation

1. Start with pgvector if PostgreSQL is already your core database and your total vector count is under 3 to 5 million records. It eliminates ETL synchronization pipelines, preserves transactional integrity, and handles standard RAG query latencies (40ms to 70ms) without adding new infrastructure.
2. Deploy Qdrant when query latency, high QPS, and complex payload filtering are primary requirements. Its single-binary Rust deployment simplifies operational maintenance, while its adaptive query planner and memory quantization options allow multi-million vector datasets to run on low-cost compute instances.
3. Choose Weaviate if your application requires built-in hybrid search (BM25 + vector) and multi-tenant isolation. Its automated tenant offloading to cloud object storage is well-suited for B2B SaaS architectures where individual customer collections are isolated.
4. Scale to Milvus when operating at 100M+ vector volumes that require horizontal compute/storage disaggregation, distributed streaming ingestion via Kafka, or GPU-accelerated indexing.

Sources

• Qdrant Architectural Overview and Storage Engine
• Qdrant Vector Search Filtering and Adaptive Indexing
• Milvus Architecture Overview and Disaggregated Topology
• Knowhere Vector Search Engine Core Repository
• Weaviate Documentation and Architecture
• pgvector PostgreSQL Extension Repository
• Filtered Approximate Nearest Neighbor Search in Vector Databases: System Design and Performance Analysis (arXiv:2602.11443)
• FAISS: Efficient Similarity Search and Clustering of Dense Vectors