Vector Databases in Production: Architecture, Filtering Strategies, and Scale Ceilings for pgvector, Qdrant, Milvus, and Pinecone

The rapid deployment of retrieval-augmented generation (RAG) and semantic search has turned vector databases from specialized academic tooling into core production infrastructure. However, engineering teams face conflicting architectural paradigms. On one side, the relational database ecosystem argues that vector extensions inside existing databases eliminate operational overhead. On the other side, dedicated vector database vendors argue that relational engines cannot handle high-dimensional geometry, high-throughput filtering, or multi-million-vector scale.

Selecting the right vector storage layer requires moving past marketing claims and examining the concrete architectural trade-offs: index memory footprints, graph traversal mechanics, metadata filtering implementations, and operational complexity.

The Core Problem: Memory Footprint and Index Geometry

Exact nearest neighbor search requires an exhaustive cosine or Euclidean distance calculation against every vector in a dataset (O(N) complexity). For production latency constraints (under 20 milliseconds), systems rely on Approximate Nearest Neighbor (ANN) indexes.

The dominant graph-based indexing algorithm is Hierarchical Navigable Small World (HNSW). HNSW constructs multi-layer proximity graphs where top layers provide long-distance routing and lower layers refine local clustering.

The primary bottleneck of standard HNSW is memory consumption:

• Raw Vector Storage: 1 million 1536-dimensional float32 vectors require approximately 6.14 GB of raw memory (1,000,000 * 1536 * 4 bytes).
• Graph Overhead: Storing adjacency lists for HNSW connections (typically with configuration parameters M = 16 to M = 64) adds 1.5 GB to 3.5 GB of additional pointer overhead per million vectors.
• Working Set Constraints: In traditional in-memory HNSW implementations, both the graph structure and vector vectors must remain resident in RAM to avoid random disk I/O thrashing during graph descent.

When datasets reach tens or hundreds of millions of vectors, pure RAM residency becomes cost-prohibitive, forcing architectures to diverge across quantization, disk-backed streaming, and decoupled distributed systems.

1. PostgreSQL with pgvector and pgvectorscale

For the vast majority of applications operating below 10 million to 50 million vectors, PostgreSQL with the pgvector extension is the standard baseline.

Architecture and Mechanics

• ACID Transactions and Colocation: Vectors live in the same relational rows as application data. Joins between relational tables (such as users, tenants, and permissions) and vector columns happen in a single query execution plan without dual-write race conditions or distributed sync pipelines.
• Standard pgvector Indexes: Supports HNSW and IVFFlat (Inverted File Flat) index structures. Version 0.7+ introduced 16-bit half-precision (halfvec) and binary vector data types (bit) to compress vector sizes by 50% and 96.8% respectively.
• StreamingDiskANN via pgvectorscale: Developed by Timescale, the pgvectorscale extension introduces an implementation of Microsoft Research DiskANN architecture. Rather than requiring full RAM residency, StreamingDiskANN stores the index graph on NVMe SSD storage and streams node neighborhoods into memory asynchronously.
• Statistical Binary Quantization (SBQ): Compresses floating-point embeddings into binary representations while retaining statistical variance thresholds, significantly accelerating vector distance computations before reranking candidates.

Scale Ceilings and Bottlenecks

• RAM Thrashing in Vanilla HNSW: Standard pgvector HNSW degrades sharply when index size exceeds PostgreSQL shared_buffers or available host RAM, resulting in unpredictable p99 query latency spikes.
• VACUUM and Update Bloat: Frequent updates to vector embeddings or heavy row churn can trigger table bloat and lock contention during index rebuilding.
• Sweet Spot: Up to 50 million vectors when paired with pgvectorscale, or under 10 million vectors on standard PostgreSQL.

2. Qdrant: Memory-Optimized Rust Engine

Qdrant is an open-source, Rust-native vector database designed around single-binary simplicity and advanced metadata filtering.

Architecture and Mechanics

• Filterable HNSW and ACORN: Traditional vector systems handle metadata filters either via pre-filtering (scanning matching IDs brute-force, which collapses at low selectivity) or post-filtering (traversing the vector graph and discarding non-matching results, which can return empty or insufficient nearest neighbors). Qdrant extends the HNSW graph by constructing additional payload-aware edges between points sharing metadata attributes. For complex multi-attribute compound filters, Qdrant utilizes the ACORN search algorithm to traverse connected subgraphs dynamically.
• Segment-Based Immutability: Collections are partitioned into independent segments. Dynamic memory buffers accept writes before converting them into immutable, memory-mapped (mmap) disk files, allowing background index optimization without blocking read queries.
• Scalar and Product Quantization with Rescoring: Qdrant supports 1-byte Scalar Quantization (SQ) and Product Quantization (PQ), reducing in-memory index footprints by up to 75% to 95%. Queries execute fast distance sweeps over quantized vectors in RAM, followed by an exact rescoring pass against full-precision vectors loaded from disk cache.

Scale Ceilings and Bottlenecks

• Cluster Orchestration: While Qdrant offers distributed consensus via Raft and horizontal sharding, operational overhead grows when managing multi-node clusters compared to running a single binary.
• Sweet Spot: 1 million to 100 million vectors where low search latency (p50 under 5ms), high QPS, and strict metadata filtering are mandatory.

3. Milvus: Disaggregated Billion-Scale Infrastructure

Milvus (developed by Zilliz and hosted by the LF AI & Data Foundation) is architected specifically for horizontal scalability across massive enterprise vector collections.

Architecture and Mechanics

• Disaggregated Storage and Compute: Milvus separates concerns into four distinct layers:
• Access Layer: Stateless proxy nodes that validate client requests, parse queries, and merge final results.
• Coordinator Layer: Manages metadata, cluster health, and assigns tasks (RootCoord, QueryCoord, DataCoord). RootCoord coordinates global timestamp sequencing (TSO) via a Time Tick mechanism to ensure consistent read views.
• Worker Layer: Independent stateless worker pools for indexing (IndexNodes), writing/segment compaction (DataNodes), and querying (QueryNodes).
• Storage and Log Broker: Uses Kafka or Apache Pulsar as a persistent write-ahead log (WAL) and MinIO or AWS S3 for long-term segment persistence.
• Knowhere Execution Engine: The core vector computation layer written in C++, wrapping Faiss, HNSW, Annoy, and GPU-accelerated execution pipelines.

Scale Ceilings and Bottlenecks

• Operational Footprint: A production Milvus cluster requires Kubernetes, etcd, Pulsar/Kafka, and MinIO/S3. This multi-component architecture introduces substantial DevOps maintenance overhead for smaller workloads.
• Small-Dataset Latency Tax: Multi-hop proxy routing and coordinator coordination introduce higher base query overhead (p50 around 15ms to 30ms) compared to bare in-process or single-binary engines.
• Sweet Spot: 100 million to multi-billion vectors, multi-tenant enterprise data warehouses, and systems requiring independent scaling of write ingestion and query throughput.

4. Pinecone: Managed Serverless Vector Indexing

Pinecone provides a fully managed closed-source cloud vector database, eliminating infrastructure provisioning.

Architecture and Mechanics

• Serverless Tier Separation: Pinecone serverless architecture stores raw vector indexes on cloud object storage (Amazon S3 / Google Cloud Storage) while provisioning dynamic caching nodes on local NVMe SSDs to serve queries.
• Automated Sharding and Partitioning: Users interact with logical namespaces and indexes without configuring replicas, segment thresholds, or memory buffers manually.
• Metadata Filtering: Built-in metadata indexing handles structured filters alongside vector queries automatically.

Scale Ceilings and Bottlenecks

• Vendor Lock-in and Cloud Portability: Closed source architecture prevents on-premises deployment or private cloud migration.
• Cold-Start Latency: Queries hitting uncached index partitions stored in object storage can experience higher latency tails compared to dedicated memory-resident instances.
• Cost Predictability: Usage-based query unit and write pricing can escalate quickly under sustained high-throughput enterprise workloads.
• Sweet Spot: Teams seeking zero-ops deployment, serverless pricing models, and rapid time-to-market without infrastructure management.

Technical Comparison Matrix

PostgreSQL (pgvector + pgvectorscale)

• Core Language / Engine: C / PostgreSQL extension, Rust (pgvectorscale)
• Index Types: HNSW, IVFFlat, StreamingDiskANN
• Quantization Support: halfvec (fp16), bit (binary), Statistical Binary Quantization (SBQ)
• Metadata Filtering: Relational SQL WHERE clauses, GIN/B-tree indexes, label-based DiskANN
• Primary Deployment Model: Self-hosted, AWS RDS, Supabase, Timescale Cloud
• Best Fit Workload: Under 50M vectors with existing PostgreSQL relational data and strict ACID needs

Qdrant

• Core Language / Engine: Rust
• Index Types: HNSW, Filterable HNSW, ACORN
• Quantization Support: Scalar Quantization (int8), Product Quantization (PQ), Binary Quantization
• Metadata Filtering: Payload indexes with dynamic query planner (subgraph walk / scan switch)
• Primary Deployment Model: Single Docker container, Kubernetes, Qdrant Cloud
• Best Fit Workload: 1M to 100M vectors requiring sub-10ms latency and complex payload filtering

Milvus

• Core Language / Engine: Go (orchestration), C++ (Knowhere core)
• Index Types: HNSW, IVF_FLAT, IVF_SQ8, DiskANN, SCANN, GPU-accelerated indexes
• Quantization Support: SQ8, PQ, FastScan, binary quantization
• Metadata Filtering: Bitset masking and scalar inverted indexing
• Primary Deployment Model: Distributed Kubernetes cluster (Helm/Operator), Zilliz Cloud
• Best Fit Workload: 100M to 1B+ vectors with decoupled streaming ingestion and cloud-native scaling

Pinecone

• Core Language / Engine: Proprietary
• Index Types: Proprietary graph and inverted index structures
• Quantization Support: Internal automatic compression
• Metadata Filtering: Integrated metadata filtering per vector
• Primary Deployment Model: Fully managed SaaS (Serverless / Dedicated Pods)
• Best Fit Workload: Zero-infrastructure RAG applications with elastic usage patterns

Practical Production Decision Rules

When designing a vector storage tier for production RAG and semantic retrieval, consider the following five architectural guidelines:

1. Do not introduce a separate vector database if your dataset is under 10 million vectors and you already run PostgreSQL. Running pgvector or pgvectorscale eliminates distributed synchronization bugs, avoids extra network hops, and allows transactional joins with user identity and authorization tables.
2. Choose Qdrant when query latency, developer ergonomics, and filtered retrieval dominate your workload. Qdrant single-binary operational model offers the lowest barrier to entry among dedicated engines while delivering market-leading filtered HNSW performance.
3. Reserve Milvus for true high-volume distributed deployments. Unless your organization manages hundreds of millions of vectors or needs to independently scale ingestion streams via Pulsar/Kafka, the operational overhead of running Milvus microservices outweighs its scaling benefits.
4. Leverage Quantization with Rescoring before scaling hardware. Storing full 1536-dimensional float32 vectors in RAM is rarely necessary. Employing int8 scalar quantization or product quantization with an in-memory oversampled rescore pass routinely yields 70% to 90% RAM reductions with less than 1% drop in top-k retrieval accuracy.
5. Evaluate filter selectivity before selecting an index strategy. If your queries frequently filter down to less than 1% of the corpus (e.g., tenant-specific document partitions), traditional HNSW graph traversals can stall. Ensure your chosen engine supports payload-aware subgraphs or index-assisted partition pruning.

Sources

• Tiger Data: Pgvector Is Now Faster than Pinecone at 75% Less Cost
• pgvectorscale GitHub Repository
• pgvector Extension Repository
• Qdrant: Combining Vector Search and Filtering
• Qdrant: Filtered Vector Search: What ACORN Fixes, and What Fixes ACORN
• Milvus Architecture Overview
• Milvus: What Powers Similarity Search in Milvus Vector Database (Knowhere)
• Pinecone Serverless Architecture
• Microsoft Research: DiskANN: Fast Accurate Bilateral Nearest Neighbor Search on Large Datasets