Dedicated, client-server vector databases like Milvus, Qdrant clusters, and Pinecone dominate enterprise discussions around retrieval-augmented generation (RAG). However, production engineering reality increasingly favors a different topology: embedded, in-process vector engines.
Running vector search directly inside the application process eliminates network round-trip overhead (typically 15-50ms over cross-datacenter or cloud VPC hops), removes dedicated database infrastructure management, and slashes hosting bills from hundreds of dollars per month to zero fixed infrastructure cost. For edge AI applications, CLI developer tools, desktop agents (such as AnythingLLM and Cursor), and serverless execution environments (like AWS Lambda and Cloudflare Workers), embedded vector engines are often the only viable architecture.
Yet the internal architectures of leading embedded vector stores differ radically. Choosing between LanceDB, sqlite-vec, DuckDB-VSS, and Chroma requires evaluating fundamental trade-offs in memory-mapping, index serialization, vector quantization, and analytical filtering.
1. Architectural Paradigms: How Each Engine Manages State
LanceDB: Disk-First Columnar Storage via Apache Arrow
LanceDB is built in Rust on top of the open-source Lance file format, a modern columnar format engineered specifically for multimodal AI and vector retrieval. Unlike Parquet, which is optimized for sequential OLAP scans, Lance is designed for fast random access and zero-copy reads.
LanceDB decouples vector indexing from RAM residency. Its primary indexing algorithm is Inverted File Product Quantization (IVF-PQ):
- Partitioning (IVF): The vector space is partitioned into Voronoi cells using k-means clustering. Only centroids are cached in memory.
- Quantization (PQ): High-dimensional vectors within each partition are compressed into short byte codes (typically 8 to 64 bytes) via sub-vector quantization.
- Disk-Resident Scanning: At query time, LanceDB identifies candidate partitions using the in-memory centroids, then reads only the corresponding quantized codes directly from disk via memory-mapped files (mmap) or cloud object storage (Amazon S3, MinIO).
- Reranking: Top candidates are deserialized and reranked using full-precision vectors read on-demand from disk.
Because only index metadata and centroids reside in memory, LanceDB can query multi-million vector datasets on machines with modest RAM (for example, searching 10 million 1536-dimensional vectors on an 8GB RAM host).
sqlite-vec: Zero-Dependency C Virtual Tables for SQLite
Created by Alex Garcia to replace the C++ Faiss-dependent sqlite-vss, sqlite-vec is written entirely in portable C with zero third-party dependencies. It integrates into SQLite through the virtual table mechanism.
Key architectural characteristics include:
- Flat SIMD Brute-Force Execution: Rather than building complex graph indices that require substantial indexing time and memory overhead, sqlite-vec implements highly optimized flat scans using AVX2, AVX-512, and ARM NEON intrinsics.
- Native 1-Bit Binary Quantization: sqlite-vec supports packed 1-bit boolean vectors. When paired with binary-quantized embedding models like mxbai-embed-large-v1 or nomic-embed-text-v1.5, vector sizes drop by 32x, and distance calculations reduce to hardware-accelerated XOR and popcount CPU operations.
- Relational Integrity: Because it lives directly within SQLite, vector searches can be natively joined against standard relational tables within standard ACID transactions.
DuckDB-VSS: Analytical Vector Extensions on Fixed-Size Arrays
DuckDB introduced native fixed-size array types in version 0.10. The official vss extension integrates vector search into DuckDB's vectorized columnar execution engine using the C++ usearch library.
- HNSW Indexing on Columnar Batches: Users create Hierarchical Navigable Small World (HNSW) indices directly over array columns.
- Metric Functions: Native support for Euclidean (array_distance), cosine (array_cosine_distance), and negative inner product (array_negative_inner_product).
- OLAP Fusion: DuckDB's vector search shines in analytical pipelines where nearest-neighbor filtering is combined with complex analytical SQL: window functions, multi-table joins, grouped aggregations, and direct queries over Parquet or Iceberg lakehouses.
Chroma: In-Process HNSWlib Collections
Chroma provides an embedded mode (chromadb.PersistentClient) that coordinates metadata storage and vector indexing in-process:
- Storage Layer: SQLite manages metadata, collections, and document text.
- Vector Index Layer: A custom vector storage layer powered by hnswlib maintains the approximate nearest neighbor graph.
- Client Usability: Chroma manages embedding generation automatically through pluggable embedding functions, handling tokenization and inference transparently.
2. Technical Comparison Matrix
Core Engine and Indexing Specifications
- LanceDB: Disk-based IVF-PQ / IVF-HNSW index; Rust and Apache Arrow core; ~500 MB to 1.2 GB RAM for 1M 1536-dim vectors; ~800 MB (PQ) / ~2.8 GB (raw) disk footprint; pre-filtering via Arrow columnar pushdown.
- sqlite-vec: Flat SIMD scan and 1-bit Hamming index; pure C (single file) core; minimal memory footprint (scan-bound); ~6.0 GB (Float32) / ~190 MB (1-bit) disk footprint; pre-filtering via SQL joins and virtual tables.
- DuckDB-VSS: Memory-resident HNSW index (via usearch); C++ and DuckDB engine core; ~3.5 GB to 5.0 GB RAM for 1M 1536-dim vectors; ~6.5 GB disk footprint; pre-filtering via vectorized SQL filter pushdown.
- Chroma (Embedded): Memory-resident HNSW index (via hnswlib); Python/Rust core with SQLite metadata; ~6.0 GB to 8.0 GB RAM for 1M 1536-dim vectors; ~6.2 GB disk footprint; pre-filtering via metadata index.
3. Memory Footprint and Scale Limits
The most critical operational differentiator among embedded vector engines is how memory scales with dataset size.
Memory Consumption vs. Vector Count (1536-dim Float32)
RAM (GB)
80 +---------------------------------------------------------+
| |
60 | Chroma |
| DuckDB-VSS |
40 | |
| |
20 | |
| LanceDB |
0 +----------------------------------------------sqlite-vec-+
0 2.5M 5.0M 10.0M
Total VectorsIn-Memory Graph Engines (Chroma, DuckDB-VSS)
HNSW requires graph connectivity layers to remain in RAM for fast traversal. At 1536 dimensions, each vector requires approximately 6 KB of raw data plus 1-2 KB of graph pointer overhead. A 10-million vector index requires 60-80 GB of available RAM. In constrained environments such as AWS Lambda (typically capped at 512 MB to 2 GB for cost efficiency), memory-resident HNSW graphs trigger out-of-memory fatal crashes.
Disk-Mapped Columnar Engines (LanceDB)
Because LanceDB stores PQ codes and raw vectors in disk fragments, its RAM footprint is restricted to the centroid tree and operating system page cache. A 10-million vector index runs comfortably within a 2-4 GB RAM ceiling with sub-20ms query latencies.
Flat SIMD and Binary Engines (sqlite-vec)
For small to medium collections (under 100,000 vectors), flat SIMD scans bypass indexing overhead entirely. At 100k vectors, a brute-force AVX2 scan executes in 5-15 milliseconds. When compressed to 1-bit binary representations, 100,000 1536-dimensional vectors occupy only 19.2 MB of storage, allowing instant in-memory or flash retrieval.
4. Metadata Filtering and Hybrid Search Mechanics
In real-world RAG systems, vector queries rarely execute in isolation; they almost always require metadata filtering (such as tenant isolation, access control lists, timestamps, or category tags).
Pre-Filtering vs. Post-Filtering Mechanics
- Post-Filtering (Naive): The vector index retrieves top-K nearest neighbors, and metadata conditions are evaluated afterward. If 95% of documents are filtered out by a tenant constraint, top-10 retrieval often returns zero valid results.
- Pre-Filtering with Columnar Pushdown (LanceDB and DuckDB): LanceDB and DuckDB evaluate SQL filter predicates directly on columnar metadata columns prior to or during vector scanning. LanceDB leverages Apache Arrow compute kernels to generate bitmap masks, skipping disk partitions that contain zero matching rows.
- Virtual Table Joins (sqlite-vec): In sqlite-vec, scalar data lives in standard SQLite B-tree tables while embeddings live in vec0 virtual tables. As Simon Willison demonstrated in his analysis of SQLite hybrid search, combining sqlite-vec with SQLite's native Full-Text Search (FTS5) via Reciprocal Rank Fusion (RRF) enables clean, single-query hybrid search without external search infrastructure:
WITH vector_matches AS (
SELECT rowid, distance,
ROW_NUMBER() OVER (ORDER BY distance) AS rank
FROM vec_documents
WHERE embedding MATCH :query_vector AND k = 20
),
text_matches AS (
SELECT rowid,
ROW_NUMBER() OVER (ORDER BY rank) AS rank
FROM documents_fts
WHERE documents_fts MATCH :text_query
LIMIT 20
)
SELECT d.id, d.title, d.content,
(COALESCE(1.0 / (60 + v.rank), 0.0) +
COALESCE(1.0 / (60 + t.rank), 0.0)) AS rrf_score
FROM documents d
LEFT JOIN vector_matches v ON d.id = v.rowid
LEFT JOIN text_matches t ON d.id = t.rowid
WHERE v.rowid IS NOT NULL OR t.rowid IS NOT NULL
ORDER BY rrf_score DESC
LIMIT 10;5. Architectural Decision Framework
Engineers evaluating embedded vector databases should align their selection with four deployment constraints:
Do you need embedded vector search?
│
YES
│
Is your dataset larger than 500k vectors
or running in low-memory serverless?
╱ ╲
YES NO
╱ ╲
[LanceDB] Do you require complex OLAP
(Disk-based IVF-PQ, analytics, aggregations, or
Arrow columnar engine) direct Parquet queries?
╱ ╲
YES NO
╱ ╲
[DuckDB-VSS] Is zero-dependency C/Wasm/
(Vectorized OLAP, mobile portability critical?
usearch HNSW) ╱ ╲
YES NO
╱ ╲
[sqlite-vec] [Chroma]
(C virtual table, (Turnkey Python SDK,
flat SIMD/1-bit) in-memory HNSWlib)1. Select LanceDB when:
- Vectors exceed 500,000 entries and must run on single-node instances or serverless workers without memory exhaustion.
- Datasets involve multimodal attributes (images, audio features, raw document text) stored alongside embeddings.
- Zero-copy reads from local NVMe or cloud object storage (Amazon S3) are required.
2. Select sqlite-vec when:
- Maximum portability is required across embedded devices, mobile apps (iOS/Android), WebAssembly, or edge workers.
- The system already relies on SQLite for core application state and requires ACID transactions across scalar and vector data.
- Binary quantization (1-bit vectors) is feasible for your embedding model.
3. Select DuckDB-VSS when:
- Vector search is an extension of an existing analytical query engine or data engineering pipeline.
- Queries frequently combine vector similarity scoring with window functions, joins against remote Parquet files, or complex grouping.
4. Select Chroma when:
- Rapid prototyping in Python or TypeScript requires an all-in-one local vector store with automated embedding management.
- Dataset sizes remain under 250,000 vectors where in-memory HNSW graph performance delivers sub-millisecond query execution.
