Subscribe
Analysis
7 min read

What a Million-Document RAG Actually Costs in Late 2026

Embedding the corpus is a one-time rounding error. The meters that never stop scale with index size and query volume, and they set the real budget.

What a Million-Document RAG Actually Costs in Late 2026

Ask a team what their RAG stack costs and you will get a number built from an embedding price page. That number is almost always wrong, and wrong in a specific direction: it budgets for the one-time fee and ignores the recurring meters, which at production query volume run one to two orders of magnitude larger.

This piece prices a concrete workload end to end: one million documents, indexed once, queried a million times a month. All rates are published list prices collected by independent pricing trackers in September and October 2026 and cross-checked against vendor pages, and every figure below links to where it came from. Verify against the vendor before you commit a budget.

The corpus

Start with 1M documents at roughly 800 tokens each. That is 800M raw tokens. Chunk them into 500-token pieces with overlap, and the chunked text inflates 10 to 25% above raw size, so plan on billing close to 920M to 1B tokens for the full ingest (EmbeddingCost). The chunking itself produces roughly 2M chunks, and therefore 2M vectors to store.

Three meters now apply, and only one of them is one-time.

Meter one: the ingest. Call it the team-lunch line

Embedding APIs charge for input tokens only. There are no output tokens, which makes embeddings 10 to 100 times cheaper per token than chat completions at the same volume (EmbeddingCost). What that buys at ~920M tokens:

Model$/M tokensBatch $/MIngest cost, ~920M tokens
OpenAI text-embedding-3-small0.020.01$18
Amazon Titan V2 (Bedrock)0.02-$18
voyage-4-lite0.020.013$18
voyage-40.060.04$55
Mistral Embed0.10-$92
Cohere Embed 5 Fast0.08-$74
Cohere Embed 5 Pro / voyage-4-large0.120.08 (voyage)$110
OpenAI text-embedding-3-large0.130.065$120
Google gemini-embedding-20.200.10$184
BGE-M3, self-hosted on spot A100~0.001-~$1

Rates from EmbeddingCost (verified September 2026) and Rework's workload guide (checked against vendor pages 30 October 2026; OpenAI's own pricing page and Voyage's pricing docs carry the same figures).

Two footnotes. Voyage's first 200M tokens per account are free, so the ingest above starts at $55, not $110, and pilots under 200M tokens can ride at zero. And the self-hosted row is a rough inference cost on a spot GPU, not a TCO: a full-time spot A100 month runs about $1,080 whether you use it or not, and the break-even against the cheap APIs lands somewhere above 15M tokens per month (EmbeddingCost).

So: the whole corpus, every word of it, turned into vectors for somewhere between $18 and $184 once. This is the smallest number in the article.

One coin on a plate beside a conveyor of coins receding into the distance

Meter two: storage, which is also cheap

A vector's raw footprint is dimensions x 4 bytes in float32. 2M vectors at 1024 dimensions is about 8 GB; at 1536 dimensions, about 12 GB, before graph overhead (the arithmetic is laid out in EmbeddingCost's storage FAQ and Big Data Boutique's engine comparison).

  • Pinecone serverless: $0.33/GB-month, so $3 to $4 a month for the index (Pinecone pricing; rates confirmed by Big Data Boutique, read from vendor pages 25 October 2026).
  • Weaviate Cloud Flex: bills stored dimensions at $0.00465 per million dimensions plus storage from $0.12/GiB, with a $45 monthly floor. Our 2M x 1536 vectors hit about $14 in dimension charges plus a couple of dollars of storage, so you simply pay the floor (Layerbase, Big Data Boutique).
  • Qdrant Cloud: no per-GB rate; you buy clusters on vCPU, RAM and disk. A published planning figure is roughly $65 a month at 10M vectors, so this corpus needs a smaller slice (Spendark).
  • pgvector: no vector SKU at all. You pay for the Postgres instance, typically $50 to $180 a month on RDS, whether vectors are in it or not (Spendark, Cipher Projects).
  • Amazon S3 Vectors, generally available since December 2025, lists storage around $0.06/GB-month, the cheapest parking spot for indexes queried rarely (Big Data Boutique).

Storage alone, on any of these, is single-digit to low double-digit dollars a month. Still not the bill.

Meter three: reads, which is the bill

This is where the shape changes. On query-metered serverless, read units do not scale with your token count or your document count. They scale with query volume multiplied by index size. Pinecone's Standard plan lists read units at $16 to $18 per million, and Rework's worked example spells out the meter semantics: roughly one read unit per GB of index size, per query.

Take 1M queries a month against the ~12 GB index. That is ~12M read units, about $200 a month at list. The $4 storage line is now noise; reads are 98% of the vector bill. Cipher Projects' planning bands say the same thing from the survey side: at 10M vectors and medium query load, a Pinecone-class bill runs $400 to $1,500 a month with read units dominating, and at 50M vectors with high QPS, "query-metered bills can climb hard" enough that capacity-metered or self-hosted engines win on pure cost.

A probe touching one glowing cube in a grid, wired to a large analog meter with its needle swinging high

The inverse failure mode is just as real: on capacity-shaped plans (Weaviate, Qdrant, pgvector instances) you pay the floor whether traffic is there or not. The Vector DB pricing consensus across these reviews is a rule teams should actually follow: pick the meter model, query-metered versus capacity-metered, before you pick the vendor logo. Wrong meter, surprise bill, excellent product notwithstanding.

One more line for completeness: embedding the queries themselves is cents to a couple of dollars a month. A million average queries at 100 tokens is 100M tokens, $2 on the small OpenAI model. And the LLM generating the answers sits behind all of this as a separate, much larger bill. This article is about the retrieval layer only.

The year-one picture

Same workload, three architectures, monthly steady state after the one-time ingest:

  • Serverless, query-metered (Pinecone-class): ~$4 storage + ~$200 reads + $2 query embeds, call it $250 a month, $3,000 a year, of which the ingest was $18.
  • Managed, capacity-metered (Weaviate Flex floor or a small Qdrant cluster at this size): $45 to $100 a month, steadier, less elastic when traffic goes quiet.
  • Self-hosted on existing Postgres (pgvector, under ~5M vectors and moderate concurrency): the instance you already pay for, plus headroom.

The structural lesson survives whichever row you pick: the ingest is a rounding error, and the recurring meter tracks how big your index is and how often you touch it. Budget against documents and you budget against the wrong variable.

The re-embed nobody prices

Every stored vector is tied to the exact model version that produced it. Switch vendors, or even upgrade within one vendor, and the corpus regenerates from scratch. Google says it outright in its own embedding docs: gemini-embedding-001 and gemini-embedding-2 produce incompatible vector spaces, so the upgrade requires re-embedding everything (Rework, citing Google AI for Developers).

The dollars are trivial, $18 to $184 above, which is precisely why they get ignored. The cost is operational: re-chunking if your strategy changed, standing up a parallel index, dual-write cutovers so search never goes dark, and recall validation before deleting the old vectors. Rework frames it correctly: a multi-day project triggered by an upgrade you did not initiate, recurring every time a vendor ships a generation.

Two mitigations have real teeth. Cohere's Embed 5 Pro and Fast variants share one embedding space, so you can index with the stronger model and query with the cheaper one without re-indexing (Cohere, 30 September 2026). And Matryoshka-style truncatable models (OpenAI v3, Voyage v4, Cohere Embed 5, Qwen3) let you shrink stored dimensions, which halves the Weaviate-style dimension meters and the storage lines, without paying anything extra on the API token bill.

Where the cuts actually come from

  • Batch APIs: OpenAI halves embedding rates for 24-hour-turnaround indexing jobs; Voyage discounts a third. Bulk ingest is never urgent. Use the discount (EmbeddingCost).
  • Quantization: binary quantization compresses vectors up to 32x. Qdrant's docs report 0.98 recall at 100 with 4x oversampling on OpenAI embeddings, and Weaviate's BQ reportedly cuts the cloud bill around 5x, because on capacity meters the RAM you save is the money you stop paying (Big Data Boutique, Spendark).
  • Idle-index placement: rarely-queried tenant indexes move to object-storage engines like S3 Vectors or turbopuffer, where an unloaded index costs only its storage bytes; budget for the cold-query latency in exchange (Big Data Boutique).
  • Know your exit: raw vectors are portable, metadata schemas and filter semantics are not. Keep vectors next to source documents and model the re-embed event before the first vendor call (Cipher Projects).

The million-document RAG stack is cheap to build and not cheap to run. The invoice agrees with you right up until the first month of queries, and after that, it never stops agreeing with the size of your index instead.

Sources

Analysis