How much does a Bedrock RAG system actually cost?

IThe five line items in a Bedrock RAG bill

Almost every team budgets a RAG system by pricing the generation model and stops there. That misses more than half the bill. A RAG system has five distinct cost lines, each priced on a different basis, and two of them — the vector store and re-embedding — are the ones that surprise people. Get all five on the table before you build.

It helps to separate the five lines by when you pay them, because the timing is what makes the bill hard to reason about. Two are tied to your data (you pay when you index or re-index, regardless of traffic), one is a continuous baseline (you pay every hour the system exists, even at zero queries), and two are per-query (they scale with how much the system is used). A RAG bill that looks wrong is almost always a confusion between these three timing buckets.

Line 1 — embeddings (indexing). Before anyone asks a question, every chunk of your corpus is turned into a vector by an embedding model (Amazon Titan Text Embeddings v2 or Cohere Embed on Bedrock). You pay per input token embedded, once, when you first index the corpus. Embedding is among the cheapest things on Bedrock — representatively cents-to-low-dollars per million tokens — so even a large corpus is usually a one-time charge in single-to-low-double-digit dollars. This line is almost never the problem; people just forget it exists.

Line 2 — the vector store (continuous baseline). The vectors have to live somewhere that can answer nearest-neighbour queries, and that store runs 24/7 whether or not anyone uses it. This is the line that breaks budgets, because the common default on AWS — Amazon OpenSearch Serverless — has a minimum capacity floor (see §III). It is a fixed monthly cost decoupled from both corpus size at small scale and from traffic. For a small system it is frequently the largest line on the entire bill.

Line 3 — query embeddings (per query). Every incoming question is embedded with the same model before retrieval. This is one short embedding call per query — genuinely negligible (fractions of a cent), even at high volume. Mentioned for completeness; it never moves the total.

Line 4 — re-ranking (per query). If you re-rank (and you usually should — it is the highest-leverage retrieval-quality step), each query sends its top-K candidate chunks through a cross-encoder re-ranker (Amazon Rerank or Cohere Rerank on Bedrock), billed per query and per the volume of text re-ranked. Modest per query, but it scales with query volume and with how wide a net you re-rank.

Line 5 — generation (per query, usually the largest at scale). The final step sends the system prompt plus the re-ranked chunks plus the question to a generation model (Claude, Amazon Nova, Llama, Mistral on Bedrock) and pays for input and output tokens. In RAG, the input is inflated by the retrieved context — often several thousand tokens of chunks per question — so generation is typically the dominant cost once query volume is non-trivial. It is also the line with the most levers (model choice, chunk count, prompt caching, max-output).

IIThe one-time embedding cost, and re-embedding on updates

The first real number to compute is what it costs to embed your corpus once. It is almost always small, but it is the cleanest place to start because the math is exact: corpus size in tokens × the embedding price per token. The subtler cost is re-embedding, which turns a one-time charge into a recurring one whenever your data changes.

Start by sizing the corpus in tokens, not documents. A rough rule: 1,000 tokens ≈ 750 words, and a typical text-heavy page is 500–800 words, so call it ~1,000 tokens per page as a planning figure. A 10,000-document corpus averaging 5 pages each is therefore on the order of 50 million tokens. Chunking adds a little overhead (chunk boundaries and any repeated titles in metadata), but for budgeting, total corpus tokens is the right input.

Now multiply by the embedding price. Representatively in 2026, Amazon Titan Text Embeddings v2 lands around $0.00002 per 1K tokens (i.e. ~$0.02 per million); Cohere Embed is in a similar low band. So embedding that 50M-token corpus costs roughly 50 × $0.02 ≈ $1 — a rounding error. Even a very large corpus of 1 billion tokens is on the order of $20 as a one-time charge. This is why embedding is never the line to optimise for cost; it is cheap by design because AWS wants your data indexed and resident.

Embedding dimensionality is a lever, but it is a storage and latency lever, not an embedding-cost one. Titan v2 lets you choose output dimensions (e.g. 256 / 512 / 1024); smaller vectors cost the same to generate but take less room in the vector store and search faster. So dimensionality affects Line 2 (the vector store) and retrieval latency — not the one-time embedding bill in any meaningful way.

IIIThe vector store: the always-on cost that surprises everyone

This is the section that changes how teams budget RAG. Unlike embeddings and inference, the vector store is a continuous cost you pay every hour the system exists — and the most common option on AWS, OpenSearch Serverless, has a minimum floor that makes a tiny, idle RAG system cost far more than its token usage suggests. Understanding the cost shape of each store is the difference between a $30/month surprise and a $700/month one.

There are four mainstream vector stores on AWS, and they have fundamentally different cost shapes. Two bill on reserved/baseline capacity (you pay for headroom whether you use it or not), and two bill closer to actual usage. The store you pick is the single biggest swing on a small RAG system's monthly bill — bigger than the model choice — because at low traffic the store baseline dominates everything else.

IVPer-query cost: retrieval, re-ranking, and generation

The per-query lines are what turn a fixed monthly baseline into a bill that grows with usage. Three things happen on every question — a query embedding, an optional re-rank, and a generation call — and the third one dominates. The key insight for RAG specifically: retrieved context inflates the input token count, so generation in RAG costs more per query than a bare chatbot.

Walk a single query through the pipeline and price each step. Query embedding: one short embedding call (the question, ~50–200 tokens) at ~$0.00002/1K — a small fraction of a cent. Across 100,000 queries a month that is still pennies. Ignore it in any budget.

Re-ranking: the retriever returns a wide net (say top-30 to top-50 chunks), and the re-ranker scores each against the question. Re-rank pricing is representatively per query (sometimes per the volume of text scored), and lands in the small-fraction-of-a-cent to low-cents range per query depending on how wide a net you re-rank. At 100,000 queries/month this is typically a few dollars to low tens of dollars — real but rarely dominant. The lever is obvious: re-rank top-30, not top-300, and skip re-ranking on trivial queries.

Generation — the line that matters. This is where RAG diverges from a plain chatbot. The input to the generation model is not just the question; it is the system prompt + the re-ranked chunks + the question. Those chunks are commonly 2,000–4,000 tokens of retrieved context per query — far more than the question itself. So a RAG query's input is dominated by context, and since you pay per input token, the amount of context you pass is a direct cost lever, not just a quality one. Output is whatever the model writes (often 300–800 tokens for a grounded answer).

Make it concrete with a representative mid-tier setup — Claude Sonnet-class generation, ~3,500 input tokens (≈3,000 of context + 500 question) and ~600 output tokens per query. At representative Sonnet rates ($3/1M input, $15/1M output): input ≈ $0.0105, output ≈ $0.009 → roughly $0.02 per query. At 20,000 queries/month that is ~$400; at 100,000 it is ~$2,000 — generation alone. Swap to a cheaper model for the easy questions (Nova Lite, Claude Haiku) and the per-query cost drops by 5–20×; that is the single biggest per-query lever.

VRAG cost across corpus sizes and query volumes

This is the table most people come for: representative all-in monthly cost for RAG systems of different sizes, broken into the lines that move. It shows the two regimes clearly — small systems are dominated by the vector-store baseline, large systems by generation tokens — and where the crossover happens. Figures are illustrative 2026 estimates to show shape and order of magnitude, not quotes.

Read the table as four representative systems, smallest to largest. The embedding column is the one-time corpus embedding (amortised, it is a rounding error monthly). The vector store column assumes OpenSearch Serverless near its floor for the small systems and growing capacity for the large ones — choosing pgvector instead would cut the small-system rows substantially (shown in the note). The generation column assumes a sensible model mix (cheap model for most queries, a frontier model for the hard minority) at ~$0.01–$0.02 blended per query. The takeaway is the shape: the bill barely moves with corpus size but moves a lot with query volume and model choice.

VIManaged Knowledge Bases vs DIY: the cost difference

There are now two kinds of knowledge base, and they bill differently. A customer-managed knowledge base passes through the underlying resources: you pay for the same embedding tokens, the same vector store and the same generation tokens you would pay in a pipeline you build yourself. A Bedrock Managed Knowledge Base has its own prices instead: $5.00 per GB of raw data per month for storage and $1.00 per 1,000 standard retrievals (pricing page, 10 October 2026), with parsing, embeddings and reranking included and no vector store to run. The rest of this section compares customer-managed with DIY.

A customer-managed knowledge base is largely a pass-through on the underlying resources. When you use one, you still pay Bedrock for embedding tokens, you still provision and pay for the vector store (OpenSearch Serverless, Aurora pgvector, S3 Vectors, Pinecone, Redis or another supported store), and you still pay generation tokens on every RetrieveAndGenerate call. The managed convenience is not priced as a big per-query premium; it is priced as you do not get to hand-tune every stage. So on pure infrastructure dollars, managed and DIY are close for the same architecture.

Where they diverge is on the two things that are not on the AWS invoice. First, engineering cost: managed ships in hours with almost no pipeline to maintain, while DIY is days-to-weeks of build plus ongoing maintenance of the parser, chunker, retriever, and re-ranker. For most teams that engineering time is the larger real cost, and it favours managed decisively. Second, optimisation headroom: DIY lets you squeeze the AWS bill in ways managed cannot — custom chunking to cut context tokens, your own hybrid-search tuning, sharing one vector store across many indexes to amortise the baseline, aggressive caching, and routing each query to the cheapest adequate model. At very high volume, that headroom can pay for the extra engineering.

The practical rule mirrors the build decision: start managed (the infrastructure cost is essentially the same and you ship far faster), and move specific stages to DIY only when volume is high enough that the optimisation headroom — not the managed markup, which is small — justifies the engineering. The cross-over is about scale and the value of fine-grained cost control, not about Knowledge Bases being expensive.

VIIThe cost surprises: what blows up a RAG budget

Most RAG budget overruns trace to a short list of avoidable surprises. None is exotic; each is a place where the cost is decoupled from what you think you are paying for. Knowing them in advance is worth more than any single price figure.

• The OpenSearch Serverless minimum: The number one surprise. A production OSS collection carries a baseline of OCUs you pay 24/7 — representatively a few hundred dollars a month — independent of corpus size at small scale and of traffic. A "small" RAG POC can cost more in idle vector-store capacity than in everything else combined. Fix: use Aurora pgvector for small/early systems, share one collection across indexes, or use a usage-priced store (Pinecone serverless) until volume justifies OSS.
• Context bloat in generation: Every query pays for the retrieved chunks as input tokens. Passing 8–10 fat chunks "to be safe" can triple the input cost per query versus 3–5 tight, re-ranked ones — across 100K queries that is real money. Fix: re-rank down to the few best chunks, use hierarchical chunking, and cap retrieved context.
• Frontier model for every query: Using a top-tier model for all traffic when most questions are easy is the most expensive default. The easy 80% of queries answer fine on a cheap model. Fix: a tiered router — cheap model first, frontier model only for the hard minority — routinely cuts generation cost 5–10×.
• Not using prompt caching on the system prompt: RAG sends the same long system prompt (and sometimes shared context) on every call. Without prompt caching you re-pay full input price for it every single query. Fix: enable Bedrock prompt caching for the static prefix — on high-volume systems it cuts a large slice of input cost.
• Full re-embeds you did not budget for: Incremental re-embedding is pennies, but changing the embedding model forces a full corpus re-embed and an index rebuild. The token cost is small; the surprise is the operational work and the re-validation. Fix: treat the embedding model as semi-permanent and benchmark before committing.
• Forgetting the supporting services: S3 storage, data-transfer, Lambda/Glue for a DIY ingestion pipeline, CloudWatch logging, and Textract for PDF parsing all add cost around the core five lines. Individually small, collectively a tail. Fix: include them in the budget from the start.
• Over-provisioned vector store at launch: Sizing the vector store for "peak imagination" rather than actual corpus size means paying for capacity you will not use for months. Fix: right-size to current corpus and traffic, and scale the store as the corpus grows — both OSS and Aurora Serverless scale up later.

VIIIWho can file AWS funding for a Bedrock build

There are two kinds of AWS money for a Bedrock build. A partner files one. You apply for the other yourself.

Partner-filed funding. An AWS consulting partner can file POC (proof of concept) funding to test a Bedrock workload, whether or not you run on AWS today. If you already run on AWS and are adding new AI work, the partner can file incremental workload (IW) funding instead. For startups this is often $50K to $150K. Depends on the workload; AWS approves. The work has to be real, and the partner has to be the one doing it.

AWS Activate, which you apply for yourself. Activate Founders gives $1,000 to start, up to $5,000, with no investor needed. Activate Portfolio gives up to $200,000 if your VC, accelerator or angel group is an Activate Provider and gives you its Org ID. A consulting partner cannot apply for Activate on your behalf. AWS made Activate credits usable for third-party models on Bedrock in April 2024.

For most startups without a VC, the partner route is the larger of the two. CloudRoute matches you with one vetted AWS partner for free. Partners pay CloudRoute a monthly membership plus 10% of AWS-approved funding.