Retrieval vs generation spend
Feeding retrieved chunks into the model (input tokens) is usually the bigger line in RAG.
Home / Dev & AI / AI & LLM Cost / RAG Cost Calculator
AI & LLM Cost
RAG Cost Calculator
Price a retrieval augmented generation pipeline end to end. Set your knowledge base size, embedding model, queries per month and answer model, and see the one-time index cost plus the monthly query bill.
Estimated monthly cost
$0
One-time index cost—
Cost per query—
Answer generation / month—
Query embedding / month—
Queries per month—
Annual query estimate—
Your RAG answers priced across LLMs
Same query load, cheapest answer model first.
What the RAG Cost Calculator does
Retrieval augmented generation, or RAG, grounds a language model in your own documents. It has two cost centres. First you embed your knowledge base into vectors once, a one-time indexing charge that scales with corpus size. Then for every question you embed the query, retrieve relevant chunks, and feed those chunks plus the question into an LLM that writes the answer.
This RAG cost calculator separates those pieces so you see the upfront index cost apart from the recurring per query cost. Because the retrieved context is fed to the model as input tokens on every question, generation is usually the largest ongoing line, and the tool compares answer models so you can pick the right one.
Heads up on pricing. The rates built into this tool are public list estimates for July 2026 and can change without notice. Providers also offer batch discounts, prompt caching and volume tiers that lower real costs, so treat the output as a planning estimate, not a quote.
The formula
index cost = corpus tokens ÷ 1M × embedding rate (one time)
per query = (query embed) + (context tokens ÷ 1M × input rate) + (answer tokens ÷ 1M × output rate)
monthly cost = per query × queries per month
per query = (query embed) + (context tokens ÷ 1M × input rate) + (answer tokens ÷ 1M × output rate)
monthly cost = per query × queries per month
Worked example
Indexing 5,000,000 tokens with text-embedding-3-small at $0.02 per million costs about $0.10 once. For 60,000 queries a month, each feeding 3,000 context tokens and writing 400 answer tokens on GPT-5 mini, generation is 180M × $0.25 + 24M × $2 per million = $45 + $48 = about $93 a month plus tiny query embedding.
Embedding pricing used for indexing (per 1M tokens, July 2026 estimates)
| Embedding model | Provider | Price / 1M tokens |
|---|---|---|
| text-embedding-3-small | OpenAI | $0.02 |
| text-embedding-3-large | OpenAI | $0.13 |
| Gemini Embedding | $0.15 | |
| Cohere Embed 4 | Cohere | $0.12 |
Embedding is usually a small one-time cost. The recurring bill comes from feeding retrieved context into the answer model, which is why the choice of LLM matters most.
Ways to lower your AI costs
- Match the model to the task. Route simple requests to a small model like GPT-5 mini, Gemini Flash or DeepSeek V3, and keep premium models for complex reasoning where quality pays off.
- Cache repeated context. Prompt caching means you stop paying full price for the same system prompt or documents on every call.
- Cap output length. Output tokens cost several times more than input, so limiting response length is often the fastest saving you can make.
- Trim prompts and history. Summarize old conversation turns and drop context the model does not actually need to answer well.
- Batch non-urgent work. Many providers offer batch pricing at around half the standard rate for jobs that can wait a few hours.
Frequently asked questions
What makes RAG expensive?
The generation step. Every query feeds retrieved chunks to the model as input tokens, so large context windows and premium models drive the ongoing cost far more than embedding does.
Is indexing a recurring cost?
Mostly no. You embed the corpus once, then only re-embed new or changed documents. That is why the tool shows index cost separately as a one-time figure.
How do I lower RAG costs?
Retrieve fewer, more relevant chunks, use a smaller answer model, cache the system prompt, and keep answers concise. Tighter retrieval is the biggest lever.
Do I pay for the vector database too?
Yes, hosting the vectors has its own cost outside token pricing. This tool focuses on the AI token and embedding spend, which is usually the larger share at scale.
Are these estimates current?
They use July 2026 pricing. Confirm live embedding and model rates before budgeting a production pipeline.