Cut your LLM API bill 7x: prompt caching, model routing, Batch, worked example

A runaway AI API bill rarely comes from long answers. It comes from input you resend every time: the system prompt, reference documents and chat history, all billed at full price on every request. This article shows where the money goes, ranks the levers by payoff, and works through a monthly bill step by step.

Prices were checked against Anthropic’s pricing, prompt caching and Batch docs and OpenAI’s pricing page as of October 2026. Prices change often; check those pages again before you decide.

Where the money goes

Every request is billed in three parts:

Part	What it is	Shape
Repeated input	system prompt, manuals, examples, tool definitions, old chat turns	large, identical each time
New input	the latest question, data retrieved for this call	small to medium
Output	the answer (plus thinking tokens, if on)	5x the input price per token

Prices as of October 2026, per million tokens (MTok):

Model	Input	Cache write	Cache read	Output
Claude Sonnet 5.5	$2	$2.50 (5 min)	$0.10	$10
Claude Haiku 5.5 (prompts ≤ 100K tokens)	$0.10	$0.125 (5 min)	$0.01	$0.50
OpenAI gpt-6.1-sol	$2	$2.50 (30 min)	$0.10	$10
OpenAI gpt-6-luna	$0.10	$0.125 (30 min)	$0.01	$0.50

On OpenAI, caching is on by default for prompts of 1,024 tokens or more; on GPT-5.6 and later a write costs 1.25x input for the default 30-minute minimum lifetime.

An app with 6,000 tokens of reference text in the prompt and a 1,000-token question pays for the repeated part on 6 of every 7 input tokens. That is the first target.

The example bill: a support chatbot

Assumptions used throughout:

• 2,000 requests a day × 30 days = 60,000 requests a month
• Each request: 6,000 repeated tokens + 1,000 new tokens, 500-token answer on average
• Claude Sonnet 5.5 for everything

Starting point (nothing done):

• Input: 7,000 × 60,000 = 420M tokens × $2 = $840
• Output: 500 × 60,000 = 30M tokens × $10 = $300
• Total $1,140 a month

Lever 1: prompt caching

Prompt caching stores the processed prefix of a prompt that is identical character for character; later calls pay only the cache-read price. On Sonnet 5.5 a cache read is 0.05x normal input ($0.10 instead of $2), in exchange for a first write at 1.25x. The 5-minute cache is refreshed on every hit.

The easiest way in is a top-level cache_control, which places the breakpoint for you:

{
  "model": "claude-sonnet-5-5",
  "max_tokens": 600,
  "cache_control": { "type": "ephemeral" },
  "system": "You are a customer support assistant ... (6,000-token manual)",
  "messages": [{ "role": "user", "content": "How do I change my delivery address?" }]
}

Check it works in the response usage: cache_read_input_tokens should be large. If it is always 0, your prefix changes on every call.

Recalculate, assuming the cache expires and is rewritten about 300 times a month (quiet night hours with gaps over 5 minutes):

• Cache writes: 300 × 6,000 = 1.8M × $2.50 = $4.50
• Cache reads: 59,700 × 6,000 = 358.2M × $0.10 = $35.82
• New input: 60M × $2 = $120
• Output: $300
• Total $460.32, 60% less, with no change in quality

Order the prompt from stable to changing: tools → system → documents → history → question. Anything that changes per call (dates, user names, ids) never goes up front.

Lever 2: route easy tasks to a small model

Most chatbot questions are “what are your hours” and “how do I track my parcel”. A small model handles them well enough. Say you measured that 60% of requests are easy: they go to Haiku 5.5, the other 40% stay on Sonnet 5.5, and Haiku also does the routing (reads the 1,000-token question, answers in 20 tokens).

Part	Calculation	Cost
Sonnet, 24,000 req	writes $4.50 + reads 142.2M × $0.10 = $14.22 + new input 24M × $2 = $48 + output 12M × $10 = $120	$186.72
Haiku, 36,000 req	writes 1.8M × $0.125 = $0.23 + reads 214.2M × $0.01 = $2.14 + new input 36M × $0.10 = $3.60 + output 18M × $0.50 = $9	$14.97
Router	60M × $0.10 + 1.2M × $0.50	$6.60
Total		$208.29

The catch: measure on your own real questions that the small model actually gets that 60% right; do not guess. How to match models to tasks is covered in choosing models.

Lever 3: trim context and output

Output costs 5x input. Ask for short answers in a fixed format and set max_tokens as a ceiling. Say the average answer drops from 500 to 300 tokens:

• Sonnet output: 7.2M × $10 = $72 (was $120)
• Haiku output: 10.8M × $0.50 = $5.40 (was $9)
• New total $156.69

Sonnet 5.5 has adaptive thinking on by default, and thinking tokens are billed as output. For simple work, lower effort from its default of high and check quality holds.

On the input side, do not resend the whole chat history every turn: keep the last few turns or a summary. And when you use documents, retrieve the relevant chunks instead of pasting the whole manual, as in the RAG article.

Lever 4: the Batch API for anything that can wait

The Batch API takes 50% off both input and output. A batch holds up to 100,000 requests or 256 MB. Most finish within an hour; anything not done in 24 hours expires (expired requests are not billed). Cache pricing stacks with the batch discount. OpenAI’s Batch is also half price.

It cannot serve the chatbot, but it fits side jobs. Say each month you tag topics on 60,000 conversations (1,500 tokens in, 50 out each):

Option	Calculation	Cost
Sonnet, standard	90M × $2 + 3M × $10	$210
Sonnet, Batch	half	$105
Haiku, standard	90M × $0.10 + 3M × $0.50	$10.50
Haiku, Batch	half	$5.25

Lesson: picking the right model matters more than Batch, but Batch is a free 50% whenever the job can wait.

Lever 5: cache answers in your own app

When a question repeats exactly, store the answer in Redis and reply without calling the API (same idea as caching the hot path with Redis). Say 15% of requests (9,000) are served from that cache, checked before the router:

• Repeated questions are usually the easy ones already on Haiku. Haiku cost per request = $11.37 ÷ 36,000 ≈ $0.000316
• Savings: 9,000 × $0.000316 + 15% of the router ($0.99) ≈ $3.83
• Bill ≈ $152.86

Barely any money, but users get instant answers. That is why this lever comes last. Watch for stale answers: clear the cache when the underlying data changes.

The bill, summed up

Step	Per month
Starting point	$1,140
+ prompt caching	$460
+ easy tasks to Haiku	$208
+ shorter output	$157
+ answer cache	$153

About 7x cheaper, and the hard questions still get the same model.

Set a budget alert and look at per-request usage before you cut anything. Without measurement you are guessing where the money goes.

Common mistakes

• Something that changes every call at the top of the prompt, such as a timestamp in the system prompt. The cache never hits
• A prompt below the minimum. Sonnet 5.5 and Haiku 5.5 cache prompts of 512 tokens or more; anything shorter is silently not cached, no error
• Editing tool definitions often. A change to tools invalidates the cached system and messages after it
• Comparing prices across generations with old token counts. Claude 4.7 and later use a newer tokenizer that produces roughly 30% more tokens for the same text. Re-measure on the model you will use
• Haiku 5.5 with prompts over 100K tokens. The price jumps to $0.50 / $2.50 per MTok, so long pasted documents change the math
• A 1-hour cache under steady traffic. The write costs 2x; it only pays when gaps between requests often exceed 5 minutes

Sources

• Pricing — Claude API Docs
• Prompt caching — Claude API Docs
• Batch processing — Claude API Docs
• Pricing — OpenAI API Docs
• Prompt caching — OpenAI API Docs