What is prompt caching?

Prompt caching reuses a model's earlier processing of the opening part of a prompt. When a request starts with the same tokens as a recent one, those cached tokens bill at a lower cache read rate on models that list one, and the rest bills at the normal input rate.

Concepts

What prompt caching is

Before a model writes anything, it processes every token of the input. Many applications send the same opening tokens on every request: a long system prompt, a set of tool definitions, a reference document, or the earlier turns of a conversation.

Prompt caching keeps the processed result of that opening part, called the prefix. When a later request begins with exactly the same tokens, the model reuses the cached prefix and processes only the new tokens that follow it. The output is generated in the usual way, so caching changes how the input is billed, not what the model writes.

Consider an assistant whose system prompt and tool definitions come to 4,000 tokens, followed by a 150-token question. When that opening part is read from the cache, about 4,000 of the 4,150 input tokens bill at the cache read rate, and only the question bills at the full input rate.

Why the prefix must match exactly

A cache match starts at the first token and stops at the first difference. A single changed character near the start of a prompt, such as a timestamp, a user name or a request ID in the system prompt, means nothing after it can be read from the cache. To raise your cache hit rate:

  • Put stable content first. Instructions, tool definitions, examples and long reference documents go at the start. The user's question, retrieved passages and anything else that changes go at the end.
  • Keep the stable part identical. Use the same order for tools and examples, the same wording and the same whitespace on every request.
  • Append instead of editing. In a conversation, add each new turn at the end and leave earlier turns unchanged, so every request extends the previous one.

A matching prefix makes a cache read likely, not certain, so measure your hit rate rather than assume it.

How cached input is billed on EmpirioLabs AI

Caching is automatic. There is no cache to create and no marker to add: on models that support it, repeated prefixes are cached on their own.

  • Which models. A model supports prompt caching when it lists a cache read price. The models listed on this page have one.
  • Billing. Cached input tokens bill at the cache read rate, which is lower than the model's normal input rate. Uncached input and all output tokens bill at the usual rates.
  • In the catalog API. GET /v1/models reports the rate as input_cache_read in pricing. On a model with no cache discount, it equals the normal input rate.
  • In each response. When caching applies, usage reports the cached tokens, for example cache_read_input_tokens on /v1/messages and input_tokens_details.cached_tokens on /v1/responses.
  • Over time. The Account Usage API returns cache_hit_rate, the share of input tokens served from cache in a time window, across all models or for one model.

Cached tokens are still input, so they count toward the context window like any other token.

Where caching helps most

  • Chat applications and agents that resend a growing conversation on every turn.
  • Many questions asked against the same long document or codebase.
  • Retrieval-augmented generation with a fixed block of instructions and examples.
  • Pipelines that send many requests with one shared system prompt.

Caching does little when every prompt differs from its first token. For the cost side, see how tokens are billed and the current cache read rates on the pricing page.

Models with prompt caching on EmpirioLabs AI

36 models list a cached input rate beside their input rate, read live from each rate card.

ModelInputCached input
GLM 4.5 Flash Z.aiFreeFree
GLM 4.6V Flash Z.aiFreeFree
GLM 4.7 Flash Z.aiFreeFree
Qwen3.7 Flash Alibaba Cloud<=32K $0.03 32K-256K $0.10 256K-1M $0.20<=32K $0.006 32K-256K $0.02 256K-1M $0.04
GLM 4.6V FlashX Z.ai$0.04$0.036
Nova Micro 1.0 Amazon$0.040$0.0224
Qwen3.5 4B Alibaba Cloud$0.04$0.02
Gemma 4 26B-A4B Google$0.15$0.05$0.15$0.025
Show all 36 models
ModelInputCached input
Nova Lite 1.0 Amazon$0.069$0.0386
GLM 4.7 FlashX Z.ai$0.07$0.063
Qwen3.6 35B A3B Alibaba Cloud$0.248$0.07$0.035
Qwen3.5 9B Alibaba Cloud$0.10$0.09$0.045
Step 3.5 Flash StepFun$0.10$0.02
Step 3.5 Flash 2603 StepFun$0.10$0.02
MiniMax M3.1 Flash MiniMax$0.15$0.1125$0.015$0.01125
Qwen3.8 27B Alibaba Cloud$0.45$0.17$0.08
DeepSeek V4.1 Flash 1017 DeepSeek$0.20$0.02
Muse Glimmer 30B Meta AI$0.35$0.20$0.05
Step 3.7 Flash StepFun$0.20$0.04
MiniMax M3 MiniMax$0.30 <=512K $0.225 $0.60 >512K $0.45$0.06 <=512K $0.045 $0.12 >512K $0.09
Nova Lite 2 Amazon$0.38$0.2128
GLM 5.1 Z.ai$1.40 <=32K $0.825 $1.40 32K-200K $1.10$0.26 <=32K $0.165 $0.26 32K-200K $0.22
Kimi K2.6 Moonshot AI$0.95$0.8939$0.1788
Step 5 Preview StepFun$1.00$0.10
Muse Spark 1.1 Meta AI$1.25$1.00
Muse Spark 1.2 Meta AI$1.25$1.00
Muse Spark 1.3 Meta AI$1.25$1.00
Qwen4 Max Alibaba Cloud$1.50$0.075
StepAudio 2.5 Chat StepFun$1.50$0.30
StepAudio 2.5 Realtime StepFun$1.50$0.30
StepAudio 3 Chat StepFun$1.50$0.30
Fugu Max Sakana AI$2.00$0.25
Kimi K3.1 Pro Moonshot AI$2.00$0.184
Fugu Ultra v1.1 Sakana AI<=272K $5.00 >272K $10.00<=272K $0.50 >272K $1.00
Fugu Ultra v2.0 Sakana AI<=272K $5.00 >272K $10.00<=272K $0.50 >272K $1.00
Fugu Ultra v1.0 Sakana AI<=272K $7.50 >272K $15.00<=272K $1.50 >272K $3.00

Frequently asked questions

Do I need to turn on prompt caching?

No. On models that support it, caching applies automatically to repeated prompt prefixes. Put the part of your prompt that does not change at the start to get the most from it.

How much does cached input cost?

Cached input tokens bill at the model's cache read rate, which is lower than its normal input rate. Each model that supports caching lists its cache read price.

Does prompt caching change the model's answer?

No. Caching affects how the input is processed and billed. The model generates its output in the usual way.

How can I check whether my requests use the cache?

The usage object in each response reports cached input tokens when caching applies. The Account Usage API also returns a cache hit rate for any time window, across all models or for one.

Ready to use better endpoints?

Check out our pricing or reach out if you want your own model deployed on our stack.