What is a context window?

A context window is the maximum number of tokens a language model can handle in one request, counting everything it reads (instructions, conversation history, documents and tool definitions) plus the tokens it writes. A request that does not fit has to be shortened or split.

Concepts

What counts toward the context window

The context window is measured in tokens. Everything the model reads in a single request counts toward it:

  • the system prompt and any instructions;
  • every earlier turn of the conversation that you send again;
  • tool definitions and tool results;
  • documents and retrieved passages you include;
  • images, audio and video, which models that accept them convert to tokens as well.

The tokens the model generates share the same window, including the reasoning tokens of a reasoning model. Each chat request carries the conversation so far in messages, so a long conversation sends its whole history on every turn and moves closer to the limit as it grows. For example, 20 turns of about 500 tokens each add roughly 10,000 tokens of history to the next request, before the new message and the answer.

Context window and maximum output

A model has two separate limits. The context window covers input and output together. The maximum output is the most tokens the model can write in one response, and it is usually smaller than the window. GET /v1/models reports both, as context_window (also context_length) and max_output_tokens.

You can set your own output limit for each request. max_tokens limits the tokens generated, and max_completion_tokens limits reasoning and answer together; when you send both, the smaller one applies. A response that reaches the limit stops early with finish_reason set to length, so raise the limit if answers end mid-sentence. On /v1/responses the same limit is max_output_tokens, and a response cut off by it ends with the status incomplete.

When a request is too long

When the input, together with max_tokens if you set it, is longer than the model's context window, EmpirioLabs AI returns 400 with the code context_length_exceeded before any generation starts, and the request is not billed. Shorten the input or lower max_tokens, then send it again. Agent frameworks that compact conversations use this code as their signal to do so. The errors reference shows the full response.

To check a request before you send it, POST /v1/messages/count_tokens returns an estimate of its input tokens and is not billed. The usage object in each response then reports the exact count.

Working within the window

  • Send what the task needs. Retrieve the relevant passages with embeddings instead of pasting whole documents.
  • Trim old turns. Summarize or drop early conversation turns once they stop mattering to the task.
  • Keep the stable part first. Instructions and tool definitions that never change belong at the start, where prompt caching can bill them at a lower rate on models that support it.
  • Size max_tokens to the task. The length check counts max_tokens together with the input, so an oversized value can reject a request whose input would otherwise fit.
  • Watch long-context pricing. You pay for every input token on every request, and some models charge a higher rate once the input passes a set number of tokens.

A larger window is not always better. A model can overlook details in a very long input, so a short, relevant context often gives more accurate answers than a long one. Compare context windows across text models, or test long prompts in the Playground.

Context windows on EmpirioLabs AI, largest first

96 models list a context window. Each model's page has its full specs and limits.

ModelContext windowMax output
Step 5 Preview StepFun1.02M tokens128K tokens
Qwen3.8 Omni Flash Alibaba Cloud1.01M tokens128K tokens
Aplomb 1 EmpirioLabs1M tokens-
DeepSeek V4 Flash 0731 DeepSeek1M tokens384K tokens
DeepSeek V4 Pro DeepSeek1M tokens384K tokens
DeepSeek V4 Pro 0813 DeepSeek1M tokens384K tokens
DeepSeek V4.1 Flash 1017 DeepSeek1M tokens384K tokens
Fugu Max Sakana AI1M tokens128K tokens
Show all 96 models
ModelContext windowMax output
Fugu Ultra v1.0 Sakana AI1M tokens128K tokens
Fugu Ultra v1.1 Sakana AI1M tokens128K tokens
Fugu Ultra v2.0 Sakana AI1M tokens128K tokens
GLM 5.2 Z.ai1M tokens128K tokens
GLM 5.3 Z.ai1M tokens128K tokens
GLM 5.3 Flash Z.ai1M tokens128K tokens
Kimi K3 Moonshot AI1M tokens128K tokens
Kimi K3.1 Pro Moonshot AI1M tokens128K tokens
MiMo V2.6 Flash Xiaomi1M tokens128K tokens
MiMo V2.6 Pro Xiaomi1M tokens128K tokens
MiMo V2.6 Pro UltraSpeed Xiaomi1M tokens128K tokens
MiniMax M3 MiniMax1M tokens512K tokens
MiniMax M3.1 Flash MiniMax1M tokens512K tokens
Muse Spark 1.1 Meta AI1M tokens128K tokens
Muse Spark 1.2 Meta AI1M tokens128K tokens
Muse Spark 1.3 Meta AI1M tokens128K tokens
Nova Lite 2 Amazon1M tokens128K tokens
Qwen3.5 Flash Alibaba Cloud995.9K tokens32K tokens
Qwen3.5 Plus Alibaba Cloud983.6K tokens64K tokens
Qwen3.6 Flash Alibaba Cloud983.6K tokens64K tokens
Qwen3.6 Plus Alibaba Cloud983.6K tokens64K tokens
Qwen3.7 Flash Alibaba Cloud983.6K tokens64K tokens
Qwen3.7 Max Alibaba Cloud983.6K tokens64K tokens
Qwen3.7 Plus Alibaba Cloud983.6K tokens64K tokens
Qwen3.8 Flash Alibaba Cloud983.6K tokens128K tokens
Qwen3.8 Max Alibaba Cloud983.6K tokens128K tokens
Qwen3.8 Max 0902 Alibaba Cloud983.6K tokens128K tokens
Qwen4 Max Alibaba Cloud983.6K tokens256K tokens
Perplexity Deep Research Perplexity919.3K tokens-
Perplexity Sonar Perplexity919.3K tokens-
Perplexity Sonar Pro Perplexity919.3K tokens32K tokens
Perplexity Sonar Reasoning Pro Perplexity919.3K tokens128K tokens
Nova Lite 1.0 Amazon317.4K tokens10K tokens
Nova Pro 1.0 Amazon317.3K tokens10K tokens
Qwen3.5 122B-A10B Alibaba Cloud258K tokens64K tokens
Qwen3.5 27B Alibaba Cloud258K tokens64K tokens
Qwen3.5 35B-A3B Alibaba Cloud258K tokens64K tokens
Qwen3.5 397B-A17B Alibaba Cloud258K tokens64K tokens
Gemma 4 26B-A4B Google256K tokens32K tokens
Kimi K2.6 Moonshot AI256K tokens16K tokens
Kimi K2.7 Code Moonshot AI256K tokens128K tokens
Kimi K2.7 Code Highspeed Moonshot AI256K tokens128K tokens
Mistral Small 4 Mistral AI256K tokens64K tokens
Qwen3.5 4B Alibaba Cloud256K tokens32K tokens
Qwen3.5 9B Alibaba Cloud256K tokens32K tokens
Qwen3.8 27B Alibaba Cloud256K tokens32K tokens
Step 3.5 Flash StepFun256K tokens128K tokens
Step 3.5 Flash 2603 StepFun256K tokens128K tokens
Step 3.7 Flash StepFun256K tokens128K tokens
StepAudio 2.5 Chat StepFun256K tokens128K tokens
StepAudio 2.5 Realtime StepFun256K tokens128K tokens
StepAudio 3 Chat StepFun256K tokens128K tokens
StepAudio 3 Realtime StepFun256K tokens128K tokens
Qwen Audio 3.1 Realtime Flash Alibaba Cloud240K tokens16K tokens
Qwen Audio 3.1 Realtime Plus Alibaba Cloud240K tokens16K tokens
Seed 2.0 Mini ByteDance229.2K tokens128K tokens
Seed 2.0 Code ByteDance229.1K tokens128K tokens
Seed 2.0 Lite ByteDance229.1K tokens128K tokens
Seed 2.0 Pro ByteDance229.1K tokens128K tokens
Seed 2.1 Turbo ByteDance229.1K tokens64K tokens
Qwen3.6 27B Alibaba Cloud224K tokens64K tokens
GLM 4.7 Flash Z.ai202.8K tokens128K tokens
GLM 4.7 FlashX Z.ai202.8K tokens128K tokens
GLM 5.1 Z.ai202.7K tokens128K tokens
Qwen3.5 Omni Flash Alibaba Cloud192K tokens32K tokens
Qwen3.5 Omni Flash Realtime Alibaba Cloud192K tokens64K tokens
Qwen3.5 Omni Plus Alibaba Cloud192K tokens32K tokens
Qwen3.5 Omni Plus Realtime Alibaba Cloud192K tokens64K tokens
Qwen3.8 Omni Flash Realtime Alibaba Cloud192K tokens64K tokens
DeepReasoning WinFunc160K tokens32K tokens
Nova Micro 1.0 Amazon133K tokens10K tokens
GLM 4.5 Flash Z.ai128K tokens96K tokens
GLM 4.6V Flash Z.ai128K tokens32K tokens
GLM 4.6V FlashX Z.ai128K tokens32K tokens
Gemini 3.8 Live Google128K tokens64K tokens
Gemini 3.8 Live Extended Thinking Google128K tokens64K tokens
Gemma 3 27B Google128K tokens16K tokens
Muse Glimmer 30B Meta AI128K tokens32K tokens
Qwen3.6 35B A3B Alibaba Cloud128K tokens16K tokens
Mistral Small 3.1 Mistral AI89.6K tokens32K tokens
Qwen3.8 LiveTranslate Flash Realtime Alibaba Cloud48K tokens4K tokens
Qwen Audio 3.0 Realtime Flash Alibaba Cloud16K tokens8K tokens
Qwen Audio 3.0 Realtime Plus Alibaba Cloud16K tokens8K tokens
Skylark Embedding Vision ByteDance8K tokens-
Text Embedding v4 Alibaba Cloud8K tokens-
Qwen3 Rerank Alibaba Cloud4K tokens-
Tongyi Embedding Vision Flash Alibaba Cloud1K tokens-
Tongyi Embedding Vision Plus Alibaba Cloud1K tokens-

Frequently asked questions

Does the response count toward the context window?

Yes. Input and output share the window, so a long prompt leaves less room for the answer. The API checks the input plus max_tokens, when you set it, against the window before it runs the request.

What happens if my prompt is longer than the context window?

The API returns a 400 error with the code context_length_exceeded before any generation, and the request is not billed. Shorten the input or lower max_tokens, then send it again.

Is the context window the same as the maximum output?

No. The context window covers input and output together, while the maximum output limits how many tokens the model writes in one response. Both are listed for each model.

How do I count tokens before sending a request?

Send the request body to POST /v1/messages/count_tokens for an estimate of its input tokens, which is not billed. The usage object in each response reports the exact count.

Ready to use better endpoints?

Check out our pricing or reach out if you want your own model deployed on our stack.