Context Window

What is a Context Window?

The Context Window is the maximum number of tokens that a language model can process during a single computation. The prompt, system prompt, and previous conversation all count toward this limit. Anything beyond it does not exist for the model.

How it works

Every model has a fixed upper limit—typically ranging from about 4,000 to well over 100,000 tokens, depending on the model. Many chat systems send the entire conversation history back to the model with every new message. Once the conversation reaches the limit, the oldest messages are removed before the new request is processed.

The model does not gradually or partially “forget” this information. Once it falls outside the context window, it disappears completely and without transition—the model is not aware that anything is missing.

Why the Context Window cannot simply grow indefinitely

Self-Attention compares every token in the context window with every other token. If the size of the context window doubles, the computational effort for these pairwise comparisons increases by a factor of four. The relationship is therefore quadratic rather than linear. A larger context window is not merely a configuration setting—it is directly tied to the available computing power.

Why the Context Window has practical implications

• Long documents: If the context window is too small, an entire document cannot be processed at once.
• Conversation history: In long chats, the beginning of the conversation will eventually be pushed out of the context window.
• Prompt budget: A long prompt consumes tokens that are then no longer available for the model’s response.

Typical example

A context window of 4,000 tokens corresponds—depending on the language and tokenizer—to roughly several thousand words. If a prompt already uses 3,000 tokens, only about 1,000 tokens remain available for the response.

Further reading

Fundamentals

→ How a Language Model Works
→ Prompts: Why Wording Changes the Answer

Related terms

→ Token
→ Tokenization