Context window
Last verified
The context window is the maximum number of tokens a model can hold in a single request: system prompt, conversation history, retrieved documents, tool definitions, tool results, and the response it generates.
Exceed it and the request fails, or older content is dropped, depending on how your client handles it.
A large context window is a capability, not a licence. Filling a 200k-token window on every request means paying for 200k input tokens on every request. The common mistake is treating the window as a budget to use rather than a ceiling not to hit.
Different providers price long context differently, and some charge a premium above a threshold.