When you ask an AI language model a question, it can process only a limited amount of information at once. The model’s context window is the maximum amount of text, measured in tokens, that it can process for a request and its response.
A token is a small piece of text. A token may be a whole word or only part of one; it can also be punctuation or a number. Because tokens do not map neatly to words, a token limit is not a fixed word limit. The context window holds the information the model can use in the current interaction; it does not automatically give the model lasting memory.
What fits inside the window
The context window can include your message and the earlier conversation, along with instructions and text supplied from documents or other sources. The model’s reply also uses tokens. Each provider counts these parts differently and reserves a different amount of room for the answer, depending on the model and service.
For example, a request to summarize a long report uses capacity for the report, your instructions, and the summary. A request that exceeds a model or API’s limit can be rejected. In a chat product, older messages might instead be dropped or summarized, depending on how the product manages the conversation. How providers handle context limits describes these options.
When older material is summarized, details can be lost. A summary might leave out an exception or make a qualified statement sound certain. If a past decision matters, check that the relevant detail remains in the context before relying on it.
What a larger window changes
A larger context window lets a model consider more material at once, which helps with tasks that depend on several parts of a long document or earlier messages in a conversation. A smaller window can suit a short, focused question and requires less material to process. Context-window tradeoffs include processing time and cost as well as how much information fits.
A larger window does not ensure that the model uses every detail well. Important information can get buried among irrelevant text, and a larger window does not by itself improve the model’s reasoning. The model also needs to track references and remember earlier constraints as it reads later material. Long-context understanding describes this distinction between fitting text and understanding its connections.
For a long document, give the model a clear question and include the sections that matter. If the task requires the whole document, ask for evidence tied to specific passages so you can check whether the answer reflects the relevant details.
Context is different from memory
A context window holds information available during a particular request or interaction. Long-term memory is information retained beyond the current context. The distinction is similar to a whiteboard used for the current task versus a filing cabinet that stores information for later; a model does not gain the latter simply because its window is larger. A plain-language definition of context windows draws this distinction.
Some systems pull selected passages from external documents or databases into a request. Retrieval can supply relevant material without increasing the model’s built-in context limit. For sensitive documents, consider what content the service receives before uploading it.