What is a context window? An AI model’s working memory, explained

What is inside the window

Think of the context window as everything the model can “see” for one request, laid out as a single sequence of tokens — the small chunks of text models actually read and write, covered in more depth in our tokens explainer. That sequence typically includes your current prompt, any earlier turns in the same conversation the application chooses to resend, any documents or instructions pasted in, and the model’s own reply as it is being generated. The window is shared by all of that: a longer input leaves less room for a longer answer, and vice versa.

It helps to picture it less like long-term memory and more like a desk. Whatever is currently spread out on the desk, the model can work with directly. Whatever is filed away elsewhere — a past conversation that was not resent, a document that was never pasted in — is simply not on the desk, however well the model might otherwise “know” about it from training.

Why the size matters

A larger context window lets a model work with more material in one go: a longer document, a fuller conversation history, more example data before it answers. That matters for tasks like summarising a long report, reviewing a sizeable codebase, or holding a genuinely long back-and-forth conversation without the model losing track of something said earlier.

The practical detail worth knowing is that context windows differ by model, and they have generally grown larger over successive model releases as the underlying techniques have improved. Rather than anchoring on a specific figure for any one model — which will be out of date quickly — treat “how big is the context window” as a question to check against the provider’s current documentation for the model you are actually using.

What happens when you exceed it

Every model has a hard limit, and different systems handle going past it differently. Some interfaces truncate automatically, quietly dropping the oldest part of a conversation to make room for the new input — which can mean the model “forgets” something you told it earlier without any visible warning. Other interfaces simply return an error and refuse the request until you shorten it. Either way, the model itself never becomes aware that something was cut; it only ever reasons over whatever ended up inside the window.

Working within the limit

A few techniques come up repeatedly wherever the material genuinely will not fit:

• Summarising — condensing earlier parts of a conversation or document into a shorter version that preserves the parts still needed, freeing up room for new content.
• Chunking — splitting a long document into pieces and processing each separately, then combining the results, rather than trying to fit the whole thing in at once.
• Retrieval-augmented generation (RAG) — instead of stuffing an entire knowledge base into the window, a system searches it first and pulls in only the passages relevant to the current question, described fully in our RAG explainer.
• Trimming conversation history — an application can choose to resend only the most recent or most relevant turns of a long conversation rather than the entire transcript.

Context window versus a model’s knowledge

It is worth separating the context window from what a model “knows” from training. Training gives a model general knowledge baked in ahead of time, fixed until it is retrained; the context window is the separate, temporary space for whatever you actually supply at request time. A model can know a great deal in general and still be unable to answer accurately about a document you never put in front of it, simply because that document was never part of the request. This is also part of why prompt engineering — covered in our companion article — cares so much about what you choose to include and in what order, given that space is finite.

Context windows in certification study

Context windows come up as a foundational concept wherever an AI certification covers how large language models actually work, including the AWS Certified AI Practitioner (AIF-C01) and Google Cloud’s Generative AI Leader certification. Both expect you to understand what a context window is and why it constrains what a model can use, without expecting you to memorise specific size figures for any product — a good match for how quickly those figures move in practice. Our /revision library covers the fuller AI-fundamentals syllabus lesson by lesson.