Working with AI
What is Context window?
The context window is the maximum amount of text an AI model can consider at once (its working memory), covering your instructions, the conversation so far, and any documents provided, measured in tokens.
Everything the model "knows" during a request must fit in the window: the system prompt, retrieved documents, the chat history, and the answer being generated. Modern models advertise windows from 128 thousand to over a million tokens (a token is roughly three-quarters of an English word; Arabic tokenises less efficiently, which quietly raises costs for Arabic-heavy workloads). Once the conversation or document set exceeds the window, something must be dropped, summarised, or retrieved selectively.
Two practical consequences. First, bigger windows are not free: you pay per token, so stuffing a 500-page manual into every request is the expensive lazy option. Retrieval (fetching only the relevant three pages) usually beats brute force on both cost and accuracy, because models also get measurably worse at using facts buried in the middle of very long inputs. Second, the window explains a familiar failure: a long chat where the assistant "forgets" what you said an hour ago has simply scrolled that part out of memory, or its summary of it lost the detail.
This is also the honest explanation behind a very 2026 phenomenon: AI-built apps that degrade as they grow. Tools like Lovable and Cursor hold your codebase in the model’s context; past a certain size it no longer fits, the model sees fragments, and each new change breaks something it can no longer see. The ceiling is not the tool’s ambition, it is the window. (We run a whole rescue service for apps that hit it.)
Frequently asked
- What happens when you exceed the context window?
- Something has to give: the model drops, summarises, or selectively retrieves part of the input. This is why a long chat where the assistant "forgets" what you said an hour ago has simply scrolled that part out of its working memory. Everything the model considers in a request, instructions, documents, and history, has to fit in the window at once.
- Does a bigger context window mean better answers?
- Not automatically, and it is not free. You pay per token, so stuffing a 500-page manual into every request is the expensive lazy option, and models get measurably worse at using facts buried in the middle of very long inputs. Retrieving only the relevant few pages usually beats brute force on both cost and accuracy.
- Why do AI-built apps break as they get bigger?
- Tools like Lovable and Cursor hold your codebase in the model’s context window. Past a certain size it no longer fits, the model sees only fragments, and each new change breaks something it can no longer see. The ceiling is the window, not the tool’s ambition, which is why apps that grow this way tend to degrade. We run a rescue service for exactly that failure.
Wondering where this fits in your business?
The free AI audit maps your operations and shows the three systems we would build first, in plain language like this.
