Beyond the token wall: why AI companions get context rot
Every AI companion starts sharp and goes vague. It is not the model losing interest — it is the conversation falling out of the window it can see.
The illusion of memory
The first hour with an AI companion is uncanny. It picks up the thread, remembers the name of your sister, refers back to the argument you had two scenes ago. It feels like being known.
Then, somewhere around the thirtieth message, it starts to drift. It asks something you already answered. A character who left the room is suddenly speaking. The wound from yesterday has healed without anyone treating it.
Nothing broke. You simply reached the edge of what the model could see.
A language model has no memory between calls. Every turn is a fresh request that carries the whole conversation with it, and that request has a size limit — the context window. When the conversation grows past it, something has to go, and what goes is the oldest material. The window slides forward. Your first scenes fall off the back of it, and from the model's point of view they never happened.
What context rot actually looks like
The failure is rarely dramatic. It is a slow loss of specificity, which is why it is so easy to blame on the model being "lazy".
- Names blur. Two secondary characters merge into one, or a name is reused for someone new.
- Promises evaporate. A character who swore to meet you at the harbour has no idea what you are talking about.
- Stakes reset. The debt, the injury, the secret someone is keeping — the plot quietly relaxes back to neutral.
- Established facts flip. A character who cannot swim goes swimming.
Each one on its own is a shrug. Together they are the difference between a story and a series of pleasant, disconnected scenes — and they are why long roleplay so often peters out rather than ending.
Three ways to hold a conversation
Broadly, three approaches are in use, and they fail in different places.
| Approach | How it holds the past | Where it breaks |
|---|---|---|
| Sliding window | The last N messages, verbatim | Everything older is simply gone. Perfect recall inside the window, total amnesia outside it. |
| Very large window | Tens or hundreds of thousands of tokens, verbatim | Cost and latency scale with every turn, and attention thins out across the middle of a long context — the model can see a detail and still not weight it. |
| Vector retrieval (RAG) | Past messages embedded and searched by similarity | Returns text that sounds relevant. It has no notion of what is currently true, so a fact and its later correction look equally good. |
| Structured story state | Facts, characters, relationships and events, written down as they happen | Costs a step per turn to maintain, and it is only as good as what the extractor bothers to record. |
It forgot my name on day three. Not the character — the story.
A closed-beta player
Why a bigger window is not the answer
The obvious fix is to make the window enormous. It is also the most expensive one, and it does not do what people hope.
First, price and speed. The whole context is re-sent and re-read on every single turn. A conversation that carries a hundred thousand tokens of history pays for those tokens again with each message, and waits longer for each reply. In a story where every turn matters, that is felt.
Second, and more stubbornly: a model attending to a very long context does not attend to it evenly. Material at the beginning and end lands harder than material in the middle. So the crucial thing your character said forty scenes ago can be technically present and practically invisible.
A large window buys time. It does not change the shape of the problem, because the problem is not capacity — it is that nothing is deciding what matters.
What we do instead
projectDigo does not try to carry the transcript. It keeps a record.
After every turn, a separate pass reads what just happened and books the change: who was present, what was established, what someone now knows or wants, what was damaged, what time it is. Those entries are append-only — a later fact does not erase an earlier one, it supersedes it, so the history of a relationship stays legible rather than being overwritten.
Before the next turn is written, that record is rebuilt into a brief: the clock, the place, who is in the room, what is unresolved, what has to stay true. The narrator model is handed the story's current state as facts, not asked to infer it from a transcript that may no longer contain it.
The result is that continuity stops depending on how long the conversation is. Scene four hundred knows what scene three established, because scene three wrote it down.
The honest trade-off
This is not free. It costs an extra model call per turn, and it can only remember what the extraction step thought was worth recording — an extractor that misses a detail loses it as surely as a sliding window would.
What it buys is a story that accumulates. Ten hours in, the world has a history you can point at, and the characters behave like people who were there for it.
Questions we get
Why not just use a model with a one-million-token context window?
Because you pay for the whole context on every turn, in both money and latency, and because attention thins out across the middle of a very long context — a detail can be present and still be effectively ignored. A large window postpones context rot rather than removing it.
Is this the same as RAG?
Related but not the same. Retrieval finds passages that resemble the query, which is useful for looking things up and unreliable for knowing what is currently true: a fact and its later correction both look like good matches. A story record stores state — this is true now, this superseded that — rather than similar-sounding text.
How far back can a character actually remember?
To the start of the session. What is remembered is the recorded state — facts, relationships, events, injuries, promises — rather than the literal wording of every message, so recall does not degrade as the session lengthens.
Does the model ever contradict its own memory?
It can, and that is what the checks after generation are for: a reply that breaks an established fact is caught and rewritten rather than published. No system catches everything, but the failure mode becomes a rare correction instead of a steady drift.