Forbi tokenveggen: hvorfor AI-følgesvenner får kontekstforvitring
De fleste AI-følgesvenner starter skarpt og blir etter hvert vage. Det er ikke modellen som mister interessen – det er samtalen som faller utenfor vinduet den kan se.
The illusion of memory
The first hour with a typical AI companion is uncanny. It picks up the thread, remembers the name of your sister, refers back to the argument you had two scenes ago. It feels like being known.
Then, somewhere around the thirtieth message, it starts to drift. It asks something you already answered. A character who left the room is suddenly speaking. The wound from yesterday has healed without anyone treating it.
Nothing broke. You simply reached the edge of what the model could see.
A language model has no memory between calls. Every turn is a fresh request carrying the whole conversation with it, and that request has a size limit — the context window. Grow past it and the oldest material is dropped. Your first scenes fall off the back of it, and from the model's point of view they never happened.
What context rot actually looks like
In most companion apps the failure is rarely dramatic. It is a slow loss of specificity, which is why it is so easy to blame on the model being "lazy".
- Names blur. Two secondary characters merge into one, or a name is reused for someone new.
- Promises evaporate. A character who swore to meet you at the harbour has no idea what you are talking about.
- Stakes reset. The debt, the injury, the secret someone is keeping — the plot quietly relaxes back to neutral.
- Established facts flip. A character who cannot swim goes swimming.
Each one on its own is a shrug. Together they are the difference between a story and a series of pleasant, disconnected scenes — and they are why long roleplay so often peters out rather than ending.
Four ways to hold on to the past
Four approaches are in use, and they fail in different places.
| Approach | How it holds the past | Where it breaks |
|---|---|---|
| Sliding window | The last N messages, verbatim | Everything older is simply gone. Perfect recall inside the window, total amnesia outside it. |
| Very large window | Tens or hundreds of thousands of tokens, verbatim | Cost and latency scale with every turn, and attention thins out across the middle of a long context — the model can see a detail and still not weight it. |
| Vector retrieval (RAG) | Past messages embedded and searched by similarity | Returns text that sounds relevant. It has no notion of what is currently true, so a fact and its later correction look equally good. |
| Structured story state | Facts, characters, relationships and events, written down as they happen | Takes real work to maintain, and it is only ever as good as what gets written down — whatever is not recorded is lost. |
I wanted a world that writes things down. When she remembers your third evening together, it is because she was really there for it.
Founder, projectDigo
Why a bigger window is not the answer
The obvious fix is to make the window enormous. It is also the most expensive one, and it does not do what people hope.
The whole context is re-sent and re-read on every single turn, so a conversation carrying a hundred thousand tokens of history pays for them again with each message — and waits longer for each reply. Worse, a model reading a very long context does not read it evenly: what your character said forty scenes ago can be technically present and practically invisible.
A large window buys time. It does not change the shape of the problem, because the problem is not capacity — it is that nothing is deciding what matters.
What we do instead
projectDigo keeps a record of the story.
What matters — who exists, what has happened between them, what is true right now — is held as the story's own state rather than left in a conversation that will eventually scroll out of view. Later scenes are written against that state. Scene four hundred knows what scene three established, because scene three was recorded rather than merely said.
How that record is built, kept current and put back to work is the part we have spent the longest on — and the part we keep to ourselves. What it delivers is simple enough: continuity that holds no matter how long you play.
The honest trade-off
This is not free. Keeping a record is more work per scene than forwarding a transcript, and a story can only remember what was worth recording — anything missed is lost as surely as a sliding window would have lost it.
What it buys is a story that accumulates. Ten hours in, the world has a history you can point at, and the characters behave like people who were there for it.
Spørsmål vi ofte får
Why not just use a model with a one-million-token context window?
Because you pay for the whole context on every turn, in money and in latency, and because attention thins out across the middle of a very long context — a detail can be present and still be effectively ignored. A large window postpones context rot rather than removing it.
Is this the same as RAG?
Related, but not the same. Retrieval finds passages that resemble the query, which is useful for looking things up and unreliable for knowing what is currently true: a fact and its later correction both look like good matches. A story record holds state — what is true now — rather than similar-sounding text.
Hvor langt tilbake kan en rollefigur faktisk huske?
I projectDigo, helt tilbake til starten av økten. Det som bevares, er historiens tilstand og ikke ordlyden i hver eneste melding, så hukommelsen svekkes ikke etter hvert som økten blir lengre.
Motsier modellen noen gang sin egen hukommelse?
Det kan den – og derfor sjekker projectDigo konsistensen før en scene når frem til deg. Vanligvis blir resultatet en korrigering du aldri ser. Ingenting fanger opp alt, men feilmodusen blir et sporadisk glipp i stedet for en jevn drift.
