Almost a million tokens for nothing
A whole night of work with AI agents and almost a million tokens. What they did stayed, while the why was lost. Knowledge can be rebuilt, but at what cost?
New to this topic? Start with World 1 of AI.
Part 2 of 6 of the series A different 7–1 every day.

Décio Dalke Jr. — Arquiteto de negócios. Sócio-gerente no ecossistema MitUP (mitup.pt).
A different 7–1 every day. New here? Start with the introduction.
A whole night of work with AI agents. By morning, the work was done, yet it was the agent that no longer knew what it had done.
What happened that night?
The task was investigative: data-entry and read tests on a legacy system, to understand how it stores its data. Slow work: try, observe, take notes. Each test led to a conclusion, and each conclusion decided the next test. The value was in the thread linking one test to the next.
At some point during the night, the conversation grew too long. AI tools have a limit on how much conversation they can hold at once (the so-called context window). When they get close to it, they summarise what came before on their own and carry on. Nobody asks. The work doesn’t stop.
After the summary, the agent carried on. Except it no longer really knew what it had done, or why. And it was worse than forgetting: it looked at an old file and concluded that its own work didn’t exist. But it was there, in the system, the next morning.
What was the score?
Almost a million tokens. For anyone who doesn’t use the jargon: a token is the unit in which an AI model’s work is measured (and paid for). In API money, a few dollars. The expensive part was the time: a whole night, more than 16 hours of dedicated agents. And redoing it will take another night.
What the agent actually did in the system stayed; what was lost, however, was the intelligence behind it: what it was doing, why it was doing it, and the analysis it had drawn from each test. None of that was recorded anywhere.
Knowledge can be rebuilt, but at a cost: the same tests, the same reasoning, the same conclusions, paid for twice.
Why wasn’t it bad luck?
The agent’s working memory is the conversation itself. Everything it knows about the task lives there, and nowhere else, unless someone has written it down outside.
When the conversation is summarised, the summary keeps whatever seems important at that moment. The reasoning behind an investigation usually looks like a rough draft, of little importance. And it’s the first thing to go.
And the meter keeps running. Although the tokens spent before the summary are still spent, what disappears is the value of what they produced.
Why did nobody see it coming?
From the outside, nothing changes. The agent keeps answering with the same confidence, in the same tone. It doesn’t say “I forgot”. It couldn’t: it doesn’t know what it has forgotten.
That makes little difference in a ten-minute conversation and all the difference over a whole night of work.
What has to exist?
The reasoning has to be recorded outside the conversation while the work is happening, because by the end there may be nothing left to record.
A spent token doesn’t come back. Neither does reasoning that was never written down.