Where memory comes from
A colleague opens a new chat, and the assistant greets them by name and asks about the report they worked on last week. “It remembered,” they say. An earlier lesson showed that a model has no memory of its own: the weights don’t change when you type, and the context window is the only place a fact can be for the model. So something else happened here. A fact was written down somewhere outside the window and put back in when the new chat started. In this lesson we take the phrase “it remembered” apart. Each time, the question is what was stored and what was read back. The other end of the same subject is a long session that holds too much, where the answers start to drift. You read a long transcript and mark where the drift begins, then name the cure that would have helped. You install and run nothing. By the end you can say where a memory is stored and read back, recognize context rot in a session of your own, and pick one of three ways out.
Two places a fact can be
Section titled “Two places a fact can be”For a model, a fact is in one of two places. Either it is in the context window right now, or it doesn’t exist for this response. That rule came from the lesson on the context window. Memory doesn’t change the rule. It adds a second place where a fact can wait, and a step that moves the fact back into the window.
Short- and long-term memory are the names for the two places [1]. Short-term memory is the context window itself: everything said and seen in this session, the instructions, your messages, the answers, the pasted documents, and the tool results. It grows with every turn and is gone when the session ends. Long-term memory is anything written down outside the window and read back later. A notes file the assistant keeps, a summary of last week’s chat, a database the agent searches, or a note you keep and paste in again at the start of each chat all count. Each of these is text in a place the model can’t see until something reads it and puts it into the window.
The word “remembered” hides that step. When an assistant greets your colleague by name, the name was in some earlier session, a product feature wrote it to storage, and at the start of the new session the product read the storage and placed the name in the window before the first message. The model in the new session saw the name the way it sees any other text in the window. A fresh session doesn’t remember the last one [2], and nothing about the model changed between the two. That’s also why a colleague with the memory feature switched off gets no greeting: the same model, and an empty place where the note would have been.
Where is this fact?
Section titled “Where is this fact?”The lesson names two places a fact can be for a model: in the context window for this response (short-term memory), or written down outside the window and read back later (long-term memory).
For each claim, ask whether the fact was typed, pasted or produced in this session, or whether something had to read it from storage and put it in the window.
Where products keep the notes
Section titled “Where products keep the notes”Products give long-term memory different names, and they all come down to text in storage, read back at a known moment. Two examples show the pattern.
A coding agent such as Claude Code reads instruction files at the start of every session. A project can hold a file that the team writes, with build commands and conventions, and the agent also keeps notes of its own, written during earlier sessions from your corrections and preferences. Both are loaded into the context window at the start of each conversation. Of the notes the agent writes itself, only the first part of their index file is loaded. Those notes are on by default [3]. The agent treats them as context, the same as a message you type, so an instruction in such a file is a strong default and never a guarantee. The lesson on what tokens cost showed what happens to rule seven in a long instruction file. Memory files are instruction files, and the same limit applies.
Chat assistants offer memory features that save notes about you and your work across chats. In the Claude chat assistant, memory saves notes on topics such as your role and your projects while you chat. It is on by default on the Free, Pro, and Max plans. On Team and Enterprise plans it is off until an owner turns it on. The memory settings list every saved topic, and you can read, change, or delete each one [4]. Another product can save other things and have another default, so read the settings page of the product you use before you rely on the feature. In every case, the model in the new chat sees whatever text the feature put back in the window. The rest of the old chat isn’t in it.
The third kind of long-term memory is the one from the previous lesson. Retrieval is memory that gets searched instead of loaded whole. The documents wait outside the window, a search picks the passages that fit the question, and only those are pasted in. Retrieval and memory differ in what they hold, notes about you against shared documents [1]. They share the mechanism: storage outside the window, and a step that reads part of it back in.
How did it know?
Section titled “How did it know?”A chat assistant with a memory feature opens a new chat and mentions a project the user discussed in an earlier chat. The lesson explains that a model has no memory of its own and that a fact is either in the context window or does not exist for the model.
An assistant with a memory feature opens a new chat and asks how the “Q3 budget review” is going. You discussed that review in a chat last month. What happened?
What is the only place a fact can be when the model produces an answer, and how did the project name get there?
When the window holds too much
Section titled “When the window holds too much”Long-term memory covers one end of the problem, where the fact isn’t in the window. The other end is a window that holds too much. As a session goes on, the window fills with old turns, tool output, pasted documents and instructions that no longer apply. Nothing has fallen out yet. The facts are all there, and the answers still get worse. The model follows early instructions less reliably, loses a detail among the text around it, contradicts something it said an hour ago, or asks for a fact you gave it already. This decline is called context rot [1].
The lesson on what tokens cost covered instruction dilution, where one rule competes with the text after it in one prompt or one file. Context rot is the session-level version. The competition is between everything in the window, and it grows with every turn, not only with the length of one file. A tool that returned a large result you never looked at takes up as much of the window as a page you wrote with care. Liu and colleagues found that models use a fact in the middle of a long input less reliably than one at the start or the end [5], and a long session puts most of the window in the middle. So a long session fails a little at a time. It drifts.
The signs are easy to name once you know them [1]. The assistant forgets a thing you said earlier in the same session. It contradicts an earlier decision. It ignores a tool result from the start of the session. It asks for information you already gave. Each of these, in a session that has run for a while, is the cue to look at how full the window is, before you look for anything else.
Summarize, start fresh, or retrieve
Section titled “Summarize, start fresh, or retrieve”The cure for context rot is to keep the window short and deliberate. The source course on agent memory lists more ways to manage a full window, for people who build agents [1]. This lesson keeps the three a knowledge worker can apply without tooling: summarize, start fresh, and retrieve. Each one moves text out of the window or keeps it from getting in.
Summarize. Replace the old turns with a short summary of what matters, and go on from the summary. Products do this for you under names such as compaction. Claude Code, for example, runs a compaction automatically as a session approaches the window’s limit, and replaces the conversation with a structured summary that keeps the instruction files and the memory notes and drops the detail of the old turns [6]. A summary keeps the decisions and loses the nuance, and the model that writes the summary can miss something that mattered. So a summary is the cure for a long session whose history matters, and it works best when you tell it what to keep.
Start fresh. Open a new session for the new piece of work, and bring only the facts it needs, in one message you can paste again. You skip the summary step and lose nothing to it, because you chose what to carry over. This is the cure when the old session is about something else, or when a summary would be as long as the facts you need.
Retrieve. Keep the documents outside the window and let a search bring in only the passage the question needs. This is the retrieval from the previous lesson, used as a cure. It is the right one when the problem is a large document that was pasted whole and is needed one paragraph at a time. The search becomes the weak point, as that lesson showed, so it trades a full window for a search you have to check.
The afternoon session drifts
Section titled “The afternoon session drifts”A user has worked with an assistant for a whole afternoon in one chat on a policy document. They pasted the full policy at the start and several long meeting notes since. The assistant has started to describe the policy wrongly and to ask for facts the user already gave.
The assistant that summarized the policy well at two o’clock now says the policy allows something it forbids, and asks you which team the document is for. You said so in your first message. What do you do?
What has changed about the window since the session started, and which cure moves the least useful text out of it?
Which of these point at context rot?
Section titled “Which of these point at context rot?”The lesson names context rot as the decline in a model's answers as the context window fills with old turns, tool output and stale instructions during one long session, and lists its signs.
Which two of these are signs of context rot rather than something else?
Context rot is about the window in one session. Which of these are about what is in the window, and which are about something else?
Exercise
Below is a shortened transcript of a long session with an assistant. The transcript is written for this lesson, and the assistant’s lines are illustrative rather than recorded. The user is drafting an internal notice about a new expense process. Only the user’s messages and the first line of each answer are shown, and the turn numbers show how far the session has come. Read it once from the top. Then, on paper or in a note, write down the number of the first turn where the answers start to drift, the sign of context rot you see there, and which of summarize, start fresh or retrieve would have helped most, with one line on why. Ten minutes is enough.
turn 1 User: Draft a notice for all staff about the new expense process. Audience is every employee, most of them not in finance. Plain language, one page at most, and every claim now needs a receipt photo in the app.turn 1 Assistant: Here is a one-page draft in plain language ...turn 2 User: Good. Here is the full 14-page finance procedure for context. [pasted]turn 2 Assistant: Thanks, I have read the procedure. The draft still matches it ...turn 5 User: Add a line on when claims are paid out.turn 5 Assistant: Added: claims are paid with the next salary run ...turn 9 User: Here are the notes of the finance meeting where this was decided. [pasted]turn 9 Assistant: Noted. The meeting confirms the receipt rule ...turn 14 User: Can you tighten the second paragraph?turn 14 Assistant: Here is a tighter second paragraph ...turn 18 User: Here are the notes from the IT meeting on the app rollout. [pasted]turn 18 Assistant: Thanks. The app rollout dates are in these notes ...turn 23 User: Update the notice with the rollout date.turn 23 Assistant: Updated. Note that the procedure requires claims in the format set out in section 7.3, with cost center codes and approver initials ...turn 24 User: That's for the finance team. The notice is for everyone, in plain language.turn 24 Assistant: Apologies. Here is the notice again, now at two pages, covering the approval matrix from the procedure as well ...turn 25 User: One page. Plain language. Every employee.turn 25 Assistant: Understood. Which team is the notice for, and is there a length limit?A good result names turn 23 as the first drift, because the answer follows the pasted procedure’s language for finance staff instead of the audience and the one-page limit set at turn 1, and turn 25 as the clearest sign, because the assistant asks for two facts the user gave in the first message. Any of the three cures is defensible with a reason. Summarize fits if the decisions of turns 5 through 23 matter and the user wants to go on in the same product. Start fresh fits well here, because a new chat needs only the turn 1 brief and the finished draft, and leaves the 14-page procedure and two sets of notes behind. Retrieve fits if the procedure gets asked about again and again, so that a search brings in one section at a time instead of all fourteen pages. If you were the user, which turn would you have stopped at, and what would you have carried into the new chat?
Stretch: Take a long chat of your own, one that ran for an hour or more, and read it from the top. Mark the first answer that ignored something you had said. Which of the three cures would you apply, and at which message?
Recap
- A model has no memory of its own. Short-term memory is the context window for this session, and long-term memory is anything written outside the window and read back in. “It remembered” always means something was stored and re-read [1].
- Products keep long-term memory as text in storage, loaded at a known moment: instruction files and the agent’s own notes at the start of a coding session [3], a chat product’s memory notes [4], or documents that a search brings in one passage at a time.
- Context rot is the decline in a long session as the window fills with old turns, tool output, and stale instructions. The facts are still there, and the answers drift, contradict, and ask for what you already gave [1]. A fact in the middle of a long context is used less reliably than one at the edges [5].
- The cures are summarize, start fresh, and retrieve. Each moves text out of the window or keeps it from getting in, and the sign that you need one is an answer that ignores something you said in the same session.
You can now
- Explains what grounding and retrieval add and what they do not fix
- Explains tokens, context and sampling in plain words
References
Section titled “References”- Addy Osmani, Ivar Soares Urdalen, Leo Simons. Memory and context: context engineering, memory kinds, memory versus RAG, context rot. Agent Engineer Course. Course.
AEC-05 - Anthropic. AI capabilities and limitations. Claude Academy. Course.
Academy ai-capabilities-and-limitations - Anthropic. How Claude remembers your project. Claude Code documentation. Reference.
Claude Code memory - Anthropic. Use Claude's chat search and memory to build on previous context. Claude Help Center. Reference.
Claude support memory - Nelson F. Liu, Kevin Lin, John Hewitt and 4 others. Lost in the Middle: How Language Models Use Long Contexts. Transactions of the Association for Computational Linguistics 12 (2024), 157-173. Paper.
Liu 2024 - Anthropic. Explore the context window. Claude Code documentation. Reference.
Claude Code context window