Where an agent's memory lives

In this lesson we decide where an agent keeps what it should still know tomorrow. Retrieval as a tool the agent calls gave the agent a search over a handbook that somebody else wrote. Memory is the store the agent writes itself: facts about a user, the state of a long task, what it learned in the last session. The lesson compares five places to keep those facts and argues for the simplest one as the first version. The last section is about how a real agent platform uses memory to keep the context window small. You pick a store for four uses at the end.

The examples run against a small fixture, site/examples/building-agents/agent-memory/memory.py in the repository, with the notes file preferences.txt next to it. Copy the folder, and each step below is python3 memory.py <step>. No model is called, so the numbers are the same on every machine.

What the agent knows and what it can find again

During a response, a fact is either in the context window right now, as part of the system prompt, the messages or this session’s tool results, or it is outside the window. A fact outside comes in only when a step of the harness reads it and puts it there. The lesson on memory in the concepts course called these short-term and long-term memory, and showed that “it remembered” always means that something was stored and read back [1].

For a builder, that read-back step is a design decision. The agent can load a store whole at the start of every session, or it can look up the part it needs when it needs it. Loading whole is simple and puts each fact in the window. Looking up keeps the window small, but the agent has to know that a fact exists and ask for it, and a lookup can miss. The rest of this lesson is about which store makes that choice easy for each kind of use.

Memory and retrieval use the same mechanism. The difference is what the store holds. Retrieval reads shared documents that the agent didn’t write, and memory reads notes about one user or one task that the agent or the user wrote [1]. An agent often has both.

Where a memory can be kept

Memory storage choices are the options for where long-term memory is kept and how the agent finds a fact in it again. The course this topic comes from names vector databases, knowledge graphs, and hybrids that add a key-value store for session state and quick lookups. It also notes that a retrieval step adds latency [1]. The table below is the list this course uses, with the case each one fits.

Store	How the agent gets a fact back	Fits when
A plain file read whole	The harness loads the whole file at the start of the session	A few dozen facts that every session needs
Key-value notes	A tool reads or writes one entry by its exact key	Many small entries, each found by a name the agent already has, such as a user or task id
A searchable index over text	A tool searches by words or by meaning and returns passages	A lot of free text, asked about in words nobody can list in advance
A database behind a tool	A tool runs a query with filters, counts or sorting	Structured records that are filtered, counted or combined
A summary in the system prompt	Nothing to look up, because it is in every request	The gist of a long history that every later turn needs, where losing the detail is accepted

Each row trades something. The file is the easiest to read and fix, and it costs its full size in every request. Key-value notes are cheap to read one at a time, and useless when the agent doesn’t know the key. A similarity index can find passages in words that differ from the document, which the keyword search in the retrieval lesson couldn’t do, and it can return a passage that’s close but wrong. For precise questions over many records you need a database, and a database needs a schema and a query tool that someone maintains. The summary is short in every request, and whatever detail it dropped can’t be read back.

Checkpoint · match

Which store fits each use?

Match each memory use to the store from the table that fits it.

Fifteen formatting rules that every reply must followChoose…A plain file read wholeKey-value notesA searchable index over textA database behind a toolA summary in the system prompt

The progress note of each of 3,000 background jobs, read and updated by job idChoose…A plain file read wholeKey-value notesA searchable index over textA database behind a toolA summary in the system prompt

Four years of meeting notes, asked about by topic in the words of whoever asksChoose…A plain file read wholeKey-value notesA searchable index over textA database behind a toolA summary in the system prompt

Invoices, to answer how many unpaid invoices each customer has that are older than 90 daysChoose…A plain file read wholeKey-value notesA searchable index over textA database behind a toolA summary in the system prompt

CheckHintSkip

Start with a file a person can read

The first version of an agent’s memory is a plain text file that the harness reads whole at the start of each session. A person can open it, see what the agent believes, and delete a wrong line. When the file is in version control, each change the agent makes shows up in a diff. When the agent gets something wrong because of a note, you can find the note. With the other stores you have to build that yourself, and at the start that visibility matters more than scale.

The worked case is a preferences file for one user. It holds twenty facts, one per line, such as date format: 25 September 2026 and email drafts: leave in drafts, never send. The harness puts the file into the system prompt, and the model reads it with every request.

Example · run it

The preferences file read whole

Run python3 memory.py whole (the code is shown here) and compare what you see with the output.

lines = load()
prompt_part = read_whole(lines)
print(f"preferences.txt: {len(lines)} lines")
print(f"read whole into the prompt: {len(prompt_part)} characters")

Output

preferences.txt: 20 lines
read whole into the prompt: 509 characters

Output verified in CI from site/examples/building-agents/agent-memory/whole.py.

The 509 characters are a short paragraph, and each request includes them. A vector index over these twenty lines would need an embedding step, a store and a search tool, and the agent would have to decide to search before it knew that a preference applied. The file is always in the window, so the model never has to remember to look.

Add a search tool when volume demands it. A word search over the file, like the one below, is the simplest form of the searchable index row in the table. Say the same file has collected a line per project for months and now holds ten thousand lines. The fixture builds that grown copy in memory and compares reading it whole with a search tool that returns only the lines a question needs.

Example · run it

The grown file, whole and searched

Run python3 memory.py grown (the code is shown here) and compare what you see with the output.

lines = grown_notes(load())
prompt_part = read_whole(lines)
found = search_notes(lines, "project 0412")
print(f"grown notes: {len(lines)} lines")
print(f"read whole into the prompt: {len(prompt_part)} characters")
noun = "line" if len(found) == 1 else "lines"
print(f"search_notes('project 0412'): {len(found)} {noun}, {len(read_whole(found))} characters")
for line in found:
    print(f"  {line}")

Output

grown notes: 10000 lines
read whole into the prompt: 481545 characters
search_notes('project 0412'): 1 line, 49 characters
  project 0412: send the status update on Wednesday

Output verified in CI from site/examples/building-agents/agent-memory/grown.py.

The whole file is now 481545 characters in every request, and the one line a task about project 0412 needs is 49 of them. At that size the first thing to change is how the file is read. The agent stops loading it whole and gets a tool that returns the lines a task needs. The file stays a file, so a person can still open it. Split the twenty lines every session needs into their own small file that’s still loaded whole, and prune the project lines that no longer apply. A vector index or a database is the next step only if the search tool starts to miss.

Checkpoint · scenario

The preferences file has ten thousand lines

The agent’s preferences file started at twenty lines and is read whole at the start of each session. It now has ten thousand. What do you change first?

Move every line into a vector database first, and make each lookup a similarity search over embeddings.Keep loading it whole and move the agent to a model with a larger context window.Replace the file with a summary written by the model, and fold the summary into the system prompt.Stop loading it whole, keep the file, and give the agent a tool that returns only the lines a task needs.

DecideHintSkip

How memory keeps the context small

Memory also works in the other direction. A long agent task fills its context window with tool results, and answers get worse as the window fills. The concepts course called that decline context rot [1]. Agent platforms offer ways to move text out of the window and bring it back when it is needed, and the Claude Platform documentation describes three that work together.

The memory tool gives the model a directory of files that it can list, read, create, edit, rename and delete. The tool runs on the client side. The model sends a request such as view /memories, and code in your application performs it on whatever storage you picked, for example one folder per user or rows in a database [2]. At the start of a task the model looks in its memory folder and reads the notes there. While it works, it adds new findings to those notes, and a later conversation picks them up again. The page presents this as fetching text at the moment a task calls for it, which leaves room in the window for the task at hand [2]. It is the plain file from the last section with a tool in front of it. Among its safety advice, the page asks you to put an upper limit on the size of each memory file and on the amount of text one read hands back [2].

Context editing removes old tool results from the conversation once it passes a size you configure, and puts a short placeholder where each result was. When context editing is used with the memory tool, the model gets a warning before results are cleared, so it can write what matters to a memory file first [3].

Compaction swaps the early part of a conversation for a summary, which the model writes on the vendor’s server, and a long task then fits in the window again [4]. That’s the last row of the table, applied to the agent’s own history. The memory tool page suggests turning on compaction and the memory tool together for a long-running agent. The summary holds the size of the conversation down, and anything the agent needs after the summary goes into a memory file first [2].

Checkpoint · choice

Where does the fact go?

A long agent task is close to its next compaction. The agent has found an exact order number that it needs after the compaction. Where should that number be kept?

Leave it in the conversation, because the summary keeps every fact the model has seen.Write it to a memory file, because the files are outside the turns that the summary replaces.Repeat it in the next user message, because the most recent turn is never summarized.

CheckHintSkip

Pitfall: A vector database for twenty facts

A team builds an assistant that should remember each user’s preferences. They set up an embedding model and a vector database, and the agent searches it before every reply. Each user has twenty preferences. The search sometimes returns the wrong three, the agent ignores the date format it wasn’t shown, and when a user reports it, nobody can open the store to see what the agent believes. Twenty facts fit in a file that’s read whole. Start there, and add a search tool when a measurement shows the file is too large to load.

Exercise

Take four memory uses for an agent you might build. The agent keeps twenty user preferences and can look up ten thousand support tickets. It keeps a running task list for one long job, and it learns facts during this session. For each one, pick a store from the table in this lesson and write one line on why each of the other stores costs more than it gives. Write it in a note of about twenty lines, one for each pick and one for each store you passed over. Doing this once gives you the questions to ask before you add a database to an agent.

A good result picks by how the agent finds the fact. The preferences go in a file read whole. The tickets are free text asked about in unpredictable words, so they need a searchable index. The task list is a small file the agent reads and rewrites, or a key-value entry per step. Facts from this session are already in the context window, so the question is which of them a later session needs. Those go to a memory file, and the rest of a long history can be compacted into a summary. Each reason for passing over a store names a cost, such as a lookup the agent must remember to make or a schema someone has to maintain. Which of your four picks would change first if the volume grew by a factor of a hundred?

Stretch: Pick one of the four uses and write down the size at which you would move it to the next store in the table, and what you would measure to know you had reached it.

Recap

1. Memory is a store outside the context window and a step that reads it back. For each store, decide whether the agent loads it whole or looks up the part it needs [1].
2. The memory store follows from how the memory is used. Use a file for a few dozen facts, key-value notes for entries found by a known key, an index for free text in unknown words, a database for filters and counts, and a summary for the gist of a long history.
3. Start with a file a person can read and edit. When it grows too large to load whole, keep the file, give the agent a search tool, and split out the part every session needs.
4. Memory keeps the context small. A memory tool lets the model read files when it needs them, and compaction summarizes old turns. Write what the agent needs after the summary to a memory file first [2] [4].

You can now

• Adds retrieval or memory and knows when basic RAG is enough

Mark lesson finishedI know this, skip it

References

1. Addy Osmani, Ivar Soares Urdalen, Leo Simons. Memory and context: context engineering, memory kinds, memory versus RAG, context rot. Agent Engineer Course. Course. AEC-05
2. Anthropic. Memory tool. Claude Platform documentation. Reference. Claude docs memory-tool
3. Anthropic. Context editing. Claude Platform documentation. Reference. Claude docs context-editing
4. Anthropic. Compaction overview. Claude Platform documentation. Reference. Claude docs compaction