Agentic retrieval and memory
Building agents · topic building-agents/retrieval-memory
Retrieval and memory are how an agent knows things outside its context window. This topic covers retrieval as a loop the agent drives rather than a single lookup, the choices for where and how memory is stored, and the many cases where a plain retrieval step is all a system needs.
Concepts
- Agentic RAG loop
- Retrieval driven by the agent: it forms a query, reads what comes back, judges whether it answers the question, and reformulates, searches elsewhere or drills into a document before answering. Compared with one fixed search per question, it handles vague queries and multi-hop questions much better, at the cost of more calls and a loop that needs a bound. glossary
- Memory storage choices
- Where an agent's long-term memory lives and how it is found again: plain files the agent reads whole, key-value notes, a searchable index over text, a database queried by tool, or summaries folded into the system prompt. Each trades simplicity against scale and precision. Start with files the human can read and edit; add indexing when volume demands it. glossary
- When basic RAG suffices
- Many systems need only the simple form: embed documents, retrieve the top matches for the question, put them in the prompt, answer. It is enough when questions are direct, the corpus is well chunked and latency matters. Reach for agentic retrieval only when evaluation shows failures on multi-step or ambiguous questions that a better index cannot fix. glossary
Links
- Builds on: The agent loop and harness, Grounding and memory
- Leads to: nothing yet
- Competencies drawing on it: Builds a tool-using agent loop
Lessons
- Where an agent's memory lives (explanation)
- Retrieval as a tool the agent calls (tutorial)
Your reference
Each lesson above adds its takeaways and its example here once you finish it. Your reference lists every lesson you have finished.
Where an agent's memory lives
Unlocks when you finish Where an agent's memory lives.
Takeaways
- Memory is a store outside the context window and a step that reads it back. For each store, decide whether the agent loads it whole or looks up the part it needs.
- The memory store follows from how the memory is used. Use a file for a few dozen facts, key-value notes for entries found by a known key, an index for free text in unknown words, a database for filters and counts, and a summary for the gist of a long history.
- Start with a file a person can read and edit. When it grows too large to load whole, keep the file, give the agent a search tool, and split out the part every session needs.
- Memory keeps the context small. A memory tool lets the model read files when it needs them, and compaction summarizes old turns. Write what the agent needs after the summary to a memory file first.
Example
The preferences file read whole · open in the lesson
Run python3 memory.py whole (the code is shown here) and compare what you see with the output.
lines = load()prompt_part = read_whole(lines)print(f"preferences.txt: {len(lines)} lines")print(f"read whole into the prompt: {len(prompt_part)} characters")Prints the lines below (verified in CI from site/examples/building-agents/agent-memory/whole.py)
preferences.txt: 20 lines read whole into the prompt: 509 characters
Retrieval as a tool the agent calls
Unlocks when you finish Retrieval as a tool the agent calls.
Takeaways
- Retrieval is a tool like any other in the
TOOLSdictionary. The basic version has the harness call it once before the model, and the agentic version lets the model call it, judge the result, and call it again. - Start with basic retrieval and a test set. Add the loop only for the failures a test set shows and a better index can't fix, such as an answer spread over two documents.
- Frame retrieved text: say where it came from, ask for the sentence relied on, and name the answer to give when it doesn't cover the question. A model without a grounding instruction can answer from a passage that doesn't cover the question.
- An agent that doesn't find an answer says so. A test set for retrieval holds questions with no answer in the documents, and the expected answer for them is
not found.
Example
The prompt the model gets · open in the lesson
Run this, and compare what you see with the output below.
result = TOOLS["search_docs"]["fn"]("How many days of annual leave do I get?")print(build_prompt(result["passage"], "How many days of annual leave do I get?"))Prints the lines below (verified in CI from site/examples/building-agents/retrieval-as-a-tool/prompt.py)
The passages below were retrieved from the company handbook for this question. Answer from them. Quote the sentence you relied on. If the passages do not cover the question, answer exactly: not found. Passages: Annual leave. Every employee accrues 25 days of annual leave per year. Unused days carry over until 31 March. Requests go to your manager at least two weeks ahead. Question: How many days of annual leave do I get?
Sources
AEC-05Memory and context: context engineering, memory kinds, memory versus RAG, context rot, Agent Engineer Course (course)AEC-08Agentic RAG: the retrieve, evaluate, refine loop; when basic RAG is enough, Agent Engineer Course (course)Academy building-with-the-claude-apiBuilding with the Claude API, Claude Academy (course)Academy claude-platform-101Claude Platform 101, Claude Academy (course)