Grounding and memory
Concepts · topic concepts/grounding
A model only knows its training data and what is in its context window. Grounding supplies the facts a task needs at the moment of asking, usually by retrieving documents. Memory carries information across turns and sessions. This topic explains both, what they fix, what they do not, and how a crowded context degrades.
Concepts
- Retrieval (RAG)
- Retrieval-augmented generation: before the model answers, a search finds the passages most relevant to the question and pastes them into the prompt, so the answer can rest on documents the model never saw in training. It is how assistants answer about your files or a company wiki. Its quality is bounded by the search; irrelevant passages produce confident wrong answers. glossary
- Grounding
- Tying a model's answer to sources it was given rather than to what it remembers. A grounded answer cites the passage it relied on and says so when the sources do not cover the question. Grounding reduces hallucination about the supplied material but does not make the model check facts on its own, and it cannot fix a source that is wrong. glossary
- Short- and long-term memory
- Short-term memory is the current context window: everything said and seen in this session, gone when it ends. Long-term memory is anything written down outside the window and read back later, such as notes files, summaries or a database the agent searches. Models have no other memory; "it remembered" always means something was stored and re-read. glossary
- Context rot
- The decline in a model's performance as its context window fills with old turns, tool output and stale instructions. Important details get lost among irrelevant ones, early instructions are followed less reliably and answers drift. Long sessions and large pasted documents cause it. The cure is to keep context short and deliberate: summarize, start fresh, or retrieve only what is needed. glossary
Links
- Builds on: What an agent is
- Leads to: Agentic retrieval and memory, Context engineering for code, Memory and session context
- Competencies drawing on it: Explains how a language model produces text and where it fails
Lessons
- Where memory comes from (explanation)
- Answers from documents the model never saw (tutorial)
Your reference
Each lesson above adds its takeaways and its example here once you finish it. Your reference lists every lesson you have finished.
Where memory comes from
Unlocks when you finish Where memory comes from.
Takeaways
- A model has no memory of its own. Short-term memory is the context window for this session, and long-term memory is anything written outside the window and read back in. "It remembered" always means something was stored and re-read.
- Products keep long-term memory as text in storage, loaded at a known moment: instruction files and the agent's own notes at the start of a coding session, a chat product's memory notes, or documents that a search brings in one passage at a time.
- Context rot is the decline in a long session as the window fills with old turns, tool output, and stale instructions. The facts are still there, and the answers drift, contradict, and ask for what you already gave. A fact in the middle of a long context is used less reliably than one at the edges.
- The cures are summarize, start fresh, and retrieve. Each moves text out of the window or keeps it from getting in, and the sign that you need one is an answer that ignores something you said in the same session.
Example
This lesson has no runnable example or prompt block.
Answers from documents the model never saw
Unlocks when you finish Answers from documents the model never saw.
Takeaways
- Retrieval is a search that runs before the model answers and pastes the best passages into the prompt. The model answers from text it can see, and the documents can change without the model changing.
- The search bounds the answer. A keyword search finds a document only when the question uses the document's words, and a wrong passage produces a wrong answer.
- A grounded answer points at its passage and says when the passages don't cover the question. Grounding makes an answer checkable and doesn't make it right: the passage can be wrong, old, or misread.
- Ask two questions of any assistant that answers from documents. What did it retrieve, and does the answer match the passage.
Example
This lesson has no runnable example or prompt block.
Sources
AEC-05Memory and context: context engineering, memory kinds, memory versus RAG, context rot, Agent Engineer Course (course)AEC-08Agentic RAG: the retrieve, evaluate, refine loop; when basic RAG is enough, Agent Engineer Course (course)Academy ai-capabilities-and-limitationsAI capabilities and limitations, Claude Academy (course)