00. Memory Systems for AI Agents — The Five-Year-Old Version¶
You are here after tools and before reliability. Now we teach the agent how not to feel blank every turn.
Think of an AI agent like a person working at a busy desk.
On the desk are today's sticky notes.
That is the part the person can see right now.
If too many notes pile up, some fall off.
That is short-term memory.
Behind the desk is a filing cabinet.
Important things get stored there.
Later, the person can open the drawer and pull back the useful sheet.
That is long-term memory.
There is also a diary.
The diary keeps dated events.
It answers, "What happened last Tuesday?"
There is also an address book.
That keeps stable facts.
It answers, "Who is this person, what do they like, and what are they allowed to do?"
One more thing matters.
A good office has cleanup time.
Old junk gets shredded.
Private papers get deleted on purpose.
Otherwise the desk becomes a mess, the cabinet becomes noisy, and the person starts using the wrong paper.
This whole module is about building that office properly.
We will see what stays on the desk-note.
What gets folded into a summary-card.
What belongs in the filing-cabinet.
What must be written as a diary-page.
What deserves a slot in the address-book.
How the librarian chooses the right sheet.
And when the cleanup-bell should ring.
Simple, no?
The placeholders you will see called back¶
| Placeholder | Meaning |
|---|---|
| desk-note | The active context window the agent can see right now |
| summary-card | A compressed recap of older conversation turns |
| filing-cabinet | Long-term searchable memory stored outside the live prompt |
| diary-page | Timestamped event memory about what happened and when |
| address-book | Stable user, org, and permission facts |
| librarian | The retrieval logic that chooses what memory to bring back |
| cleanup-bell | The forgetting system that deletes stale, risky, or low-value memory |
What's coming¶
- 01-why-memory-matters.md — Why stateless agents feel impressive for one turn and foolish by turn four.
- 02-context-window-management.md — How the desk-note fills up, and what we keep when it does.
- 03-conversation-history.md — How to store turns using windows, summaries, and rolling buffers.
- 04-memory-compression.md — How to shrink old chats into facts, summaries, and reusable signals.
- 05-long-term-vector-memory.md — How the filing-cabinet becomes searchable with embeddings.
- 06-episodic-memory.md — How diary-pages capture experiences, failures, and outcomes over time.
- 07-semantic-memory.md — How stable facts get extracted from messy conversation.
- 08-user-profile-stores.md — How the address-book stores preferences, permissions, and org context.
- 09-memory-retrieval-patterns.md — How the librarian decides what to fetch, and when.
- 10-forgetting-strategies.md — How the cleanup-bell protects quality, cost, and privacy.
- 11-shared-memory-multiagent.md — How several agents read and write shared memory safely.
- 12-memory-evaluation.md — How we test whether memory helps instead of harms.
- 13-honest-admission.md — What remains messy, risky, and genuinely unresolved.
Top resources¶
- Lilian Weng — LLM Powered Autonomous Agents — Clear section on short-term memory, long-term memory, and planning trade-offs.
- MemGPT paper and repo — Good mental model for pushing memory out of the prompt and paging it back.
- LangGraph memory docs — Practical patterns for thread memory, checkpoints, and durable state.
- Generative Agents paper — Helpful for episodic memory, reflection, and event retrieval ideas.
- Pinecone learn articles on embeddings — Useful for vector search intuition without heavy math.
- OpenAI Memory product docs — Good product view on user preference memory and safety controls.
- Anthropic context engineering guides — Strong practical advice on prompt budgets and retrieval discipline.
See the full office now.
The desk-note is small.
The filing-cabinet is large.
The diary-page remembers events.
The address-book remembers stable facts.
The librarian fetches.
The cleanup-bell deletes.
That is the whole game.
If we do not build this, the agent keeps starting life from zero.
If we build it badly, the agent remembers nonsense.
So first we must answer one hard question.
Why does memory matter at all?
Bridge. Before we discuss clever storage, we must feel the pain of not having memory. → 01-why-memory-matters.md