Skip to content

13. Honest Admission — We still do not know memory cleanly

~16 min read. Memory systems work surprisingly well in places, but many core questions remain open and uncomfortable.

Built on the ELI5 in 00-eli5.md. The cleanup-bell — the rule that decides what stays and what goes — reminds us that AI memory is still partly policy guesswork.

1) We still do not know what should become memory

See the first uncertainty. 

user says something
      │
      ├── temporary preference?
      ├── durable preference?
      ├── joke?
      ├── one-off frustration?
      └── sensitive fact we should not store?
Humans infer this naturally.

Agents do not. A sentence can look durable and still be temporary.

A sentence can look trivial and still matter later. So the address-book is never perfect.

The summary-card is never perfectly faithful. The filing-cabinet can become a museum of half-useful scraps.

Simple, no? Memory quality begins with judgement.

Judgement is still hard.

2) Hallucinated memories are especially toxic

A normal hallucination harms one answer.

A hallucinated memory can harm many future answers. That is worse.

If the agent stores, "User hates tables,"

without evidence, future behaviour shifts quietly.

If the diary-page stores the wrong event, future plans may avoid a step that never truly failed.

Look at the compounding risk. 

bad extraction ──→ bad memory ──→ repeated retrieval ──→ repeated wrong behaviour
This is why source references matter.

This is why confidence matters. This is why user visibility matters.

Still, even with those, false memories remain a live problem.

3) Privacy and creepiness remain unsolved in product terms

Teams can say, "We only store helpful preferences."

Users may still feel watched. The discomfort is not only legal.

It is experiential. When the librarian pulls the wrong private detail,

trust drops fast. When the cleanup-bell is invisible,

users assume nothing is ever forgotten. When profile memory is hard to inspect,

people feel trapped by prior interactions. So what to do?

Be explicit. Ask for consent when needed.

Expose controls. Prefer under-remembering to creepy over-remembering in consumer products.

That is not a theorem. It is a product judgement.

4) Worked example: one ambiguous preference

A user says, "Today I need a very detailed answer."

Later they say, "As usual, keep it brief."

What should the agent store? If it stores the first line as a durable fact,

future answers become too long. If it ignores the line entirely,

that session may be less helpful. A reasonable system might do this.

  • store a session override for detailed answers today

  • keep durable profile unchanged

  • update profile only after repeated signals See.

This sounds sensible. It still depends on heuristic choices.

How many repetitions count? How strong must the wording be?

No universal answer exists yet.

5) Open debates that honest teams should admit

Debate one:

Should memory be mostly user-visible, or can some internal memory remain hidden?

Debate two: Should memory extraction be model-driven,

rules-driven, or hybrid?

Debate three: Should the agent proactively surface what it remembers,

or only use memory silently? Debate four:

How much should deleted memory remain in aggregated model training? Debate five:

Can we prove that a system truly forgot something? Right now, not cleanly.

Look. Senior people should be able to say this plainly.

Memory systems are useful. They are also immature.

The desk-note is manageable. The address-book is governable.

The diary-page is queryable. But the full reliability story is not solved.

That is the honest admission.

Where this lives in the wild

  • Consumer chat assistants — product manager must balance helpful memory against the risk of crossing into creepy personalization.

  • Enterprise copilots — security lead face unresolved questions about deletion guarantees across caches, logs, and derived indexes.

  • Healthcare AI assistants — compliance officer must decide what should never become durable memory despite potential usefulness.

  • Autonomous research agents — ML engineer face compounding error when bad extracted memories are reused across long runs.

  • Personal companion apps — designer must decide how visible, editable, and revocable memory should be for trust.

Pause and recall

  1. Why is hallucinated memory often worse than a one-off hallucinated answer?

  2. In the worked example, why was a session override safer than a profile update?

  3. What makes privacy risk in memory a product issue, not only a legal one?

  4. Name two open debates that still lack clean universal answers.

Interview Q&A

Q: Why should senior engineers admit uncertainty in memory design instead of presenting one best practice? A: Retention, extraction, and visibility all depend on product goals, risk tolerance, and user expectations. There is no single clean optimum yet.

Common wrong answer to avoid: "Because the field is new" — novelty matters less than the fundamental ambiguity of value, privacy, and trust trade-offs.

Q: Why are hallucinated memories more dangerous than transient hallucinations? A: They persist and can influence many future turns. The error becomes stateful.

Common wrong answer to avoid: "Because users notice them more" — the bigger issue is repeated downstream corruption.

Q: Why might under-remembering be preferable to over-remembering in consumer systems? A: Slightly weaker personalization is often less damaging than trust loss from creepy or incorrect recall.

Common wrong answer to avoid: "Because storage is cheaper" — cost is not the main trade-off here.

Q: Why is proving deletion still hard in memory architectures? A: Data can exist in summaries, indexes, caches, and downstream derivatives. Removing every accessible influence is operationally difficult.

Common wrong answer to avoid: "Because databases cannot delete fast enough" — the challenge is distributed representation, not raw delete speed.

Apply now (5 min)

Exercise: List three memory behaviours you would keep conservative in a consumer product. Then list two behaviours you would make user-visible by default. Sketch from memory: Draw the chain from ambiguous user statement to session override, durable profile, or do-not-store decision.

Bridge. Memory made the agent more continuous, but also more failure-prone. Next we study reliability failures directly. → ../../04_resilient_agent_systems/00-eli5.md