Memory Layer vs Context Layer: Perfect Recall, Wrong Answer

Memory and context are different layers, and an agent on a database needs both

The short answer. A memory layer stores what the agent experienced. A context layer stores what the data means. Memory makes an agent consistent with its own past; a context layer makes it consistent with the database's present. Mem0, Zep, and Letta are memory infrastructure, and none of them can tell an agent which of three revenue columns finance actually reports. On any non-trivial schema you need both layers, and neither substitutes for the other.

A memory layer stores what the agent experienced: past conversations, user preferences, extracted facts, and task outcomes. A context layer resolves what the data the agent operates on means: which of three revenue columns is canonical, whether soft-deleted rows count, how tables actually join. Tools like Mem0, Zep, and Letta are memory infrastructure; they make an agent consistent with its own past. A context layer makes an agent consistent with the database's present, and no amount of recall substitutes for it.

That is the whole distinction. The rest of this post is why it matters in practice, two worked examples where flawless memory still produces a wrong answer, and a routing table you can lift into your own architecture docs.

A memory layer only knows what passed through the agent

Read the memory vendors' own definitions and the shape of the layer is clear.

Mem0 separates memory into layers, in its own words, "so agents remember the right detail at the right time": conversation memory (in-flight messages inside a turn), session memory (short-lived facts for the current task or channel), user memory (long-lived knowledge tied to a person, account, or workspace), and organizational memory (shared context available to multiple agents or teams). Zep builds a temporal knowledge graph of "the entities, relationships, and facts that matter to a user or subject," ingesting chat and business data and invalidating outdated facts as new ones arrive. Letta, built on the MemGPT line of work, gives agents self-editing memory blocks: "structured sections of the agent's context window that persist across all interactions," which the agent itself reads and updates as it learns.

These are different architectures (extraction pipeline, temporal graph, self-edited blocks), but they share one property: the contents originate in what passed through the agent. A fact enters Mem0 because a user said it. An edge enters Zep's graph because an interaction or an ingested record asserted it. A Letta block changes because the agent decided to write to it. Memory is downstream of experience. That holds even for Mem0's organizational tier: sharing a memory store across agents widens who can read a fact, but it does not change where the fact came from. A wrong guess about which column means revenue is still a wrong guess once every agent inherits it, and now it is a wrong guess with quorum. For a deeper tour of these architectures, see our memory layer architecture guide and the 2026 memory tools roundup.

That property is exactly right for the problems memory solves: not re-asking the user's name, remembering that this customer prefers concise answers, recalling that the last deployment attempt failed. And it is exactly the wrong property for a different class of problem.

What memory cannot know: two worked examples

Consider an analytics agent with a mature memory setup, wired to a Postgres warehouse. Its memory is perfect: every past session retained, every user preference extracted, every prior answer recallable. Both examples below are hypothetical, but the underlying patterns are among the most common semantic traps in real schemas.

Example one: duplicate revenue columns. The orders table carries revenue_total, revenue_net, and amount_usd. Two are legacies of a 2023 billing migration; only revenue_net is the number finance reports. Nothing in the schema says so. In session one, the agent picks revenue_total (a reasonable guess, it is the friendliest name) and reports quarterly revenue. The query is valid, rows come back, the user thanks it. Memory now contains a confirmed experience: revenue questions resolve via revenue_total. Every future session retrieves that memory and repeats the choice, consistently, confidently, and wrong by however much the two columns diverge. Memory did its job flawlessly. It preserved the agent's experience. The experience was a wrong semantic guess, and memory has no mechanism to detect that, because the truth lives in the database's history and in the finance team's heads, not in any conversation the agent had. We wrote about this failure class in multiple columns, same metric.

Example two: soft-deleted rows counted as active. The customers table uses soft deletion: a deleted_at timestamp, never a physical DELETE. Every human-written query in the codebase carries the filter; the convention is enforced by review, not by the schema. The agent is asked for active customer count and runs:

-- What the agent writes
SELECT count(*) FROM customers;

-- What the convention requires
SELECT count(*) FROM customers WHERE deleted_at IS NULL;

The first query is not an error in any sense a memory layer could catch. It parses, it executes, it returns a plausible number. The agent stores the interaction, the user does not notice the churn-inflated figure, and the wrong convention is now remembered as a working pattern. A schema dump would not have saved it either: deleted_at being a soft-delete marker rather than an audit column is semantics, not structure, as we argue in reverse-engineering an undocumented schema.

Both failures have the same three-step anatomy, and it is worth naming, because it is the whole reason recall does not help:

1. The guess happens on first contact, before there is any experience to remember. No amount of memory can recall a session that has not happened.
2. Nothing contradicts it. The SQL is valid, rows come back, the number is plausible, no error surfaces, and the user says thanks.
3. Memory launders the guess into a fact and replays it every subsequent session, with rising confidence and no new evidence.

A context layer resolves the meaning the schema never states

A context layer is the missing counterpart: a semantic map of the data itself, covering what's there, what it means, and how it connects. Applied to the two examples: it knows revenue_net is the canonical revenue column and revenue_total is deprecated with a reason and a date; it knows customers is soft-deleted and that "active" means deleted_at IS NULL; it knows which join path between orders and customers the business actually uses. Crucially, this knowledge is maintained against the database and validated by the people who own the semantics, not accumulated from any one agent's sessions. It is the same context for every agent, on their first query and their thousandth.

The two layers answer different questions. Memory replays the agent's past choice, including a wrong first guess at a deprecated column. The context layer resolves the canonical column regardless of what any agent did before.

Routing questions to the right layer

The practical takeaway is a routing rule: memory owns everything the agent learned by interacting; the context layer owns everything that is true about the data regardless of who is asking. When a question straddles both ("give me the revenue chart the way I like it"), memory supplies the presentation preference and the context layer supplies the column.

Liftable routing table: which layer owns which question. Copy this into your agent architecture doc as-is.

Two smells indicate you are routing wrong. If you find yourself writing schema facts into agent memory ("remember: always filter deleted_at"), you are using per-agent, drift-prone storage for shared ground truth; every new agent starts ignorant and every schema change silently invalidates old memories. If you find yourself asking the context layer who the user is, you have inverted the mistake.

You need both, and they compose cleanly

None of this is an argument against memory layers. An agent with only a context layer is semantically correct and socially amnesiac: it re-asks names, forgets preferences, and repeats analyses it already ran. An agent with only memory is personable and, on any non-trivial schema, quietly wrong in ways that compound. The composition is straightforward because the layers do not overlap: memory sits beside the agent and accumulates experience; the context layer sits between the agent and the database and resolves meaning at query time. The agent consults memory to understand the person and the task, and the context layer to understand the data, and neither layer is asked to fake the other's job.

If your agents already have memory and still misread production schemas, the gap is the second layer, and we measured how far that gap runs on real estates in the AI SQL production gap.

Where Datapace fits

Datapace is building that second layer: a semantic context layer for AI on databases, with a policy gate in front of execution and a Context API and MCP interface so agents can ask what a column means before they query it, starting with Postgres. The same graph is designed to carry the operational reality of the estate alongside the meaning (cost, performance, usage and freshness, lineage), so that "which column is canonical" and "which tables has nobody read in a year" are answered from one place rather than two tools. It is what this post describes, under construction rather than shipped. If the duplicate-column and soft-delete failures look familiar, see how we approach safe AI access to production databases, or book a call and we will walk your schema for the same traps.

Sources

1. Mem0 documentation: Memory Types
2. Zep documentation: Concepts
3. Letta documentation: Memory Blocks