Which AI agent memory tool should you use in 2026?
Short answer. Mem0 for memory as a service behind any framework. Zep when facts need validity intervals. Letta when the agent curates its own context. LangMem inside LangGraph. Cognee for a graph built from your own code and table rows. None of the five resolves what your database means; that fifth category is a context layer, not a memory tool, and it composes with any of them. First check whether you need a memory tool at all, and check it on your own history length: on LOCOMO, at about 26,000 tokens per conversation, sending the whole thing beat every tool inside Mem0's own table, while on LongMemEval's much longer histories Zep's paper reports the reverse. What holds across both, and in our own public run on free models, is that the tools win on cost.
The roundup rests on one idea: choose from measurements you can take, not from feature lists. Every claim about how a tool works comes from that vendor's own documentation. Benchmark numbers, and the disputes between them, are attributed in line to whoever produced them, which is not always the vendor whose product they describe.
A taxonomy: sort tools by what they remember
"Memory" in agent systems is not one thing. The tools in this roundup implement at least four distinct types:
- Conversation and user memory. What was said, summarized and promoted into durable facts about a user or session. This is the crowded category.
- Entity and relational memory. Facts structured as a graph of entities and relationships, often with time awareness, so the agent can reason about how facts connect and when they stopped being true. Two tools here implement it from opposite ends: Zep from conversations, Cognee from your own sources.
- Agent-managed context. The agent itself curates its own memory, deciding what to keep in view and what to set aside; in Letta's current design that memory is a filesystem the agent edits.
- Procedural and behavioral memory. Learned rules about how the agent should behave, typically fed back into prompts.
Each of these four maps onto a storage and retrieval pattern; the agent memory layer architecture guide covers the schemas and retrieval behind them. There is a fifth thing an agent on a database needs, which none of these four is: context grounded in the database itself rather than in conversations. It is a separate layer with its own article; the last section says where the two meet.
The 2026 memory tool landscape by source and scope. The conversation-derived quadrants are crowded. Cognee reaches into the database-grounded quadrant by copying rows and keys. The outlined region below it is a different layer, not a memory tool.
Mem0: extracted facts scoped to users, agents and sessions
Mem0 stores memories as extracted facts scoped by identifiers rather than as a hierarchy of types. Each memory is tied to a user_id, an agent_id, a run_id for a session or task, and on the hosted platform an app_id for a tenant, and retrieval is a semantic search filtered by those identifiers. When new messages arrive, an extraction step distils them into facts and adds them. Mem0's memory operations page describes that pipeline as add-only: memories accumulate rather than being overwritten, and a fact that stops being true is retired by an expiry date set when it is written, or by an explicit delete scoped to a user, agent, app or run, rather than by an update in place. Its older concepts page still describes an extraction pass that updates and deletes facts, so check which behavior your installed version has. (Mem0 memory operations) The documentation is explicit that, apart from a procedural memory type for step-by-step task knowledge, Mem0 does not sort memories into named types: the semantic and episodic labels exist in the SDK but are not wired up. (Mem0 docs)
The fit is strongest when your problem is a personal assistant or support agent that should stop asking users the same questions: facts about a user persist across sessions and agents, and cost per turn stays low because only the retrieved facts enter the prompt.
Zep: entity memory on a temporal knowledge graph
Zep's core abstraction is a temporal knowledge graph: nodes are entities, edges are facts and relationships, and the graph updates as new data arrives from chat, business data, documents, or JSON. Its distinguishing feature is fact invalidation. When a fact stops being true, the time it became invalid is stored on that fact's edge, so history is preserved rather than overwritten. Zep exposes six primitives: facts, entities, episodes, thread summaries, observations, and user summaries, assembled into token-efficient context blocks for the agent. (Zep docs)
Zep is a hosted service. Its open-source engine is Graphiti, Apache 2.0, which is what you run if you self-host: Graphiti's README says you bring your own graph database and build user and conversation management, retrieval tuning and tooling yourself, and the getzep/zep repository on GitHub holds examples and integrations, not the product. (Graphiti repository) Pick Zep when your agent needs to reason about how facts relate and evolve: a customer changed plans, a preference was superseded, an account moved owners. Flat fact stores lose that temporal structure; a graph with validity intervals keeps it.
Cognee: entity memory that reads the database itself
Cognee is the other major option in the entity and relational row, and the only tool in this roundup that ingests a database rather than a conversation. It is Apache 2.0, past 30,000 GitHub stars, and its pipeline runs sources through a cognify step that chunks them, extracts entities and relationships, and writes the result into a knowledge graph beside a vector index. Sources can be text, documents, code, or structured data. (Cognee repository)
The relational migration path is what matters for this taxonomy. Pointed at a Postgres or SQLite source, Cognee scans the schema for tables, primary keys, and foreign keys, turns every table row into a graph node, and turns each foreign key constraint into an edge between nodes. Structure is the only source of meaning in that step: if a schema does not declare its foreign keys, the nodes arrive unconnected. (Cognee: relational database to knowledge graph)
The maintainers have also published retrieval tuning on the framework, evaluated on HotPotQA, TwoWikiMultiHop, and MuSiQue, three multi-hop question-answering benchmarks. It is first-party work on a system the authors build, and the paper calls itself a preliminary version, so read it as engineering detail rather than a head-to-head verdict. (arXiv:2505.24478)
Pick Cognee over Zep when the graph has to contain the contents of your own systems, not only the facts your agent heard. Pick Zep when temporal validity is the requirement: Cognee's migrated graph records what the rows say, not the interval over which each fact was true.
Letta: agent-managed context, the MemGPT lineage
Letta descends from MemGPT, the 2023 research line that treated a model's context window like an operating system's main memory with the agent paging facts in and out, and it takes a different stance from the tools above: the agent manages its own memory. Today every Letta agent uses MemFS, a git-backed memory filesystem: the agent's memory is a repository of files, checked out onto whatever machine the agent runs on, that the agent reads and edits with ordinary file tools, with git holding the history of every change. The earlier design of labeled memory blocks, a human block for the user and a persona block for the agent, plus a separate archival store, is documented under the legacy V1 SDK, and the letta/letta Docker image belongs to that path: Letta's documentation now lists it as deprecated and points self-hosters to the App Server. (Letta MemFS, Letta Agent SDK memory)
Letta fits when you want a persistent agent with an evolving identity, something closer to a long-lived digital coworker than a stateless tool, and you are comfortable giving the agent authority over its own context.
LangMem: memory primitives for LangGraph
LangMem, from the LangChain team, provides functional primitives for extracting important information from conversations, maintaining semantic and episodic memory collections, and refining agent behavior through prompt optimization, which is procedural memory in practice. It is storage-agnostic in principle but integrates natively with LangGraph's long-term memory store, using the BaseStore interface with backends from in-memory up to AsyncPostgresStore. (LangMem docs)
If you are already on LangGraph, LangMem is the path of least resistance, and three facts belong next to that. It is Python only, so a LangGraph.js loop has no LangMem leaf. In the LOCOMO table below it was the slowest system, 60 seconds at p95 against 17 for the baseline, from 127 retrieved tokens, though that row was produced by Mem0 rather than by LangChain, so measure it on your own loop before ruling it out. And it has shipped no release since October 2025, though commits are still landing on the repository. It is less a standalone product than a set of building blocks that assume you own the surrounding agent architecture.
Mem0 vs Zep vs Letta vs LangMem vs Cognee, side by side
Here is the verdict the taxonomy produces: Mem0 for conversation and user memory as extracted facts, Zep for entity memory on a temporal knowledge graph, Cognee for entity memory built from your own sources and databases, Letta for agent-managed context in the MemGPT tradition, LangMem for memory primitives native to LangGraph. The table shows why.
Mem0
- Memory type
- conversation and user memory
- Core abstraction
- extracted facts scoped by user, agent, run and app
- Time awareness
- add-only ingestion; optional expiry date per memory
- Who curates
- the system, via extraction
- Best for
- personalized assistants and support agents
- Deployment
- Apache 2.0, self-hosted, or the hosted Mem0 Platform
- Languages
- Python and JavaScript SDKs
- Database-grounded context
- no
Zep
- Memory type
- entity and relational memory
- Core abstraction
- temporal knowledge graph
- Time awareness
- fact validity intervals on edges
- Who curates
- the system, via graph updates
- Best for
- agents reasoning over evolving facts
- Deployment
- hosted Zep Cloud; Graphiti, Apache 2.0, to self-host
- Languages
- Python, TypeScript and Go SDKs
- Database-grounded context
- no
Cognee
- Memory type
- entity and relational memory
- Core abstraction
- knowledge graph plus vector index
- Time awareness
- event timeline with temporal search; valid_to on nodes only, not filtered at search
- Who curates
- the system, via schema and extraction
- Best for
- graphs built from your own sources
- Deployment
- Apache 2.0, self-hosted Python pipeline
- Languages
- Python, plus an MCP server
- Database-grounded context
- partial: rows and keys copied in
Letta
- Memory type
- agent-managed context
- Core abstraction
- MemFS, a git-backed filesystem of memory files the agent edits
- Time awareness
- agent rewrites files as it learns; git keeps the history
- Who curates
- the agent itself
- Best for
- persistent, long-lived agents
- Deployment
- Apache 2.0 App Server (letta server), self-hosted, or Letta Cloud
- Languages
- TypeScript Agent SDK; Python API client
- Database-grounded context
- no
LangMem
- Memory type
- semantic, episodic, procedural primitives
- Core abstraction
- LangGraph BaseStore collections
- Time awareness
- up to your implementation
- Who curates
- the developer, via primitives
- Best for
- teams already building on LangGraph
- Deployment
- MIT library inside your LangGraph process
- Languages
- Python only
- Database-grounded context
- no
One dimension the table leaves out, because it catches teams late rather than early: how a memory is corrected or removed. Mem0 accumulates facts, so a correction is a new fact plus either an expiry date set at write time or a delete call scoped to a user, agent, app or run. Zep deletes a user's threads, artifacts and graph in a single call, which is how it answers a right-to-be-forgotten request. Letta's memory is files in a git-backed repository, so a correction is a file edit and the history stays in git. Ask any tool you shortlist for its erasure path before you store anything about a real person.
Notice the last row: four noes and one partial, and none of it is a criticism of these tools. They were built to remember interactions, and they do it well. Cognee is the one reaching across from the memory side, and the last section is about how far it gets.
Match the memory type to the tool.
| If your agent needs to remember... | Memory type | Reach for |
|---|---|---|
| User preferences and past conversations across sessions | Conversation and user memory | Mem0 |
| How facts about entities relate and change over time | Entity and relational memory | Zep |
| A graph built from your own documents, code, and database rows | Entity and relational memory | Cognee |
| Its own evolving identity and working notes | Agent-managed context | Letta |
| Memory inside an existing LangGraph app | Framework-native primitives | LangMem |
| Learned behavior rules fed back into prompts | Procedural memory | LangMem (prompt optimization) |
| What is in your databases, what it means, how tables connect | Not memory: a context layer | none of these |
Choose with a decision tree
The table above is keyed on memory type, and memory type is the conclusion the reader is trying to reach. A decision procedure has to start from facts you can observe about your own agent. Four gates do it, in order, numbered 0 to 3 as in the figure, and each comes with a measurement.
Gate 0: does the baseline already pass? Build 50 to 100 questions from your own agent logs, spread over the five abilities defined by LongMemEval, a benchmark of questions over long chat histories: information extraction, multi-session reasoning, temporal reasoning, knowledge updates, and abstention. Run them with the whole transcript in context, or in a file the agent can search, on your production model. Grade every run with the same judge model and the same rubric, and hand-check a sample of verdicts. Record accuracy, p95 latency, and the total cost per question; for a memory tool that total includes what it spends at ingest, turning history into memories, not only what it spends at the answer. If accuracy meets target and p95 and cost fit your budget, stop. You do not need a memory tool yet.
The evidence for this gate is the LOCOMO table in the next section: inside Mem0's own run the full-context baseline beat every tool on accuracy, and lost on cost. That was on histories of roughly 26,000 tokens per conversation, the length Mem0 sent in full. On LongMemEval's histories of about 115,000 tokens, Zep's own paper reports the reverse, full context at 55.4 and 60.2 against Zep at 63.8 and 71.2 with gpt-4o-mini and gpt-4o, which is why the gate is measured on your history length and not read off anyone's table. (Rasmussen et al., arXiv:2501.13956)
One more baseline belongs in that gate, and in 2026 it is the one most teams already have: the memory the model vendor ships. OpenAI persists conversation state server-side, so a thread continues by reference instead of being resent. Anthropic ships a memory tool that lets Claude keep files in a memory directory on your own infrastructure and read them back in later sessions. Google's Agent Development Kit has a memory service, with a managed Memory Bank behind it that turns finished sessions into searchable memories. LangGraph's own store persists long-term memories with no LangMem on top. None of them does entity resolution or temporal invalidation, and each ties that memory to one vendor's runtime, but all of them are configuration rather than a new dependency, so run them in gate 0 beside the window and the file. What a memory product has to beat is not nothing. It is what you already have. (OpenAI conversation state, Anthropic memory tool, ADK memory)
Gate 1: what must the agent recall that never passed through it? Tag each failing question by where the correct answer lives. If it lives in code or in table rows, Cognee is the only tool of the five built to ingest them, with the caveat from its own issue tracker that a whole-database migration makes the graph unwieldy and a schema-plus-samples path exists for that reason. If it lives in documents, that does not pick a tool: Zep's graph takes text and JSON, Mem0's platform takes text and PDF files, and Cognee runs them through cognify, so keep going. If the answer depends on what a column or a join means, none of the five holds it: add a context layer, the fifth category, defined in the uncovered-category section below, and keep going for the conversational side. If everything the agent needs was said to it, keep going.
Gate 2: must it know when a fact stopped being true? Count the knowledge-update and temporal failures as a share of all failures, and decide the threshold before you look; a fifth of all failures is a reasonable default. Above it, Zep's validity intervals are the mechanism you are paying for. Cognee has a temporal mode that timestamps extracted events for time-bounded search, and a valid_to stamp on nodes, but its documentation says search neither filters nor down-weights closed nodes, so applying validity is the caller's job; Zep applies it at retrieval, which is the difference this gate pays for. (Cognee time awareness, fact validity) Below the threshold, keep going.
Gate 3: who owns the agent loop and curates memory? This is an architecture fact you already know, and language decides before anything else: LangMem is Python only, so a LangGraph.js loop has no LangMem leaf. Letta cuts the other way and less sharply: the filesystem-style Agent SDK is TypeScript, while letta-client, the official Python library, is current and drives the same server through its API, so a Python team gives up the SDK ergonomics rather than the tool. (letta-client on PyPI) The Deployment and Languages rows in the table above have the license, hosting model and SDKs for all five. If the agent edits its own memory files inside Letta's runtime, Letta. If your loop is a LangGraph graph and you are willing to wire primitives, LangMem. If you want memory as a service behind any framework, curated by extraction, Mem0.
Version churn is the last fact for this gate, as of September 2026. Mem0's open-source SDK went 2.0 on 16 April 2026 with breaking changes: entity identifiers moved into a filters argument for search and retrieval, defaults changed, and the graph store left the SDK in favor of entity linking inside the vector store, so the "Mem0 with graph" row in the LOCOMO table below is not reproducible on the current package. Letta's block model now sits under the legacy V1 SDK heading in its documentation, so code written to blocks is on a surface the vendor has stopped documenting as current. LangMem is at 0.0.30, unchanged since 27 October 2025, which reads as pre-1.0 and quiet rather than churning; the repository is still maintained. (mem0ai v2.0.0 release, langmem on PyPI)
Two picks compose. A Zep or Mem0 agent that also answers questions about a database still needs the context layer from gate 1. They answer different questions.
The rationale: choose on what you can measure
The tree rests on one empirical fact and its consequences.
Public accuracy benchmarks do not separate these tools. One benchmark has published numbers for four of the five: LOCOMO, a released set of ten two-person conversations sampled from an earlier 50-conversation release to keep the longest, which Mem0's full-context run sent at about 26,000 tokens each, graded by an LLM judge whose accuracy is reported as the J score. (Maharana et al.) Cognee has never reported on it, and LangMem's row was produced by Mem0, not by LangChain. Here is every published number, with who produced it.
| System | Who ran it, when | Answering model | J score (%) | Memory tokens | Total p95 (s) |
|---|---|---|---|---|---|
| Full context, no tool | Mem0, April 2025 | gpt-4o-mini | 72.90 | 26,031 | 17.117 |
| Mem0 with graph | Mem0, April 2025 | gpt-4o-mini | 68.44 | 3,616 | 2.590 |
| Mem0 | Mem0, April 2025 | gpt-4o-mini | 66.88 | 1,764 | 1.440 |
| Zep, as run by Mem0 | Mem0, April 2025 | gpt-4o-mini | 65.99 | 3,911 | 2.926 |
| LangMem, as run by Mem0 | Mem0, April 2025 | gpt-4o-mini | 58.10 | 127 | 60.40 |
| Zep, corrected integration | Zep, May 2025 | not stated | 75.14 | not reported | not reported |
| Letta, file store only | Letta, August 2025 | gpt-4o-mini | 74.0 | not reported | not reported |
| Zep, tuned retrieval | Zep, December 2025 | gpt-4o-mini | about 80 | not reported | not reported |
| Mem0, managed platform | Mem0, April 2026 | not stated | 92.5 | under 7,000 | not reported |
| Zep | Zep research page, August 2026 | gpt-5.4 | 94.7 | 5,760 | not reported |
Read it as a scientist would. Inside the one run where grader and setup are held constant, the no-tool baseline beats every tool. The other rows come from three labs each reporting its own product, Zep, Letta and Mem0, on gpt-4o-mini in every 2025 row that states a model, with their own judge, their own integration and their own run count, and with category 5, the adversarial questions, dropped by Mem0 and Zep because its ground truth is missing. Zep's own December post puts the plateau near 80 and attributes it partly to the benchmark's ambiguous questions and inconsistent ground truth. The two 2026 rows sit above that plateau. Mem0's comes with no answering model, no judge, no run count and no baseline, and its post says the score reflects the managed platform with optimizations absent from the open-source SDK. Zep's row names gpt-5.4 as both reader and judge on 1,540 questions, the set without category 5, with no run count and no baseline; a gpt-5.4 reader also puts that row on a different answering model from every gpt-4o-mini row above it, which is the composition problem in one line. So LOCOMO cannot rank the tools. It can only tell you a tool lands near the baseline on conversational recall.
What the same table does separate is cost and latency. Retrieving a subset of memory instead of sending the whole history cut tokens by a factor of 7 to 15 and p95 latency by a factor of 6 to 12 for Mem0, Mem0 with graph and Zep in Mem0's table; LangMem cut tokens by a factor of 200 and was 3.5 times slower, because its search step was slow. Zep's December runs keep retrieval p50 under 200 milliseconds. The token gap follows from the design, so it replicates wherever the history exceeds what you are willing to send. The latency gap does not: it depends on how fast the tool's own retrieval step is, which is why gate 0 measures p95 per tool instead of assuming it. That is why gate 0 is a cost gate and comes first.
Choose on properties of your own problem. Each gate maps to one variable the reader owns: working set against budget, provenance of the missing facts, share of temporal questions, ownership of the loop. Each has a measurement you can take before spending anything. A feature checklist inverts this: it lists vendor properties and leaves you to guess which ones matter.
The fifth category has no scoreboard. LOCOMO and LongMemEval both test recall of conversations. Neither contains a question like "what does orders.status = 3 mean". No memory benchmark score bears on the context-layer leaf. Be precise about how far that goes. The leaf is unfilled by memory tools, not unoccupied: catalogs, semantic layers and text-to-SQL context stores are all built for it. Nor is it wholly unmeasured, since BIRD hands each question an evidence sentence saying what a coded column means, and Spider 2.0 makes the model consult database documentation across enterprise schemas, which is that question under a different name. What does not exist is a scoreboard that puts a memory tool and a context layer on the same questions. The test there is one you write yourself: twenty questions about column semantics and joins that appear in no conversation, asked of the agent cold.
A two-week pilot settles it. Pre-register the thresholds: accuracy target, p95 budget, cost per question with ingest counted, temporal share. Run the baseline and the candidate from the tree on the same questions with the same grader and the same model. Repeat at least three times and report the spread, as Zep did with ten runs per configuration. If the candidate does not beat the baseline on the pre-registered budget while holding accuracy, the tree sent you to the wrong leaf, or to no leaf.
Hindsight from a year of benchmark fights. The rows above were produced in public between April 2025 and August 2026, and three things the table cannot show matter more than any single row. Mem0's April 2025 claim was contested inside eight days, by Zep's corrected integration. In August 2025 the MemGPT authors at Letta reported that they could not find a way to run LOCOMO through MemGPT without major refactoring, that Mem0 did not answer their request for the method, and that a plain file store on gpt-4o-mini landed within two points of Zep's corrected graph; their explanation was that agents are post-trained on filesystem tools, which is why simpler designs kept pace with structured ones. And the top published number has kept moving: past 90 from both Mem0 and Zep in 2026, on the managed platform in Mem0's case and on a gpt-5.4 reader and judge in Zep's, neither with a shared baseline, so a pick made on a single table was made on a number that did not stand.
Our own benchmark: what the run shows so far
Everything above rests on other people's tables. So we built a benchmark and published it: datapace-ai/agent-memory-benchmark on GitHub holds the harness, every answer and judge verdict, and a write-up in the form of a paper. It feeds each system one session at a time in date order and asks the question only after the last session, so a tool can use only what it stored. It runs on free models, which means anyone can rerun it for nothing and check us.
Gate 0 above asks you for 50 to 100 questions. The first version of this section reported ten, which was below our own bar, and the order those ten produced did not survive the next forty. What follows is the run at 50 of a planned 100 LongMemEval questions, ten per ability, one seed, the same free model answering for every system and grading under three published rules. The ceiling is an oracle condition, the same model shown only the sessions that contain the evidence, which is the best this model and this judge can do on these questions.
| System | Accuracy, LongMemEval rule | 95% interval | Prompt tokens per answer |
|---|---|---|---|
| Oracle ceiling | 82 | 72 to 92 | 6,724 |
| File search, at most 8 tool calls | 78 | 66 to 90 | 15,291 |
| Mem0 | 68 | 54 to 82 | 501 |
| Last 32k tokens of the history | 64 | 50 to 78 | 26,932 |
| LangMem | 56 | 42 to 70 | 1,320 |
Fifty questions, one seed. The products sit one to two orders of magnitude to the left on tokens, and every interval still overlaps its neighbour on accuracy.
Configuration, so the rows can be read: run through 10 September 2026 with inclusionAI's Ling 3.0 Flash Fin on OpenRouter's free tier as the answerer, the judge, and the model inside the products at ingest; mem0ai 2.0.20 and langmem 0.0.30 with a local bge-small embedder in place of their defaults; retrieval of the top ten memories; no tuning for any system; each question's history reduced to twelve sessions, about 33,000 tokens at the median, with every session holding evidence kept. Mem0 returned an error on 5 of its 50 questions and those are scored as failures. A product's row is therefore the product with Ling inside it, not the product as sold with a frontier model.
Four findings, and one of them is a correction to what this section said a week ago.
- The products buy tokens, not accuracy. Mem0 answers from about 500 prompt tokens against 15,300 for file search and 26,900 for the window, thirty to fifty times fewer, and that ratio has held from the tenth question to the fiftieth. Accuracy has not. At ten questions the products were one question behind the baselines and we wrote that they buy context rather than recall. At fifty, file search leads Mem0 by 10 points and LangMem by 22. One seed and overlapping intervals cannot separate them, so the honest reading is that the cost claim survived the larger sample and the accuracy claim did not, which is also where the LOCOMO table landed.
- The ranking moves with the sample and with the rule. At 25 questions Mem0 was ahead of both baselines; at 50 it is ahead of the window and behind file search. The Zep rule ties Mem0 with the window at 70, and Mem0's partial-credit rule puts file search on top at 84. The harness prints "ranking is not stable across judge rules" for exactly this reason, and anyone quoting a single row of it, us included, is quoting noise.
- The answering model moves the ceiling more than any tool. On the ten questions both tracks have finished, with the same judge, a second free model (Nemotron 3 Super) dropped the oracle from 90 to 60 before any memory tool was involved. Part of that gap may be leniency, because the judge is the same model that answered on the first track; a second judge is the next step for that reason. It remains the concrete reason vendor numbers produced under different models do not compose.
- Ingest is the cost vendors do not show. Mem0 and LangMem spend seconds per session turning conversations into memories, against a median answer under two seconds. Graphiti, Zep's engine, and Cognee spend on the order of a hundred model calls per question building a graph, an estimate, since the products' internal calls are not metered. We are not printing our own ingest seconds: they came off a free endpoint that failed about one call in four, so they measure the endpoint rather than the products. The ratio is what holds, every product spending tens to hundreds of times longer at ingest than at the answer, and it is why gate 0 budgets the whole question rather than the answer alone.
Memory did not help the hardest column either. On the ten questions whose correct answer is that the information is not there, both baselines scored 80 and both products 70, and the judge treats a long refusal more kindly than a terse one, so abstention is the noisiest measurement in the table as well as the least flattering.
Limits to keep in view:
- Two of the five tools are measured. Letta was configured but not run, and its adapter targets an interface the vendor has deprecated.
- Cognee and Graphiti, as we configured them, ask the model for JSON through the response format field, which the free Ling endpoint refuses, so they ran only under the second model and are still partial. Cognee also documents a tool-call mode that Ling would accept, which the next run uses.
- Graphiti is the engine, not Zep's hosted product, so nothing here is a Zep Cloud result.
- The findings therefore bear on gate 0, not on the Zep or Letta leaves.
- The judge on the first track is also its answerer.
- One seed, and 50 questions of a planned 100. The order of the products is not a verdict, and this table will move again.
The remaining questions and the second and third seeds are landing in the same public repo: the track summary is the live table, and the results page is the write-up.
The one thing none of them does
Run the taxonomy to its end and a gap appears. Four of the five learn only from what flows through the agent: messages, extracted facts, documents fed in. Ask a Mem0-backed agent what orders.status = 3 means in your Postgres database and it can only answer if someone happened to say it in a conversation it stored. Two follow-ups sit on top of this taxonomy: once several agents write to one store, can they safely share it, and what happens to hosted agent state when the vendor goes away, as Instant Cloud did when OpenAI hired the InstantDB team.
Cognee is the exception and deserves a direct answer, because it does read the database. Look at what it takes: rows as nodes, foreign keys as edges, embeddings over both. That is the database copied into a graph, and copying is a different contract from grounding. orders.status = 3 arrives as an integer on a node, carrying no more meaning than it had in the column, and foreign keys carry the joins someone already declared, not the ones nobody wrote down. Cognee's own maintainers drew the same line: migrating an entire database produces "excessive noise in the knowledge graph" with limited semantic value, which is why they added a lighter schema-plus-samples path. (cognee issue #1350)
So a question like "which of these three revenue columns does finance trust" is not a memory question, and no tool in this roundup will answer it. It belongs to a different layer with a different contract, and it is a large enough topic to argue separately: we draw the boundary in memory layer vs context layer, define the layer in what is an agent context layer, and take up the data catalogs that have moved onto the same ground in OpenMetadata 2.0 rebrands the catalog as a context layer. If you want to see the shape of it before buying anything, build one for Postgres yourself. Memory personalizes, context grounds, and the two compose rather than compete.
Where Datapace fits
Datapace builds database-specific AI agents that understand how data teams run and govern their databases across engines. They work from one graph that documents the entities, the relationships between them, what they mean, and the infrastructure around them, and they propose the routine work back to a person for approval: documentation, right-sizing, performance, migration mapping. The fifth category above is the part of that graph no memory tool holds, which is why the two compose. Agents reach it behind a policy gate, through an API and MCP, rather than through raw access to production.
We are running pilots rather than selling a general release, which is why Datapace is not a sixth product in the table above. If that is the problem you are hitting, see safe AI access to your databases or book a call. And if you would rather test a claim in this article than take it, the benchmark is public and runs on free models.
Sources
- Mem0 Documentation: Memory Types
- Zep Documentation: Concepts
- Letta Documentation: MemFS
- Letta Documentation: Memory, Agent SDK
- LangMem Documentation
- Cognee repository on GitHub (Apache 2.0)
- Cognee: Relational Database to Knowledge Graph: Query with LLMs (May 2025)
- Markovic et al., Optimizing the Interface Between Knowledge Graphs and LLMs for Complex Reasoning (arXiv:2505.24478)
- cognee issue #1350: ingest relational schema and examples instead of entire database
- Chhikara et al., Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory (arXiv:2504.19413), Table 2
- Zep: Lies, Damn Lies, and Statistics: Is Mem0 Really SOTA in Agent Memory? (6 May 2025)
- Letta: Benchmarking AI Agent Memory: Is a Filesystem All You Need? (12 August 2025)
- Zep: The Retrieval Tradeoff: What 50 Experiments Taught Us About Context Engineering (9 December 2025)
- Maharana et al., Evaluating Very Long-Term Conversational Memory of LLM Agents, the LOCOMO benchmark (arXiv:2402.17753)
- Wu et al., LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory (arXiv:2410.10813)
- getzep/zep repository README, stating the repository is examples and integrations and pointing to Graphiti
- datapace-ai/agent-memory-benchmark: harness, run records and the write-up
- Rasmussen et al., Zep: A Temporal Knowledge Graph Architecture for Agent Memory (arXiv:2501.13956), LongMemEval results
- Mem0: The Token-Efficient Memory Algorithm (16 April 2026)
- Graphiti repository README, what self-hosters bring and build
- Zep: Research, LoCoMo and LongMemEval results with methodology (page last modified 27 August 2026)
- Letta Documentation: Self-hosting, App Server
- mem0ai v2.0.0 release notes (16 April 2026)
- langmem on PyPI, release history
- Cognee Documentation: LLM providers and structured output modes
- Cognee Documentation: Time awareness
- Cognee Documentation: Fact validity
- letta-client on PyPI, the official Python library for the Letta API
- OpenAI: Conversation state in the Responses API
- Anthropic: the memory tool, a memory directory Claude reads and writes across sessions
- Google Agent Development Kit: Memory, and the managed Memory Bank behind it
- BIRD-SQL, the text-to-SQL benchmark that hands each question an external-knowledge evidence sentence
- Spider 2.0, enterprise text-to-SQL workflows with database metadata and documentation to consult