TL;DR. Agentic memory files, the CLAUDE.md and AGENTS.md and rules files that steer coding agents, grow 226 percent over their lifetime and almost never shrink. A mining study of 247,694 instruction lifetimes across 1,867 repositories explains why: appending a rule is cheap, but once the reason for a rule is lost, deleting it without risking a regression means reasoning about its interaction with every subset of the other rules, which costs O(2^n). The authors call it catastrophic remembering, the inverse of catastrophic forgetting. Their tested fix is unglamorous and effective: keep the rationale next to the rule as a comment. That cut excess regrowth from 211.3 percent to 1.4 percent and improved instruction-following by up to 23.1 percent. The general lesson is one agent memory keeps teaching: storage without provenance degrades the agent it serves.
If you run Claude Code, Cursor, or any coding agent against a real repository, you own one of these files. It started as five lines about the test command. It is now a few hundred lines of accumulated scar tissue: rules about migrations, rules about a flaky CI runner that may no longer exist, a warning added by someone who left in March. You have almost certainly never deleted anything from it. A paper posted to arXiv this month, "Why Does CLAUDE.md Keep Growing? Catastrophic Remembering in Agentic Coding" (arXiv 2608.11095), is the first hard measurement of that experience, and its explanation for why the file only grows is structural rather than behavioral. The bloat is not a discipline failure. It is the equilibrium.
What 247,694 instruction lifetimes show
The study mines 1,867 public repositories that carry agentic memory files and tracks every instruction in them as a lifetime: the commit that introduced it, every commit that modified it, and the commit, rarely arriving, that removed it. That yields 247,694 instruction lifetimes, which is a large enough corpus to say something about the population rather than about anecdotes.
Three findings set up everything else. The files grow 226 percent over their lifetime, more than tripling from their initial size. They almost never shrink: deletion events are rare enough that growth is effectively monotonic. And rewrites do not reset the dynamic. Per the paper, a refactored file resumes growing at a faster per-commit rate than it had before the rewrite, so the periodic cleanup that teams reach for treats the symptom and leaves the mechanism running.
The mechanism is a cost asymmetry, and it deserves to be stated precisely, because it is the part that generalizes beyond markdown files.
Append is O(1). Delete is O(2^n).
Instructions enter these files the same way: something breaks. The agent runs the test suite in parallel and two tests clash on a port, so someone appends "run database tests single-threaded." The agent touches a generated file, so someone appends "never edit anything under src/gen." Each append is cheap, takes effect immediately, and visibly prevents a recurrence. The incentive gradient points one way.
Now run the tape forward eighteen months. The person who added the single-threaded rule has left. The commit that introduced it says "update CLAUDE.md." The CI runner that caused the port clash was replaced last quarter. Is the rule safe to delete?
Here is the study's central move: that question is not merely annoying, it is exponentially expensive. An instruction in a file of n instructions can interact with any subset of the others. "Run tests single-threaded" might be load-bearing for a later rule about test timeouts; the rule about timeouts might exist only because of the parallelism rule; nothing records which. With the rationale gone, certifying that a deletion causes no regression means reasoning over the power set of the remaining instructions, O(2^n) in the worst case, and natural-language rules offer no compiler, no type checker, and no dead-code analysis to prune that search. So at every individual decision point, the rational move is to keep the rule. Multiply that decision by every rule and every contributor, and monotonic growth is not a failure of discipline. It is what correct local reasoning produces.
Software has met this ratchet before: feature flags nobody dares remove, config keys that outlive the system they configured, dead code kept because deletion needs an archaeologist. Agentic memory files are the same pattern with two aggravations. The rules are natural language, so their interactions are unbounded and unverifiable. And the file is fed to a model on every session, so the cost of the accumulation is charged continuously.
The ratchet: every step is locally rational, and the file can only move one way.
Bloat is a performance bug, not a hygiene issue
The reason to care is not tidiness. A model reading your memory file gives attention to every rule in it, including the eighty that no longer apply. The study measured what leaner files buy: instruction-following improved by up to 23.1 percent under its intervention. Read the direction of that number. The rules you actually care about, the ones preventing yesterday's incident, are being missed today at a measurable rate because they share a file with the fossilized remains of 2025's build system.
Stale rules are worse than dead weight, because some of them are now wrong. A rule that says "the orders table has no updated_at column, join against the audit log instead" was true when written and is false after the migration that added the column. The agent obeying it is not malfunctioning. It is faithfully executing memory that nobody could safely remove. That is the exact shape of the problem we described in perfect recall returning the wrong answer: retrieval is not the hard part of agent memory, trusting what you retrieve is.
The fix that worked is provenance
The study's tested intervention is almost embarrassingly simple: keep the rationale inline, next to the rule, as a comment. In their evaluation, that cut excess regrowth from 211.3 percent to 1.4 percent, which is close to eliminating the ratchet, and it produced the instruction-following gains above.
The mechanism is worth spelling out, because the intervention looks like documentation and is actually a complexity reduction. A bare rule can only be deleted by reconstructing lost context, the O(2^n) search. A rule that carries its reason, its date, and its kill condition can be re-evaluated by reading one line:
# CLAUDE.md
- Run database tests single-threaded.
<!-- why: testcontainers port clash on the shared CI runner (INC-412, 2026-03).
remove when: CI moves to isolated runners. -->
- Never edit files under src/gen.
<!-- why: generated by protoc on every build; edits are silently overwritten.
remove when: codegen is checked in instead of build-time. -->
When the CI runners are replaced, the first rule announces its own death. The deletion decision drops from exponential to constant time. Provenance is not metadata here. It is the difference between memory that can be maintained and memory that can only accumulate.
CLAUDE.md is agent memory at its most primitive
A markdown file in git is the smallest agent memory system that exists: append-mostly writes, shared by every session and every contributor, no owner per entry, no expiry, no record of where an entry came from. The study matters beyond coding agents because every property it measures recurs in the grown-up versions of the same idea.
Vector-store memories accumulate entries whose embedding similarity says nothing about whether the underlying fact still holds. Team-scoped memory multiplies the writers, which is why shared agent memory needs access control built into its structure rather than filtered on retrieval; the same multiplication makes rationale loss faster, since the person who could explain an entry is now one writer among many. Benchmarks in this space overwhelmingly market recall. Almost none report whether the system can tell you why it believes an entry, when the entry was last true, or what it would take to retire it.
There is a neat symmetry with the problem we wrote about yesterday. Postgres's query statistics table evicts evidence silently and too eagerly when agent traffic floods it. CLAUDE.md evicts nothing, ever. Opposite retention policies, identical root cause: both stores keep content without keeping why it is there, so one cannot defend what it discards and the other cannot discard anything at all. Any memory that lacks provenance ends up at one of the two failure poles, forgetting what it needed or drowning in what it kept.
What to do with your file this week
The intervention is retroactive-friendly, so the order of operations is straightforward. Add a why-comment and a remove-when condition to every rule you can still explain; for rules nobody can explain, mark them explicitly as unexplained with a date, because "provenance unknown since 2026-08" is itself actionable in a way a bare rule is not. Require rationale on every new rule at review time, and note that agents append to these files too now, so the requirement has to bind the agent's own edits: an agent told to record why it added a rule is cheap insurance against it re-learning this entire paper's finding on your repository. Scope files so rules live next to the code they govern rather than in one global file. Then schedule a pruning pass against the recorded kill conditions, which is now a linear scan instead of an archaeology project.
The honest caveats: comments can drift like anything else, the study's numbers describe its corpus and its evaluation setup rather than your repository, and 23.1 percent is the top of the reported range, not the expectation. None of that weakens the structural claim, which is the part you can verify against your own git log this afternoon: your memory file has only ever grown, and nobody on your team can say which half of it still matters.
Where Datapace fits
Datapace is building the context layer between your databases and your AI: resolved meaning validated by the people who own the data, the workload evidence beside it (cost, performance, usage and freshness, lineage), and a policy gate over what an agent may do and access, served over MCP. The paper's fix is provenance at the scale of one file: every claim carries why it is believed and when it stops being true. A context layer is the same commitment at the scale of your schemas, where "validated by an owner" and "inferred, unreviewed" are different classes of memory and the difference is stored, not assumed. If you are deciding what your agents should remember about your data in the first place, start with what a context layer for AI agents actually is, or book a call.
Sources
- "Why Does CLAUDE.md Keep Growing? Catastrophic Remembering in Agentic Coding", arXiv 2608.11095, August 2026. Growth, deletion-cost, and intervention figures as reported by the paper.
- Full text (HTML version), arXiv.
- Claude Code documentation, "How Claude remembers your project": the memory-file mechanics (CLAUDE.md scoping and loading) referenced above.