llms.txt security risks: what 6,214 scanned domains showed

TL;DR. In late August, security researcher Alon Hertz reported scanning thousands of corporate domains for llms.txt, the markdown file sites publish to guide AI agents, and finding 120 whose files referenced packages or domains nobody had registered. He registered the names, shipped beacons, and watched coding agents inside Fortune 500 networks install them. The same file format, measured from the server side by Ahrefs this spring, is barely fetched by the AI search crawlers it was written for. The audience llms.txt was built for never showed up, and the audience that did show up executes what it reads. That gap between how agent-facing files are published and how they are consumed covers far more than this one format.

On clerk.com, documentation published by Clerk, the authentication vendor, contained the instruction npx clerk-next-fix-auth-protection. The command names a real binary that ships inside @clerk/eslint-plugin, so on a machine that already has the scoped package, it works as written. Run bare, it does something else: npx finds no local match and falls through to the public npm registry, where Clerk never published anything under that name. According to the research and the coverage that followed, someone else registered it and loaded it with live malware. An agent following the vendor's own docs, with no injection and no typo anywhere, installs an attacker's package.

That case anchors the census that Alon Hertz published in late August under the title "Data Became Code," and it is worth reading past the headline about tricking Fortune 500 agents. The finding underneath is about a file format. llms.txt spent 2026 being dismissed by the search industry as a dud, on good evidence. The security result does not contradict that evidence. It completes it: the only reliable reader the file ever found is a coding agent with a shell.

A file built for crawlers found a different reader

The format began as a context-window fix. Jeremy Howard of Answer.AI proposed /llms.txt in September 2024 as a curated markdown map at a site's root: the pages that matter, linked in clean text, so a model assembling context does not have to wade through navigation and JavaScript. Adoption stayed niche until Mintlify rolled the file out across the docs sites it hosts, at which point thousands of companies, Anthropic and Cursor among them, were publishing one, many without deciding to.

Then came the measurement. Ahrefs pulled server logs for 137,000 domains and found that while 28 percent published an llms.txt file, 97 percent of the valid files received zero requests in the study month. Of the little traffic that existed, about 1 percent came from AI retrieval bots of the ChatGPT and Perplexity kind; SEO audit tools alone generated far more. No major provider has committed to parsing the file for search. As a visibility play, the verdict is in.

But the file has a second consumer the crawler logs undercount: the coding agent already inside a task. An agent asked to integrate a vendor's SDK goes looking for that vendor's docs, and llms.txt is the door built for it. This reader arrives with tools attached and runs what it reads. Which means the question that matters for llms.txt was never "does Google fetch it" but "what happens when the text it serves is wrong."

The census: 227 references to code nobody owned

Hertz's team scanned 6,214 live domains belonging to Fortune 500 companies, big tech, and defense contractors, and collected over 8,000 llms.txt-style files. In 120 of them, each from a different site, the files referenced packages or domains that were registered nowhere: 227 install references in total pointing at names nobody owned. These are the researcher's own reported numbers; independent coverage corroborates them from the writeup, not from an audit of the raw scan.

The experiment stage is what moves this from hygiene report to incident forecast. The team registered the unclaimed names on npm and PyPI, embedding a minimal beacon that reported installation and nothing more. The first callback arrived in under four minutes, from inside a company Hertz describes as worth several hundred billion dollars, and callbacks kept coming, from agents built on Claude, Codex, and Hermes. Nobody phished anyone. The agents did what published documentation told them to do.

The docs author wrote a name and assumed one resolution. The resolver had a second one.

The mechanism is name resolution without an ownership check

Nothing here required inventing a new attack class. Alex Birsan's 2021 dependency confusion research breached dozens of companies by exploiting the same resolver behavior: when a name is not found where you assumed, package tooling helpfully looks somewhere public, and whoever owns the name there wins. The AI-era variant, registering names that language models hallucinate, already has a name, slopsquatting, coined by the Python Software Foundation's Seth Larson.

The llms.txt case is a third member of the family, and in one specific way the worst one. Dependency confusion needs a leaked internal name; slopsquatting needs the model to make a mistake. Here the dangling name is printed in the company's own published documentation, the one source an agent is explicitly designed to trust, and the mistake is nobody's in particular. A doc referenced a binary by its bare name. A package got renamed, or never shipped, or the install step assumed state the doc never stated. In a README read by humans, that is a support ticket. In a file read by something that executes, it is remote code execution waiting for a claimant, which is exactly what the Clerk namespace became in the wild.

What makes the 227 figure worth remembering is that every one of those references passed every control its publisher had. Docs pipelines deploy static text without code review. Dependency scanners audit lockfiles and manifests; a package name inside a .txt file is invisible to all of them. Link checkers verify that URLs return 200, and a dangling package reference is not even a link. Reference rot in documentation has been harmless for thirty years, so no tooling anywhere treats it as an attack surface. The consumers changed. The tooling has not.

Your other agent-facing files have the same property

llms.txt is the instance that got measured, not the extent of the class. The same publish-side blindness applies to any text a machine consumes with tools in hand. CLAUDE.md and AGENTS.md files accrete instructions that nobody prunes and every agent obeys. MCP servers ship an instructions field that lands in the system prompt without logging or review. And the context a catalog or context layer serves an agent is, structurally, the same object: descriptive text, machine-consumed, acted on with credentials.

For every one of these surfaces, two questions decide the risk. Who can write to it, and does anything verify what it references before an agent acts? The llms.txt census documents what happens when the second answer is "nothing": the file's references silently detach from the artifacts they meant, and the gap becomes claimable. The write side is no better a story. A docs CMS, a marketing contractor, a compromised Mintlify-style generator, any of these can edit text that thousands of agents will treat as instructions, through a pipeline that no code reviewer watches.

What checking a reference actually takes

The fix is unglamorous: treat agent-facing text as code, because agents do. Concretely, for the dangling-reference class the census measured, a first-pass CI check is small. Enumerate install commands and package references in llms.txt, AGENTS.md, and docs, then confirm each name resolves in its registry to a publisher you recognize:

# sketch: fail the build on unowned names in agent-facing docs
grep -rhoE '(npx|npm install|pip install|uvx) [A-Za-z0-9@/._-]+' \
  llms.txt AGENTS.md docs/ 2>/dev/null \
  | awk '{print $NF}' | sort -u \
  | while read -r name; do
      npm view "$name" maintainers >/dev/null 2>&1 \
        || echo "dangling or unowned: $name"
    done

The PyPI analogue queries the registry's JSON API; the principle is identical. Beyond the linter: fully qualify commands in docs you publish (a scoped @vendor/package cannot be claimed by a stranger the way a bare binary name can), route agent sandboxes through a registry proxy with an allowlist instead of letting npx fall through to the public registry, and log installs at the network and package-manager level, because an agent's own transcript is testimony, not evidence. Hertz's beacons were visible to any egress monitoring that was looking; in at least one multi-hundred-billion-dollar company, evidently nothing was.

One honest limit: an ownership check is point-in-time. It catches the dangling class, where a referenced name has no owner at all, and that is the class the census measured. It does not catch a name whose owner turns malicious later, or a registry account takeover. Verification narrows the surface; it does not close it.

Where Datapace fits

The census reads as a security finding, and it is equally a metadata finding: 227 claims, sitting in files no dependency scanner parses, each asserting that a name resolves to something its publisher trusts, none of them re-checked after publication. Datapace is building a semantic context layer for AI on databases: the resolved meaning of your schemas, validated by the people who own the data, with the operational evidence on the same graph (cost, performance, usage and freshness, lineage) and a policy gate over what an agent may do and access, served over MCP. Serving context to agents is precisely why this census lands close to home, and it is why provenance sits in the design: context an agent receives should carry where each claim came from and when it was last validated, so that acting on a stale reference is a policy question and not a silent default. If you are inventorying the files your agents already treat as instructions, book a call.

Sources

1. Alon Hertz, Data Became Code: We Ran Code Inside Fortune 500s Using Files They Published for AI Agents, Medium, August 2026 (6,214 domains, 120 files, 227 dangling references, beacon methodology, callback timing).
2. Cybernews, llms.txt files let hackers trick AI agents into malware, September 2026 (independent writeup of the census and the under-four-minute install).
3. Tom's Hardware, Researchers easily trick Fortune 500 companies' AI agents into running arbitrary code, September 2026 (Clerk case, agent models involved).
4. Jeremy Howard, Answer.AI, /llms.txt, a proposal to provide information to help LLMs use websites, September 3, 2024.
5. Ahrefs, We Analyzed 137K Sites: 97% of llms.txt Files Never Get Read, 2026 (adoption rate, request data, requester breakdown).
6. Alex Birsan, Dependency Confusion: How I Hacked Into Apple, Microsoft and Dozens of Other Companies, 2021.
7. Socket, The Rise of Slopsquatting: How AI Hallucinations Are Fueling a New Class of Supply Chain Attacks, 2025 (term origin, hallucinated-package research).
8. Mintlify, What is llms.txt? Breaking down the skepticism (platform rollout and adoption).