Is llms.txt worth publishing?

The format was proposed at llmstxt.org in 2024 (llmstxt.org, 2024). The idea is simple and reasonable. A crawler landing on a large site has to reconstruct what matters from navigation, internal links and a lot of boilerplate. An llms.txt file short-circuits that by letting the site owner say, in plain language, here are the pages that matter and here is what each one covers.

It is a curation artefact, not a technical directive. It does not grant or deny access to anything, it does not change how a page renders, and it carries no authority a crawler is obliged to respect. It is closer to a hand-drawn map left at the entrance than to a rule posted on the door.

This is the part the enthusiastic guides tend to skip. For llms.txt to affect whether you appear in an AI answer, the engine generating that answer would have to fetch your llms.txt and treat it as a signal. As of 2026, none of the major search engines document doing so, and Google has stated publicly that it does not support the file and does not plan to (Search Engine Land, 2025).

Google's own documentation on AI features reinforces the point by omission. It describes appearing in AI Overviews and related surfaces as a function of being indexed and snippet-eligible, and makes no mention of llms.txt as an input (Google Search Central, 2025). If the file mattered to the largest AI search surface, that is where it would be named.

There is a narrower use that is real. Some model providers reference the format inside their own agent and developer tooling, where a cooperating agent is pointed at a documentation site and uses the file to navigate it. That is a legitimate convention, but it is a provider-specific workflow between an agent and a site it was told to read, not the open-web answer-engine behaviour that decides whether you get cited in a normal query.

Suppose an engine did read your llms.txt. It would still, to quote you, fetch the underlying page and lift a passage from it. The summary might help it decide which page to open, but the citation comes from the page. If that page is a client-rendered shell an answer crawler cannot parse, or a wall of prose with no self-contained claim to lift, no amount of curation in llms.txt rescues it.

This is why treating llms.txt as a visibility lever gets the order of work wrong. The load-bearing work is the same as it has always been: pages an answer crawler can retrieve, structured so a specific claim can be quoted and attributed. llms.txt sits on top of that as a convenience, at best. It is never the thing that makes an unparseable page citable (Osoro Solutions, 2026).

The case for publishing is not that it definitely works. It is that the cost can be genuinely small and the downside is close to zero. If you already maintain a canonical list of your most important pages, turning it into an llms.txt is a short task. The file is easy to serve, easy to keep current, and if the convention gains traction you are already positioned.

There are also second-order reasons that have nothing to do with any engine reading the file. Writing an llms.txt forces you to decide which pages actually matter and to describe each one in a sentence. That is a useful editorial exercise, and the descriptions it produces are often reusable elsewhere.

If publishing llms.txt turns into a multi-week initiative, with a new pipeline to generate it and a debate about which of a thousand URLs to include, the maths stops working. The upside is speculative and the cost has become real, and that same time spent on server-side rendering, extractable page structure or consistent entity signals has a known return.

The failure mode to avoid is treating the file as a headline task. It belongs in the category of cheap, optional hygiene, alongside keeping a sitemap current. It does not belong on the critical path, and it is not a substitute for the work that is.

The balanced answer is neither the hype nor the dismissal. llms.txt is a sensible, low-cost artefact that no major search engine currently rewards, that a few agent tools genuinely use, and that cannot stand in for the retrieval and extraction work underneath. Publish it if it is cheap and you can keep it accurate. Ignore it if it would cost real time. And in either case, judge your AI visibility by whether your actual pages get fetched and quoted, because that is the thing the engines are still deciding on.