What is llms.txt?
An llms.txt file: a plain-text map of your most important pages, written for AI crawlers.
What llms.txt is
llms.txt is a proposed convention for publishing an AI-friendly index of a website's key content at /llms.txt in the site root. The idea is that language models retrieving the site can read the file and get an ordered map of the most useful pages, without having to crawl the whole site first.
The file is Markdown by convention: a top-level project description, then sections with links and one-line summaries of each linked page. A companion llms-full.txt (also proposed) concatenates the full text of those pages into one fetchable document.
Does llms.txt work
The honest answer as of 2025-26 is: no major AI provider has publicly confirmed consuming llms.txt at inference or index time. Google has explicitly rejected it, comparing it to the long-deprecated keywords meta tag. Independent analyses find that under 1% of AI-bot traffic actually fetches llms.txt files, and hosting one shows no measurable correlation with citation rate.
It is not harmful. For sites that already ship a full sitemap, robots.txt, and clean semantic HTML, adding an llms.txt costs almost nothing and demonstrates GEO discipline. But it should not be sold internally or to clients as a ranking or citation lever without the caveat, and effort is better spent on Bing indexation, freshness, and off-site co-citation.
Frequently asked questions
- Should I add an llms.txt to my site?
- If it takes an afternoon and slots into your existing sitemap generator, add it - the cost is trivial and it is on-brand for a serious GEO programme. If it would take real engineering time or displace effort on Bing indexing, IndexNow, or off-site co-citation, skip it. The evidence base does not support treating it as a lever.
- What is llms-full.txt?
- A companion file that concatenates the full text of the pages listed in llms.txt into one fetchable document. The idea is to save an AI agent a round-trip per page. Adoption and consumption evidence are even thinner than for llms.txt itself; treat it as optional dogfooding, not a lever.
Every Osoro GEO scan measures this and the rest of the CITE framework on your own site - free, in about 30 seconds. How the CITE framework works →