What Is llms.txt? How to Write One and What It Does for SEO

Short answer: llms.txt is a markdown file at /llms.txt that introduces your site and lists its key pages for AI agents and LLM tools. The spec requires only an H1 with the site name. It is not a ranking signal: Google says you do not need it to appear in AI Overviews. It helps most when an AI agent or a coding assistant needs your docs, prices or API quickly.

What is llms.txt?

llms.txt is a plain-text file written in markdown and placed at the root of a website. It answers two questions for a language model: what is this site, and which pages are worth reading. The spec gives the reason: an LLM's context window is too small for a whole website, and HTML pages carry menus, ads and JavaScript that are hard to turn into clean text.

Since v2 the file can sit at the root (/llms.txt) or under a path such as /docs/llms.txt. Each file covers the pages below its path; where several apply, an agent uses the most specific one. The concept in short: llms.txt.

Does llms.txt help SEO and AI Overviews?

For Google, no. Google's AI features page (updated 10 December 2025) says: "There are no additional requirements to appear in AI Overviews or AI Mode, nor other special optimizations necessary." The next lines say you don't need AI text files. The one condition is that the page is indexed and eligible to be shown with a snippet.

For other AI crawlers, no major provider has published that it reads llms.txt for ranking or citations. The bot documentation of OpenAI, Anthropic and Perplexity explains control through robots.txt; none says their crawler reads llms.txt. Even so, Perplexity's own docs site publishes an llms.txt and invites AI agents to read it first. So treat llms.txt as a convenience for people and agents reading your docs, not as a ranking lever.

Our own numbers agree. In the 10 days from 9 to 18 October 2026, the CiteStat server returned /llms.txt 182 times. The log has no user agent, so we cannot tell who fetched it, and the count includes our own scanner. Nobody requested /llms-full.txt.

Who uses llms.txt?

Mostly developer documentation, where coding assistants and AI agents read the most. We ran the same check that CiteStat's rule uses on four public files:

llms.txt	Size	H1 first	> summary	## sections	Links
docs.claude.com (Anthropic)	86 KB	Yes	No	Yes	783
docs.stripe.com	98 KB	Yes	No	Yes	468
developers.cloudflare.com	17 KB	Yes	Yes	Yes	112
vercel.com/docs	5 KB	Yes	Yes	Yes	27

Source: CiteStat, scripts/growth/llms-txt-check.ts, 19 October 2026, one fetch per URL. All four pass. Stripe's file even opens with a direct instruction to coding assistants: check npm for the latest version instead of using a version number remembered from training data. That is the most useful way to use llms.txt today: tell agents what they tend to get wrong.

The table also exposed a bug of ours. CiteStat's old rule wanted at least 3 of 4 elements (H1, summary, ## section, link), so it reported Anthropic's and Stripe's files as "not following the specification". The spec requires only the H1. We fixed the rule: a file now fails only when its first line is not an H1.

How to write a valid llms.txt

Per llmstxt.org, the parts come in this order:

1. An H1 with the name of the site or project. This is the only required part.
2. A summary starting with >, with what is needed to understand the rest.
3. Paragraphs or lists, without headings, for more detail.
4. H2 sections, each a "file list": - [name](url): notes.
5. A ## Optional section for secondary links an agent can skip when it needs a shorter context.

An example for a coffee shop:

# Example Coffee

> Coffee roasted in Da Lat, shipped across Vietnam. Prices include VAT.

Orders over 500,000 VND ship free.

## Products
- [Cau Dat Arabica](https://example-coffee.vn/arabica.md): prices, roast level
- [Buon Ma Thuot Robusta](https://example-coffee.vn/robusta.md)

## Support
- [Returns](https://example-coffee.vn/returns.md)
- [Contact](https://example-coffee.vn/contact.md)

## Optional
- [Blog](https://example-coffee.vn/blog.md)

The spec suggests a clean markdown copy of each page at the same URL with .md added, and links in llms.txt that point to it. If you cannot do that yet, links to the normal HTML pages are still valid. v2 also recommends HTTP link relations: rel="alternate" type="text/markdown" for the markdown copy and rel="describedby" for the llms.txt file.

llms.txt vs robots.txt vs sitemap.xml

File	For	What it does
robots.txt	Every crawler, GPTBot and ClaudeBot included	Allows or blocks crawling. Major crawlers obey it.
sitemap.xml	Search engines	Lists every URL to index, with last-modified dates.
llms.txt	LLMs, AI agents, coding assistants	Introduces the site and picks the pages worth reading. Blocks nothing.

If you want ChatGPT or Claude to stop crawling your pages, llms.txt cannot do it; see AI crawlers and robots.txt.

Five common mistakes

• The first line is not an H1. A comment or summary placed before # Name breaks the spec.
• Using llms.txt to block AI. llms.txt has no blocking syntax. Bots follow robots.txt.
• Linking to JavaScript-only pages. Many agents do not run JavaScript, so the target page is empty. See do AI crawlers run JavaScript.
• Pasting the whole site into one file. llms.txt is a curated map. Long content belongs in the .md pages it links to.
• Stale facts. Prices or policies in llms.txt that differ from the site make agents answer wrongly. Update the file when you update the page.

What should come before llms.txt?

AI search quotes what it can read in your HTML. Three things come first: the main content in the first HTML response, a robots.txt that does not block the AI crawlers you want, and paragraphs that answer a question directly so they can be quoted (citability). llms.txt is the fourth step, part of Agent Readiness.

How does CiteStat check llms.txt?

When you scan a URL, CiteStat fetches /llms.txt, checks that the first line is an H1 and counts the optional parts: summary, ## sections and link lists. A missing llms.txt is a low-priority note. The same scan checks robots.txt for 10 AI crawlers, the words readable without JavaScript, and 300+ other rules from Google's and Bing's official documentation.

The free plan includes 10 scans a day. For several sites, or the AI render report that shows which passages AI Overviews and Copilot would quote, see pricing: Pro is 500.000₫/month (excl. VAT) for 100 scans a day.