llms.txt

llms.txt is a convention proposed by Jeremy Howard at Answer.ai in 2024 for helping language models understand a website's structure. The file lives at /llms.txt on the apex domain and contains a curated text manifest of the site's most important content, in a format optimized for LLM consumption.

The format is intentionally simple: a top-level description of what the site is, followed by sections with links and one-line descriptions. Think of it as a sitemap.xml for LLMs — but where sitemap.xml is exhaustive (every URL), llms.txt is curated (just the URLs that matter for understanding the site).

Adoption is uneven in 2026. The convention is proposed, not standardized — Anthropic's Claude, OpenAI's ChatGPT, and Perplexity have indicated they'll consume llms.txt when present, but the protocol doesn't have the universal support that robots.txt has. Adoption is growing fastest in technical-product categories where AI search is a major referral source.

What goes in llms.txt: an organization-level intro (who you are, what you do), key product or content surfaces (with one-line descriptions), pricing or product structure where applicable, and links to deeper documentation or guides. The file is plain text, not Markdown — though Markdown rendering is permitted by the spec.

llms.txt sits alongside robots.txt (per-bot access rules) and sitemap.xml (per-URL discovery). Each serves a different consumer: robots.txt for crawlers' permission, sitemap.xml for crawlers' discovery, llms.txt for LLMs' understanding. Sites investing in AI search visibility ship all three by 2026.