llms.txt v2: What Changed and What To Do About It

The short version: v2 makes the file easier for AI agents to find, and removes the tooling that most published advice was built around. It also states plainly what the file is for. That last part is the useful bit, because it settles an argument the industry has been having for two years.

What llms.txt is, in one paragraph

It is a markdown file, usually at yoursite.com/llms.txt, that gives an AI agent a curated route into your site. An H1 with your business name is the only required element. Under it sits a one-line summary and a few short lists of links, each with a description: this page explains pricing, this one is the service overview, this is the case study with the numbers in it. robots.txt tells crawlers what they may access. sitemap.xml lists everything indexable. llms.txt says what is worth reading, and in what order.

Four things changed in v2

1. Agents can find your markdown without guessing

This was the most requested fix in two years of adoption. Given a page, how does an agent find the clean markdown version of it, or the llms.txt file that covers it? Previously it had to guess the URL. v2 uses standard link relations instead: rel="alternate" type="text/markdown" points at the markdown version of a page, and rel="describedby" points at the llms.txt that covers it. You can declare both as HTML <link> elements, or as an HTTP Link: response header.

The header version is the one to note. It works for non-HTML files, and it can be set in your server or CDN configuration without editing a single page template. For most sites, this is a config change, not a project.

2. Two URL forms for markdown pages are now valid

v1 specified appending .md to the full page URL, giving page.html.md. Plenty of publishing tools shipped the other pattern anyway, replacing the extension to give page.md. v2 allows both, and specifies index.html.md or index.md for URLs without a filename. Pick one form and use it consistently across your site.

3. A file in a subfolder now has a defined scope

v1 permitted llms.txt files outside the root without saying what they covered. v2 defines it: a file covers the pages beneath its own path, and where more than one applies, the most specific wins. So /docs/llms.txt owns everything in /docs/. This matters if you only control part of a domain: a documentation subfolder, a sub-brand section, a project site on shared hosting. It is also why the proposal deliberately avoided the /.well-known/ convention, which only ever exists at the root of a domain.

4. The context-expansion tooling is gone, and so is the meaning of "Optional"

v1 described a tool, llms_txt2ctx, that expanded your file into one large block of context for a model. v2 drops it and states the expected behaviour directly: an agent reads or searches your llms.txt, then follows the links it needs. The links should point at LLM-friendly content.

The knock-on effect catches people out. The Optional section only existed to tell those expansion tools what to leave out. It is still permitted, and still a sensible convention for secondary links, but it no longer carries any mechanical meaning. If a guide tells you to put pages in Optional so models will skip them, that guide is describing v1.

v1 to v2, at a glance

Element	v1, 2024	v2, Aug 2026
Finding the markdown	Guess the URL	Declared link relations
Markdown URL form	`page.html.md` only	`page.html.md` or `page.md`
Files in subfolders	Allowed, undefined	Covers its path, most specific wins
How it is consumed	Expanded into context	Read, then follow the links
Optional section	Machine-readable skip list	Convention only

Why the revision happened now

Because the proposal outgrew its assumptions. When it was written in September 2024, the idea that models would routinely read live websites was a prediction. It is now ordinary. Thousands of sites publish a file. Documentation platforms generate one automatically, and so do WordPress plugins including Yoast and AIOSEO. Chrome's Lighthouse audits sites for one as part of its agentic browsing checks. OpenAI, Anthropic and Google all publish llms.txt files for their own developer documentation.

Howard's own read is worth repeating: the expectation was that llms.txt would matter for inference rather than training, and that is how it has been used. It helps a model answering a question right now. It is not a training opt-in, an opt-out, or an access control.

The honest read: does it earn its keep?

Not yet, if you measure it the way most people are being told to measure it. On the evidence available, the effect on traffic and AI citations is close to zero. Search Engine Land reported that eight of nine sites saw no measurable traffic change after implementation. Google's John Mueller has pointed out that no AI crawler has actually claimed to consume llms.txt. Independent adoption studies through 2026 keep landing in the same place: files are being published, but almost nobody can show a fetch log that proves an assistant read one and sent a customer.

Where it demonstrably does work is narrower and more technical. Coding agents follow llms.txt files reliably, which is why every AI lab and documentation platform ships one. If your product has developer docs, this is already load-bearing. If you run a plumbing company, it is not.

The case for doing it is not the upside. It is that the cost is an hour of someone's afternoon, and the file becomes useful the moment agents start routing on it.

So: publish one, keep it accurate, and do not report it as a growth initiative. Anyone selling llms.txt as an AI ranking factor is selling something the data does not support. Anyone telling you to skip it entirely is ignoring how cheap it is to be ready. Treat it the way you treated a sitemap in 2006. Table stakes, not strategy.

What to do this month

01Check whether you already have a file, and whether a plugin is generating it. Look at /llms.txt and read what it actually says about your business.
02Cut it back. Ten to thirty links that answer the questions buyers ask, each with a real one-line description. Not your whole sitemap.
03Add the link relations. One Link: header rule at the CDN covers the whole site.
04Stop relying on Optional to hide anything. If you do not want a page read, that is a robots.txt conversation.
05Test it. Give an assistant only your llms.txt and ask it three questions a prospect would ask. If it cannot answer, the file is the problem.
06Review it quarterly, and watch your access logs for AI user agents rather than watching your rankings.

A template to copy

Copy this, then replace every line.

# Your Business Name

> One sentence on what you do, for whom, and where. This is the line a model reads first.

Founded 2011. Serving the UK and Ireland. Enquiries: [email protected]

## Services

- [Service overview](https://example.com/services.md): What we do, who it suits, how engagements run
- [Pricing](https://example.com/pricing.md): Current rates, minimum engagement, what is included

## Proof

- [Case study: 184% organic growth](https://example.com/case-studies/example.md): B2B manufacturer, 12 months, method and numbers

## Optional

- [About](https://example.com/about.md)

The .md URLs assume you publish markdown versions of those pages. If you do not yet, point at the normal HTML URLs and add the markdown later.

Common questions

Is llms.txt an official web standard?

No. It is an open community proposal, authored by Jeremy Howard of Answer.AI and now at version 2. It is not an IETF or W3C standard, and it is not an access control mechanism.

Do I need to rewrite my existing llms.txt for v2?

Usually not. A valid v1 file is still a valid v2 file. What you should add is the link relations, and what you should stop doing is treating the Optional section as a way to make models ignore pages.

Will llms.txt improve my AI search visibility?

On current evidence, no measurable effect for most sites. It is worth publishing because it costs very little and positions you for agent-driven traffic, not because it lifts citations today.

Does it replace robots.txt or my sitemap?

Neither. They do different jobs and are designed to coexist. Keep all three.

If you want the file done properly, with the link relations configured and markdown versions of the pages that matter, that sits inside my AI search and answer-engine work, usually alongside an AI readiness audit so we can see what agents are fetching before we change anything. The full v2 changes are on the record at llmstxt.org.

llms.txt just changed. Here is what actually matters.