The short answer

An llms.txt file is a Markdown file at a website's root that lists the site's pages in a format large language models can read easily. It is a proposal from 2024, not a web standard, and Google says it ignores llms.txt for search. Its real use is helping AI coding assistants and documentation readers navigate a site's content.

Key takeaways

llms.txt is a Markdown index of a site's pages, proposed in 2024. It is not a web standard.
Google says it ignores llms.txt — the file will neither help nor harm search rankings.
Its real users are AI coding assistants reading documentation, not search engines.
The file uses an H1, an optional summary, and H2 sections with lists of [name](url) links.
Good SEO basics matter more: llms.txt is no substitute for crawlable pages and helpful content.

What an llms.txt file is

An llms.txt file is a Markdown file that gives large language models a clean index of a website's content. The proposal, published on llmstxt.org in 2024, suggests placing the file at the site root, or at any path within it, covering the pages under that path. A model reading the file gets the site name, a short summary, and organised lists of pages with links — without having to parse navigation menus, sidebars and JavaScript.

The file is a proposal, not a web standard. It was created because documentation sites are hard for language models to read: their content is spread across many pages, wrapped in site chrome, and often loaded by JavaScript. A single Markdown index solves that problem for the specific case of documentation. It was not designed as an SEO tool, and it does not behave like one.

What the file looks like

The format is plain Markdown. The rules from llmstxt.org are short:

An H1 with the project or site name — the only required section.
An optional blockquote with a summary of the site.
Optional detail sections.
Zero or more H2 sections, each containing a Markdown list of [name](url) links, optionally followed by a colon and a short note.
An "Optional" H2 section is the convention for secondary links that an agent can skip.

A small example for a documentation site:

```markdown

Acme Docs

Acme's product documentation: guides, API reference and tutorials.

Guides

Getting started: install and first run
Configuration: every setting explained

Optional

Changelog: release notes ```

A companion convention lets each page offer a Markdown version of itself at the same URL with .md appended, linked with rel="alternate" type="text/markdown". The llms.txt file can point to those versions so a model reads clean Markdown instead of HTML.

Who actually uses it

The real users of llms.txt are AI coding assistants and documentation readers. Several AI companies publish llms.txt files for their own documentation, and Chrome's Lighthouse audits for one. When a developer asks a coding assistant about a library, the assistant can read the library's llms.txt and find the right pages without crawling the whole site.

Search engines are not users. Google says you do not need to create new machine-readable files, AI text files or markup to appear in its AI features, and that it ignores llms.txt — keeping one will neither harm nor help a site's visibility or rankings in Google Search. No official statement from Bing supporting llms.txt has been published; Bing and Copilot draw on Bing's index and robots.txt. Treating llms.txt as an SEO lever is a misunderstanding of what the file is for.

How to add one, step by step

The process in brief

List the pages that matter
Write the H1 and summary
Group links under H2 sections
Put secondary links under "Optional"
Save it at /llms.txt
Keep it updated

If your site is documentation-heavy and AI assistants read it, adding the file is cheap. The steps are:

List the pages that matter. Take the main guides, references and tutorials — the pages an assistant would need. Leave out thin pages, archives and tag pages.
Write the H1 and summary. One line for the project name, one or two sentences saying what the site covers.
Group links under H2 sections. One section per area: Guides, API reference, Tutorials. Each entry is a [name](url) link with a short note after a colon.
Put secondary links under "Optional". Changelogs, older versions and side pages go here, so an agent can skip them.
Save it at /llms.txt. Publish it at the site root so tools can find it at the standard location.
Keep it updated. A stale index is worse than none: it sends assistants to moved or deleted pages. Regenerate it when the docs change, ideally as part of the build.

A worked example: a documentation site with 40 pages

Take a software company with 40 documentation pages. Without llms.txt, an AI assistant answering a question about configuration must crawl the docs homepage, follow navigation, and guess which pages matter. With llms.txt, it reads one file and sees the structure: 8 guides, 12 API reference pages, 6 tutorials, and the rest under "Optional".

The team writes the file in an afternoon. It lists the 26 pages that matter, grouped under three H2 sections, and puts the changelog and old version notes under "Optional". The file is generated from the same source as the docs, so it updates when the docs update. The team does not expect any change in Google rankings — Google ignores the file — and measures success differently: fewer wrong answers from AI assistants when developers ask about the product.

Mistakes to avoid with llms.txt

Expecting ranking gains. Google ignores the file for search. Any time spent on it is time not spent on pages, links and content that do rank.
Listing every page. An index of 500 thin pages helps nobody. List the pages that answer real questions.
Letting it go stale. A file that points to moved pages sends assistants to 404s. Generate it from the docs, not by hand.
Copying HTML into it. The file is Markdown links and short notes, not page content. Keep each note to one line.
Hiding content behind it. The file is an index, not a replacement for crawlable pages. Search engines still need the real pages.

Five llms.txt terms worth knowing

llms.txt: a Markdown index of a site's pages, proposed for language models to read.
H1/H2: Markdown heading levels; the H1 names the project, H2 sections group the links.
robots.txt: the file that tells crawlers which pages they may fetch — a real standard that search engines follow.
AI Overviews: Google's AI-generated summaries in search results, which draw on indexed pages, not llms.txt.
Query fan-out: Google's technique of issuing multiple related searches to build an AI Overview response.

For the wider picture, read the what AEO stands for guide and the is ChatGPT good for SEO guide. The best AI SEO rank tracking software post covers tools for monitoring AI search visibility, and Adziq's SEO service page covers technical work if you want it run for you.

Common questions

What is an llms.txt file?

It is a Markdown file, usually at /llms.txt, that gives large language models a clean index of a website's content: the site name, a summary, and lists of pages with links. The proposal comes from llmstxt.org and was first published in 2024.

Does llms.txt help SEO?

No, not for Google Search. Google says it ignores llms.txt, and keeping one will neither help nor harm a site's visibility or rankings. It can help AI assistants that read documentation, which is a different job from ranking.

Should I add an llms.txt file to my site?

It costs little if your site is documentation-heavy and AI assistants read it. For a normal marketing site it adds nothing to search visibility. Spend the time on crawlable pages, internal links and helpful content first.

What does an llms.txt file look like?

An H1 with the project name, an optional blockquote summary, then H2 sections each containing a Markdown list of [name](url) links, optionally with notes after a colon. An 'Optional' H2 section holds secondary links an agent can skip.

Is llms.txt the same as robots.txt?

No. robots.txt tells crawlers which pages they may fetch, and search engines follow it. llms.txt is a voluntary content index for language models, and Google ignores it for search.

Sources

Adziq sells marketing services and does not accept payment for placement in its articles. See our editorial policy.

LLMs Txt File SEO: A Working Guide With Steps and Examples