How to Create an llms.txt File for Your Website

An llms.txt file is a plain-text Markdown file you place at your website's root (yoursite.com/llms.txt). It has one required section, an H1 with your project name, plus an optional summary and H2-grouped links to your most important pages. Write it in a text editor, upload it to your root directory, and update it as your site changes.

How Is an llms.txt File Different From robots.txt and a Sitemap?

An llms.txt file, a robots.txt file, and a sitemap.xml file solve three different problems. llms.txt hands AI models a curated summary of your best content, robots.txt tells crawlers what they may or may not access, and sitemap.xml lists every indexable URL on your site for search engines. None of the three overlaps completely with another, which is why most sites end up running all three at once.

File	Purpose	Audience	Controls access?
llms.txt	Curated summary and links to your most important pages, written for AI models	LLMs and AI assistants (ChatGPT, Claude, Perplexity, Gemini)	No
robots.txt	Rules for which crawlers can or cannot access parts of your site	Search engine and AI crawlers, as a directive	Yes
sitemap.xml	Complete, machine-readable list of every indexable URL	Search engine crawlers	No

Neither robots.txt nor sitemap.xml was designed with language models in mind. robots.txt only blocks or allows access, it does not explain what your content means or which pages matter most. sitemap.xml lists URLs but carries no context about priority or purpose. llms.txt fills that specific gap: it is the only one of the three written as a short, human-readable explanation of what your site is and what an AI system should read first.

What Do You Need Before You Create an llms.txt File?

Before you write a single line, you need two things: a short list of canonical pages worth surfacing to an AI system, and access to your site's root directory. Skipping either step usually produces a file that is either empty or unusable.

Start by listing the content you would want an AI assistant to read first if it only had time for a handful of pages: your documentation, your flagship product or service pages, your blog or changelog, and any page that answers the questions your users ask most. If you want the full background on what an llms.txt file actually is and why it exists, that companion piece covers the origin of the format in more depth.

You will also need root-level access to your site, either through FTP, your hosting provider's file manager, or your site's codebase if it is self-hosted. Without that access, you cannot place the file at yoursite.com/llms.txt, which is the location the format expects.

It's also worth deciding what to leave out. Pages behind a login, paywalled content, and internal admin tools do not belong in an llms.txt file, since an AI crawler cannot access them any more than a human visitor without credentials. A shorter, accurate list of five to ten pages is more useful to an AI system than a long list padded with links it cannot actually read.

What Is the Correct Format for an llms.txt File?

The correct format comes directly from the original llms.txt specification, proposed by Jeremy Howard and published on September 3, 2024 (source: llmstxt.org). It defines a single required element, an H1 heading with your site or project name, followed by three optional elements: a blockquote summary, free-text context paragraphs, and H2-grouped sections of links.

Each link inside an H2 section follows a simple pattern: [name](url): description. The name is the page title, the url is the full or relative link, and the description is a short, plain sentence explaining what a reader (or an AI system) will find there. There is no strict limit on how many H2 sections or links you include, but the spec is built around brevity, not exhaustiveness.

A Minimal Working Example

A short, fictional example makes the structure easier to picture than a bare description:

# Acme Docs

> Acme Docs is the documentation site for Acme's API platform, covering authentication, endpoints, and integration guides.

## Docs

- [Getting Started](https://docs.acme.com/getting-started): Setup steps for a new Acme account and API key.
- [Authentication](https://docs.acme.com/auth): How to generate and rotate API tokens.

## Guides

- [Webhooks](https://docs.acme.com/guides/webhooks): Configuring event notifications for account activity.

This example follows the required H1, an optional summary line, and two H2 sections grouping related links with one-line descriptions each. For more worked examples across different site types, see our collection of real llms.txt examples you can copy.

How Do You Write an llms.txt File Step by Step?

Writing an llms.txt file manually takes five steps, and none of them require a generator tool or a developer.

1. List your canonical pages. Pull together the handful of pages that best represent what your site or product does.
2. Draft the file in a plain text editor. Any text editor works; the file is Markdown, not code.
3. Order sections by priority. Put your most important content, usually documentation or your core product pages, in the first H2 sections.
4. Write a one-line description for each link. Keep each description factual and specific, not marketing copy.
5. Save the file as llms.txt. No extension changes, no capital letters, exactly that filename.

Most site owners finish all five steps in under an hour for a first version, even for a site with dozens of pages, since the file only needs to reference a handful of them. Once the file is written, the remaining work is placement and verification, covered next.

Where Should You Upload Your llms.txt File, and How Do You Verify It?

Your llms.txt file belongs at your site's root path, so it is reachable at yoursite.com/llms.txt. Documentation-heavy sites sometimes also place a secondary copy at a docs subpath, such as yoursite.com/docs/llms.txt, but the root location is what the specification and most AI crawlers expect by default.

To verify the file is live, open the URL directly in a browser and confirm the raw Markdown displays correctly, with no formatting errors or broken links. If you prefer a command-line check, run curl -I https://yoursite.com/llms.txt and confirm the response returns a 200 status code. Either method takes under a minute and confirms the file is both reachable and not accidentally blocked by a redirect or a robots rule.