llms.txt explained: what to put in it and whether you need one

What llms.txt is meant to do

Quick test: Open yoursite.com/llms.txt. If you see a 404, you can still start with 10 lines.

llms.txt is a suggested Markdown file that offers language-model tools a curated map of a website’s important pages. It is intended to make useful material easier to identify; it is not a replacement for your website, a sitemap, or a promise of visibility in AI answers.

The proposal describes placing the file at the site root, where it can point to selected content. In practice, think of it as a short index written for a technical reader: it tells a reader what your organization does and where to find authoritative information. The file itself does not make those underlying pages accurate, complete, or easy to use.

Before deciding to create one, check whether your site has a clear set of public pages worth pointing to. For a Web3 project, that might include a product overview, documentation, security information, token details, and support guidance—but only include material that is published and approved for public reference.

The proposal and its rationale are described at llmstxt.org. Read it as a convention to evaluate, not proof that a particular platform uses the file or will cite your pages. For the wider picture of how technical signals fit into AI search, see our guide to technical AEO.

Does SEO or AI visibility need it?

Most sites should decide based on the usefulness of the page map, not on an assumption that publishing it will improve search performance. If you can point to a small, stable collection of useful public pages, a well-maintained file may help communicate your preferred reading path. If your pages are outdated, contradictory, or still changing, fix those issues first.

Use this quick decision check:

• Can your team name the pages that give a reliable account of the project?
• Are those pages public, current, and accessible to a reader without special access?
• Can someone take responsibility for updating the file when a page changes?
• Would the file add a clearer route through the site than simply linking to its main pages?

If the answers are mostly yes, drafting the file is a reasonable, low-complexity technical task. If not, prioritize the source material. A polished index that points to unclear claims is not useful evidence about your organization.

For a practical Web3 site review, collect links to the project overview, product documentation, security disclosures, official social channels, and support information. Remove duplicates and resolve conflicting descriptions before you publish. If the objective is broader discoverability in AI answers, pair this decision with an AI visibility review rather than treating one file as a complete visibility plan.

llms.txt and schema.org compared

llms.txt and schema.org address different jobs. llms.txt is a proposed human-readable index of selected pages; schema.org provides structured vocabulary that can describe entities and page content in a more consistent format.

Use schema.org when you need to describe a real entity, product, article, or organization using appropriate properties. Keep the markup aligned with information readers can see on the page. Use llms.txt, if you choose to publish it, to orient a reader toward a curated set of resources. Neither format repairs weak source content, and they are not substitutes for each other.

A useful implementation order is to make important pages clear first, apply suitable structured data where it fits, then consider whether a separate text index helps navigation. The official vocabulary is available at schema.org. For more on choosing and validating markup, read our guide to schema markup for AI search.

Your 6-point llms.txt checklist

Follow these steps in order, and keep a dated copy of each version. A short, honest file is better than a long one, so stop at the pages that matter.

• List the 10 to 20 pages you want machines to read.
• Write one plain line per page: what it covers.
• Put the file at the root of the domain.
• Check that robots.txt does not block those pages.
• Keep it current when pages change.
• Test with 5 assistant questions before and after.

Terms in one line

llms.txt (in plain words: a plain file that points AI tools to your most useful pages).

Root of the domain means the top-level web address, such as example.com/.

Writing a useful file, line by line

Rule of thumb: List 10 to 20 stable public pages. Remove any page that you would not want quoted.

A useful llms.txt file is brief, accurate, and built from pages that a reader should trust as primary sources. Start with a plain-language description of the organization, then group or list the pages that best explain its work.

A simple outline can look like this:

• A top-level heading with the project name
• A short description that does not overstate what the project does
• Links to core pages, each with a short explanation
• Optional groupings for documentation, security, product information, or support

For example, a link description might read: “Documentation — product setup, supported networks, and usage instructions.” Use a description that matches the destination; do not label a marketing page as technical documentation. Link to canonical public URLs, not temporary previews or pages requiring a login.

Before publishing, check every URL, title, and description. Confirm that token details, contract references, security statements, and product claims are consistent with the linked pages and approved by the people responsible for them. Keep confidential roadmaps, internal dashboards, and unreleased material out of the file. Assign an owner so updates to the site prompt a review of the index, and keep the text readable rather than filling it with repeated keywords.

Publishing it on your site

Implement llms.txt by preparing the source pages, creating a readable file, publishing it at the intended location, and checking the live result. The practical work is usually straightforward when content ownership and website access are clear.

First, make a shortlist of public pages that explain the organization accurately. Check that each page loads and that its content is current. Next, write the file as plain Markdown-style text with a short summary and clearly labeled links; avoid turning it into a second copy of your website. Then place it at the site root so it can be reached at the expected path, usually /llms.txt.

After publication, open the URL in a browser and verify that the response shows the intended text rather than an error page, redirect loop, or unrelated content. Check the links from the live file, not just from your draft. Add ownership to your normal site-maintenance routine, particularly when product documentation or official project details change.

Keep a record of the published version and its source pages. If you are already organizing AI-facing technical work, include the file in a wider llms.txt and AI crawler setup review. That makes it easier to assess the file alongside page accessibility and other technical signals without treating it as a stand-alone fix.

How to test if it helps

The practical evidence is whether the file is correct, accessible, and useful as a map—not whether its presence alone proves that an AI system has used it. Keep those questions separate when evaluating the work.

You can verify things your team controls: that /llms.txt is available, the contents match the approved draft, the links work, and the linked pages provide the information their descriptions promise. You can also review whether important source pages are clear and consistent. Record the date of the check and the version of the file so a future reviewer can see what changed.

Do not infer adoption from publication alone. If you monitor answers from AI products, record the product, the prompt, the date, the answer, and any cited sources. Compare observations cautiously: prompts and answers can vary, and a citation appearing in one check does not establish that the file caused it. A useful citation audit examines the sources and wording that appear, not just whether your brand name is present.

For a Web3 project, check answers against the current official documentation and disclosures. If an answer misstates a chain, feature, or token detail, improve the authoritative page first; then review whether your file points readers to the right source.

What llms.txt cannot control, and the next step

llms.txt is a proposed convention, so publishing it does not establish that Google or any AI product reads it, uses its links, or cites your pages. Each platform controls its own crawling, selection, and answer behavior, and those outcomes are not controlled by the file’s author.

The most useful next step is to make the underlying information dependable. Keep one clear source for important project facts, label technical and promotional material accurately, and ensure that public pages agree on names and descriptions. A file that points to these resources can then serve as a concise guide rather than an unsupported claim about platform behavior.

If you already have strong source pages, create a draft and ask the content owner and technical owner to check it. If the source material is uneven, prepare a page list first and prioritize corrections. For Web3 teams planning a broader effort, our AI search visibility guide explains how technical work fits alongside content and external references.

Send your domain and the pages you consider authoritative. We’ll review the proposed sources, check whether an llms.txt file is a sensible next step, and outline the specific fixes to make before publication.

How it works

1. Choose the source pagesList public pages that explain your organization, product, documentation, and support. Confirm that each is current and authoritative.
2. Resolve conflicting informationCompare descriptions and project facts across the selected pages. Correct inconsistencies at their source before linking to them.
3. Draft the Markdown fileWrite a concise introduction and add clear links with accurate descriptions. Leave out private, temporary, or unapproved material.
4. Publish at the site rootMake the file available at /llms.txt and check the live URL, its contents, and each destination link.
5. Assign maintenance and reviewRecord who owns updates and check the file when important pages change. Judge it on accessibility and accuracy, not assumptions about platform uptake.