About llms.txt

Proposed by llmstxt.org, this file sits at the root of a website and offers a clear, concise overview. It is written in Markdown, a lightweight markup language that lets you structure content in a way that is simple and readable.

Today, two main files help search engines:

• The robots.txt file, which regulates access for indexing crawlers. But this file is a tool of exclusion: it does not provide information about the content itself.

• The sitemap.xml file, which offers an organized list of URLs. But it is of little use to LLMs, since it only gives addresses and how often they are updated, with no context and no descriptive content.

The /llms.txt file

The /llms.txt file supplies information that is structured, concise, and shaped for what LLMs need, alongside the files that already exist.

Unlike the other two, /llms.txt lets you put a site's content in context, with summaries, specific links, and extra detail. It lets LLMs such as ChatGPT reach the important information directly, without having to wander through hundreds of pages.

Adopting it complements existing standards such as robots.txt and sitemap.xml, and makes it far easier for LLMs to interpret what you publish.

We are at the start of the LLMO adventure. I am convinced it will bring a real renewal of (digital) communication, carried by an alliance between these technical tools and quality content steered by human creatives.