An llms.txt file is a markdown file placed at the site root, or at any subpath, that gives LLM-friendly overview content and links to detailed pages (1). You write it with an optional byte-order mark, an H1 with the project or site name as the only required section, and a blockquote with a short summary containing the key information needed to understand the rest of the file. Google states that it does not use llms.txt files for Google Search, so publishing one will not change your visibility or rankings there (2).
How do I write an llms.txt file?
Create a markdown file named llms.txt at the root path of your site, /llms.txt, or at a subpath such as /docs/llms.txt (1). A file covers the URLs under its path, and where more than one applies, agents should use the most specific one.
Start with an H1 containing the project or site name (1). A blockquote with a short summary comes next, containing the key information needed to understand the rest of the file. After that come zero or more prose sections of any type except headings, then zero or more H2 sections that hold file lists.
Each entry in a file list is a markdown link in the form [name](url), optionally followed by a colon and notes about the file (1). The proposal also suggests that pages with information agents might need provide a clean markdown version at the same URL as the original page, with .md appended (page.html.md) or with the extension replaced by .md (page.md). The file itself should stay small enough to fit in context, because the detail lives behind the links and is fetched only when needed. To help clients find these files, the proposal recommends standard link relations, with rel="describedby" pointing from a page to the llms.txt file that covers it.
Recala's own file is generated by the site's code each time it is requested, with the featured articles it lists coming from the published blog, and it links to a second file, llms-full.txt, that holds the full article list. Recala's llms.txt file opens with an H1 and a blockquote saying what the company does, followed by the sections About Recala, Plans, Core pages, Featured articles and Optional. Recala's generated llms.txt has grown past the size its code aims for, so anyone generating the file should check its size after publishing.
Parts of an llms.txt file and the rule for each
| Element | Rule in the proposal |
|---|---|
| Location | At /llms.txt on the site root or at a subpath; covers URLs under its path, and the most specific file wins |
| H1 title | Name of the project or site; the only required section |
| Blockquote | A short summary with the key information needed to understand the rest of the file |
| Optional prose | Paragraphs, lists and similar markdown of any type except headings, giving more detail on how to interpret the files |
| H2 file lists | Markdown lists where each item has a required link [name](url), optionally followed by a colon and notes |
| Link targets and size | Links should point to LLM-friendly content such as markdown versions; the file stays small enough to fit in context |
| Markdown page versions | Clean markdown at the same URL with .md appended (page.html.md) or the extension replaced (page.md) |
What should I expect from llms.txt when Google ignores it?
Google says that Google Search, including its generative AI features, does not use LLMS.txt files, so you do not need to create one to appear in AI Overviews or AI Mode (2). It is fine to maintain an llms.txt file for other services or systems that read it.
The proposal describes llms.txt as information read on demand, when an agent needs something while assisting a user (1). It expects the file to be useful mainly for inference rather than training. Google says that for Google Search you can ignore tactics like creating unnecessary AI text files such as llms.txt, so we do not treat the file as a ranking lever (2).
Recala has not measured any effect of its own llms.txt file on rankings or AI citations. There is no before-and-after comparison and no control group, and Recala does not guarantee rankings or citations from publishing the file. Treat it as a low-cost option, not as a growth lever.
llms.txt is a proposal for LLM-readable content and is not an access control, so it neither grants nor blocks crawler access the way robots.txt does (1).
What should I prioritise instead of llms.txt, and how do I check that crawlers can reach my site?
Put access rules in robots.txt, not in llms.txt (1). The robots.txt file must be placed in the top-level directory of a site, on a supported protocol (3).
For AI features in Google Search, the best practices for SEO remain relevant, and the existing SEO fundamentals continue to be worthwhile (4). Allow crawling in robots.txt and in any CDN or hosting layer, link important pages internally, keep important content in text, and make sure structured data matches the visible text. A page must be indexed and eligible to show a snippet before it can appear as a supporting link in AI Overviews or AI Mode.
For ChatGPT search, OpenAI recommends allowing OAI-SearchBot in your robots.txt and allowing requests from its published IP ranges, because sites opted out of OAI-SearchBot will not appear in ChatGPT search answers (5). Google generally caches the contents of robots.txt for up to 24 hours, though it may cache it longer when refreshing the cached version isn't possible (3). Recala keeps every crawler rule in its robots.txt and none in its llms.txt. Google's crawlers treat all 4xx errors except 429 as if a valid robots.txt file did not exist (3). When it fetches robots.txt, OpenAI may add a robots.txt marker to its user-agent string, which helps site owners tell those requests apart from other requests in their logs (5). OpenAI says it can take about 24 hours from a site's robots.txt update for its systems to adjust for search results.
A robots.txt file cannot enforce crawler behaviour, so for secure content other blocking methods such as password protection are better (6).
Next step
Continue with Recala to turn this plan into a concrete next step.
Sources
The /llms.txt file, v2 — llmstxt.org, llms-txt proposal, accessed 5 November 2026
Optimizing your website for generative AI features on Google Search, Google Search Central, last updated 10 July 2026developers.google.com
How Google interprets the robots.txt specification, Google, last updated 23 October 2026developers.google.com
AI features and your website, Google Search Central, last updated 10 December 2025developers.google.com
Overview of OpenAI Crawlers, OpenAI, accessed 5 November 2026developers.openai.com
Robots.txt Introduction and Guide / Google Search Central / Documentation / Google for Developers, Google, last updated 10 December 2025developers.google.com
recala.co
