Overview
This page is the atomic definition. The deep-dive lives at llms-txt.
Definition
llms.txt is a plain-text manifest served at /llms.txt that lists a site’s pages in a structured, agent-readable format. The proposal originated with Jeremy Howard in 2024 and uses Markdown with H1 for the site name, blockquote for the summary, and H2 sections grouping links. An optional /llms-full.txt ships full page bodies inline. Search engines and LLM agents can fetch a single file to learn the site’s structure without crawling the HTML graph.
When it applies
Ship llms.txt on any site that wants AI agents (search bots, browsing agents, ChatGPT, Claude) to find canonical content quickly. It is most valuable on documentation sites, references, and content vaults.
Example
https://llmbestpractices.com/llms.txt lists every page on the site, grouped by category, so an agent can fetch one file and route to the right page without recursive crawling.
Related concepts
- llms-txt - the deep-dive with format rules and adoption status.
- ship-llms-txt - the step-by-step tutorial for shipping the file.
- ai-txt - the parallel manifest that handles training opt-out.
- for-ai-agents - the agent-onboarding page on this site.
- llm-info-standard - the broader LLM-info standards survey.
Citing this term
See llms.txt (llmbestpractices.com/glossary/llms-txt).