Llmstxt Guide

llms.txt Guidelines

Author and maintain a project's /llms.txt file (and the per-page markdown mirrors it points to), so LLMs can ingest the site at inference time within a bounded context window.

What llms.txt Is

A markdown file served at /llms.txt (or a subpath) that gives an LLM a curated, concise map of a site's most useful content. Complements: does not replace: robots.txt (which is about crawler permissions) and sitemap.xml (which is exhaustive and HTML-oriented).

File Format (strict ordering)

The file must contain these elements in this order:

1. H1 heading (required): project or site name
2. Blockquote (optional): short summary with key understanding info
3. Descriptive content (optional): zero or more markdown sections, excluding headings
4. H2-delimited file lists (optional, repeatable): curated link sections

Only the H1 is required. The other sections are optional but follow this ordering when present.

Link list grammar (inside an H2 section)

## Section name

- [Link title](https://example.com/page.md): Optional notes
- [Another link](https://example.com/other.md)

Each list item is:

• A markdown hyperlink such as [name](https://example.com/page) (required)
• Optionally followed by : and brief notes

The "Optional" section

An H2 section literally titled ## Optional designates secondary resources that can be skipped when a shorter context is needed. Processors use this distinction to produce two variants of expanded context.

Markdown Mirror Convention

For every HTML page worth giving to LLMs, serve a clean markdown version at the same URL with .md appended:

• https://example.com/docs/intro.html → https://example.com/docs/intro.html.md
• https://example.com/docs/intro → https://example.com/docs/intro.md
• https://example.com/docs/intro/ → https://example.com/docs/intro/index.html.md
• https://example.com/ → https://example.com/index.html.md

The llms.txt file links to these .md mirrors, not the HTML originals.

Expanded Context Variants

When a site provides llms.txt + linked .md mirrors, a processing tool (e.g. llms_txt2ctx) can produce expanded single-file context bundles:

• llms-ctx.txt: expansion without the ## Optional URLs (shorter context)
• llms-ctx-full.txt: expansion including the ## Optional URLs (full context)

Some projects also publish llms-full.txt as a single-file dump of everything (e.g. https://hono.dev/llms-full.txt): this is a common convention even though the spec primarily defines llms.txt.

Core Principles

• Concise, clear language: every link description earns its tokens
• Brief informative notes: say what the link gives, not "click here for more"
• Avoid ambiguous terms / unexplained jargon: LLMs can't ask follow-up questions
• Mirror at parity: every linked .md must actually exist at the URL
• Curate, don't dump: llms.txt is a guide, not a sitemap; aggregate-too-large content is the whole point of the format
• Test with multiple LLMs: single-model testing hides framing issues

Gotchas

• The H1 is the only required element; everything else is optional but must follow strict ordering when present (blockquote → prose → H2 lists, never reordered)
• Descriptive prose between the blockquote and the first H2 must not contain headings: sub-section structure is reserved for the H2 file lists
• Linking the HTML URL instead of the .md mirror is the #1 authoring mistake: LLMs receive raw HTML with nav chrome, breaking the whole purpose
• A page without filename needs index.html.md, not just .md
• llms-full.txt (single dump) is a popular sibling convention but not part of the spec: don't confuse it with the spec's llms-ctx-full.txt (which is a processing output)
• ## Optional is a literal section title with semantic meaning to processors: don't rename it ("Extra", "Bonus", "Additional") if you want the short-context variant to work

Progressive Disclosure

• Read references/create.md - Load when authoring a new /llms.txt from scratch for a project or site
• Read references/markdown-mirrors.md - Load when producing the per-page .md mirrors that /llms.txt links to (URL conventions, content rules)
• Read references/examples.md - Load when looking for reference /llms.txt implementations to model your own on
• Read references/processing-tools.md - Load when expanding /llms.txt + mirrors into a single-file context bundle (llms-ctx.txt, llms-ctx-full.txt)