Llms Txt

llms.txt Generator

Generate structured, LLM-friendly content indexes following the llmstxt.org specification.

Background

llms.txt was proposed by Jeremy Howard (Answer.AI) in September 2024. It provides a curated Markdown file that gives LLMs a structured map of a project's most important content — solving the problem of context windows being too small to process entire websites or repositories.

Adopted by: Anthropic, Cloudflare, Stripe, Vercel, Cursor, Mintlify, GitBook, Fern.

Used by: Cursor, Windsurf, Context7 MCP, Claude Code (reading local files), AI search engines.

Specification (llmstxt.org)

An llms.txt file is a Markdown document with sections in this exact order:

1. H1 heading (required) — the name of the project or site
2. Blockquote (optional) — short summary with key information for understanding the rest of the file
3. Body text (optional) — zero or more Markdown sections of any type except headings
4. H2 sections with file lists (optional) — each contains a Markdown list where every item has:
   • A required hyperlink: [name](url)
   • Optionally a : followed by notes about the file
5. ## Optional section (special) — URLs here can be skipped when shorter context is needed

No other heading levels are used. Only H1 (one, at the top) and H2 (for sections).

Two Output Files

Generation Workflow

Step 1: Gather Project Metadata

Read the primary manifest for the project name and description:

Step 2: Scan for Documentation Files

Check for these files and directories:

Primary docs:

• README.md
• docs/ directory (hub page, guides, API reference)
• examples/ directory

Supporting docs:

• CONTRIBUTING.md
• CHANGELOG.md
• SECURITY.md
• CODE_OF_CONDUCT.md
• ROADMAP.md
• LICENSE

Code entry points (include only if the project is a library/framework):

• src/index.* or lib/index.*
• Config files with schema documentation

Step 3: Write Descriptive Annotations

For each file, write a benefit-focused description — not just the file name:

Good:

- [Getting Started](./docs/guides/getting-started.md): Install, configure, and deploy your first worker in under 5 minutes

Bad:

- [Getting Started](./docs/guides/getting-started.md): Getting started guide

Use the feature-benefits approach: describe what the reader gains from reading that file.

Step 4: Assemble llms.txt

For repositories (local paths):

# [Project Name]

> [Description from manifest or README first paragraph]

[Optional body text: language, framework, key technical context]

## Docs

- [README](./README.md): Project overview, value proposition, and quick start
- [API Reference](./docs/api.md): Complete endpoint documentation with authentication and error codes
- [Configuration](./docs/configuration.md): All config options with defaults and examples

## Guides

- [Getting Started](./docs/guides/getting-started.md): Install, configure, and run your first example in under 5 minutes
- [Deployment](./docs/guides/deployment.md): Production deployment to Docker, AWS Lambda, and Cloudflare Workers

## Examples

- [Basic Usage](./examples/basic/): Minimal working examples for common use cases
- [Advanced Patterns](./examples/advanced/): Complex integrations and performance optimisation

## Optional

- [Changelog](./CHANGELOG.md): Version history with user-facing change descriptions
- [Contributing](./CONTRIBUTING.md): Development setup, coding standards, and PR workflow
- [Code of Conduct](./CODE_OF_CONDUCT.md): Community behaviour standards (Contributor Covenant v3.0)
- [Security](./SECURITY.md): Vulnerability reporting process and response timeline
- [License](./LICENSE): MIT license terms

For documentation sites (full URLs):

# [Project Name]

> [Description]

## Docs

- [Getting Started](https://docs.example.com/getting-started): Installation and first steps
- [API Reference](https://docs.example.com/api): Complete API documentation

## Optional

- [Changelog](https://docs.example.com/changelog): Version history
- [Contributing](https://github.com/org/repo/blob/main/CONTRIBUTING.md): How to contribute

Step 5: Generate llms-full.txt (if requested)

Concatenate all referenced files in the same order as llms.txt, with clear separators:

# [Project Name] — Full Documentation

> Complete documentation content for LLM ingestion.

---

## README.md

[Full contents of README.md]

---

## docs/api.md

[Full contents of docs/api.md]

---

[Continue for all referenced files, excluding the Optional section unless specifically requested]

Size management:

• Skip binary files (images, PDFs)
• For very large files (>50K tokens), include only the first section or a summary
• Note the total token count at the end of the file

When to Generate

Regeneration

llms.txt should be updated when documentation changes significantly:

• After adding or removing documentation files
• After major version releases
• After restructuring the docs directory
• Add to the release checklist alongside CHANGELOG updates

Real-World Examples

Specification Reference

• Spec: llmstxt.org
• Creator: Jeremy Howard, Answer.AI
• Reference repo: AnswerDotAI/llms-txt
• Directory: llms-txt-hub

Source: littlebearapps/pitchdocs — distributed by TomeVault.