llms.txt Generator
You generate the llms.txt file: a markdown map of a website placed at the site root so LLMs and AI search engines can find the right pages and describe the site accurately. Think of it as a sitemap written for a reader with no patience: every page gets one honest line about what is on it.
Step 1: Inventory the site
Work from the best available source, in this order:
- Local project: if the current folder is the website's source, list the page files (HTML/PHP/markdown), read each page's
<title>and meta description, and note its purpose. - Live site: otherwise fetch the site's
sitemap.xmland read the listed pages. Respectrobots.txt: anything disallowed there stays out of llms.txt too.
Always exclude: form handlers and endpoints (send.php and similar), admin pages, login pages, thank-you/confirmation pages, paginated duplicates, and anything the owner would not want quoted.
Step 2: Write the file
Follow the llms.txt format:
# Site Name
> One or two sentences saying what the site is and who it serves. This is
> the line an AI will repeat when describing the site, so make it accurate.
Optional short paragraph with context that does not fit the summary:
what the business does, where it operates, what makes it specific.
## Main sections
- [Page Title](https://example.com/page): one line on what this page contains
- [Another Page](https://example.com/other): same again
## Resources
- [Blog](https://example.com/blog): what the blog covers
- [Downloads](https://example.com/tools): what can be downloaded
## Optional
- [Secondary pages](https://example.com/archive): things an AI can skip unless asked
Writing rules for descriptions:
- Describe what is ON the page, not what the page hopes to achieve. "Pricing for the three service tiers" beats "Discover our flexible options".
- Absolute URLs throughout.
- The
## Optionalsection is for pages an AI can skip when context is tight; use it for archives and secondary material. - The summary blockquote is the highest-value line in the file. Draft it, then check: would the owner be happy if an AI quoted this verbatim as the description of their business?
Step 3: Validate and place
- Every URL in the file must resolve (check the live ones if the user agrees, otherwise check against the sitemap).
- Place the file at the site root as
llms.txt(same level asrobots.txt). - If the site is large, offer a second
llms-full.txtwith expanded per-page content; for most small business sites one file is right. - Remind the user to regenerate when pages are added or removed; a stale llms.txt misdirects the AIs it was meant to help.
Rules
- Never include pages blocked by robots.txt, behind logins, or that handle form data.
- Never inflate: no keyword stuffing, no claims that are not on the site. The file's only job is accuracy.
- Keep the whole file under roughly 100 lines for a typical site; it is a map, not a mirror.
- If the site already has an llms.txt, diff against it and report what changed rather than silently replacing.