LLM and SEO readiness (Aptos docs)
LLM / machine-readable
- Frontmatter: Every new or renamed doc needs accurate
titleanddescriptionin English; they surface in/llms.txtand.mdexports. Mirror updates insrc/content/docs/zh/per agent guidelines (Spanishes/is out of scope for agent-maintained docs). - Curated lists: If a page should appear in the LLM index or early in full corpus, update
src/lib/llms-curated-ids.ts(LLMS_INDEX_SECTIONS,LLMS_SMALL_DOC_IDS,FULL_PRIORITY_DOC_IDSas appropriate). Build fails if an id is missing from English non-draft docs. - Index copy: User-facing explanations live in
src/content/docs/llms-txt.mdx,src/content/docs/build/ai.mdx, and the Chinesezh/counterparts—keep URLs and feed names aligned withsrc/pages/llms-index.ts. - HTML → Markdown: Shared logic is
src/lib/llms-html-sanitize.ts. When minifying forllms-small.txt, collapse spaces/tabs only—never all\s(newlines must survive for fenced code and Markdown structure). - Route wiring: Custom handlers are swapped in via
src/integrations/llms-txt-index.ts; endpoint implementations live undersrc/pages/llms-index.ts,src/pages/llms-small.txt.ts,src/pages/llms-full.txt.ts. - Robots:
public/robots.txtshould stay consistent with the sitemap URL (https://aptos.dev/sitemap.xml) and, when feeds change, the commented LLMs.txt pointers at the bottom. Keep theContent-Signalline intact so AI crawlers see the same preferences the rest of the ecosystem uses. Keep theAgentmap:line pointing at/.well-known/ai-catalog.json. - Agent discovery surface:
.well-known/api-catalog,.well-known/mcp/server-card.json,.well-known/agent-skills/index.json,.well-known/ai-catalog.json, the globalLinkheader invercel.json, and the Head override's<link rel="…">tags must advertise the same URLs. Re-runpnpm test tests/agent-discovery.test.tsafter any change and see the Agent discovery & readiness section ofCLAUDE.mdfor the full checklist.
SEO
- Per-page metadata: Starlight frontmatter
title/descriptionfeed OG, Twitter, and schema insrc/starlight-overrides/Head.astro—avoid empty or placeholder descriptions on public pages. - Sitemap: Produced by
@astrojs/sitemapinastro.config.mjsassitemap-0.xml+sitemap-index.xml.src/integrations/sitemap-xml-alias.tscopies the urlset to/sitemap.xmlon build, andvercel.jsonrewrites/sitemap.xml→/sitemap-0.xmlas a fallback. Keeprobots.txtpointing at/sitemap.xml. - Hreflang / alternates: Head override builds language alternates from
SUPPORTED_LANGUAGES—new locales need config updates, not only content folders. - Draft / hidden content: Do not rely on curated LLM ids or public sitemap for content that must stay off production; follow existing draft filtering in the
.mdpipeline and curation tests.
Verification
After substantive changes:
pnpm test tests/llms-curated-ids.test.ts tests/llms-html-sanitize.test.ts tests/agent-discovery.test.ts
pnpm lint && pnpm check
For full coverage: pnpm test and a production pnpm build when touching routes or integrations.
File map
| Area | Primary locations |
|---|---|
| Curation | src/lib/llms-curated-ids.ts, src/lib/llms.ts |
| llms.txt body | src/pages/llms-index.ts |
| Feed endpoints | src/pages/llms-small.txt.ts, src/pages/llms-full.txt.ts |
| Plugin override | src/integrations/llms-txt-index.ts |
| Markdown export / sanitize | src/lib/llms-html-sanitize.ts, src/pages/[...slug].md.ts |
| User docs | src/content/docs/llms-txt.mdx, build/ai.mdx, zh/ |
| SEO head | src/starlight-overrides/Head.astro |
| Crawlers / sitemap hint | public/robots.txt |
| Agent discovery — well-known | public/.well-known/api-catalog, public/.well-known/mcp/server-card.json, public/.well-known/agent-skills/index.json, public/.well-known/ai-catalog.json, public/.well-known/oauth-protected-resource, public/.well-known/openid-configuration, public/.well-known/oauth-authorization-server, public/auth.md |
| Agent discovery — Link header | vercel.json (global /(.*) entry) |
| Markdown negotiation + WebMCP | src/middlewares/markdown-negotiation.ts, src/scripts/webmcp-register.ts, src/types/webmcp.d.ts |
| Agent discovery regression tests | tests/agent-discovery.test.ts |