LLM Wiki

Karpathy's LLM Wiki

Build and maintain a persistent, compounding knowledge base as interlinked markdown files. Based on Andrej Karpathy's LLM Wiki pattern.

Unlike traditional RAG (which rediscovers knowledge from scratch per query), the wiki compiles knowledge once and keeps it current. Cross-references are already there. Contradictions have already been flagged. Synthesis reflects everything ingested.

Division of labor: The human curates sources and directs analysis. The agent summarizes, cross-references, files, and maintains consistency.

When This Skill Activates

Use this skill when the user:

• Asks to create, build, or start a wiki or knowledge base
• Asks to ingest, add, or process a source into their wiki
• Asks a question and an existing wiki is present at the configured path
• Asks to lint, audit, or health-check their wiki
• References their wiki, knowledge base, or "notes" in a research context
• Wants to batch-ingest hundreds/thousands of documents into wiki format

Wiki Location

Configured via skills.config.wiki.path in ~/.hermes/config.yaml (prompted during hermes config migrate or hermes setup):

skills:
  config:
    wiki:
      path: ~/wiki

Falls back to ~/wiki default. The resolved path is injected when this skill loads — check the [Skill config: ...] block above for the active value.

Before setting wiki.path, verify the target actually exists. A stale default like ~/wiki is easy to leave behind even when the real wiki lives elsewhere.

For the workspace-hub multi-wiki layout, the preferred root is usually:

skills:
  config:
    wiki:
      path: /mnt/local-analysis/workspace-hub/knowledge/wikis

This points at the domain-wiki root (engineering/, marine-engineering/, maritime-law/, naval-architecture/, etc.) rather than a single flat ~/wiki folder.

The wiki is just a directory of markdown files — open it in Obsidian, VS Code, or any editor. No database, no special tooling required.

CLI Tool (workspace-hub)

The llm-wiki CLI at scripts/knowledge/llm_wiki.py provides 6 commands for operating wikis programmatically. All commands use the pattern:

uv run scripts/knowledge/llm_wiki.py <command> --wiki <domain>

batch-ingest is designed for scale:

• Checkpoint-based resume (.checkpoint.jsonl in wiki root)
• --dry-run for preview
• Progress reporting every batch
• Skips already-processed records
• Used to ingest 22K conference papers → 12K source pages in one run
• Proven: 100 records/batch, ~400 records per 10 seconds

Location: knowledge/wikis/<domain>/ (not ~/wiki). This is a multi-wiki ecosystem — multiple domain wikis coexist under knowledge/wikis/. Force-add to git despite .gitignore since wiki content is the compounding artifact.

Architecture

Multi-Wiki Pattern

In workspace-hub, wikis are organized as a multi-domain ecosystem under knowledge/wikis/<domain>/, not a single ~/wiki. Each domain (marine-engineering, maritime-law, naval-architecture) has its own complete three-layer structure. Cross-wiki linking connects related topics across domains.

Three Layers (per domain wiki)

knowledge/wikis/<domain>/
├── CLAUDE.md             # Schema: conventions, structure rules, domain config
├── raw/                  # Layer 1: Immutable source material
│   ├── papers/           # PDFs, standards, papers
│   ├── standards/        # Standards documents
│   ├── articles/         # Web articles, clippings
│   └── assets/           # Images, diagrams
└── wiki/                 # Layer 2: The LLM-maintained wiki
    ├── index.md          # Content catalog with sectioned entries
    ├── log.md            # Chronological action log (append-only)
    ├── overview.md       # Domain synthesis summary
    ├── entities/         # Entity pages (things: equipment, orgs, vessels)
    ├── concepts/         # Concept pages (ideas: methods, principles)
    ├── sources/          # Source summary pages (one per ingested document)
    ├── comparisons/      # Filed query outputs
    └── visualizations/   # matplotlib plots, Marp slide decks

Layer 1 — Raw Sources: Immutable. The agent reads but never modifies these. Layer 2 — The Wiki: Agent-owned markdown files. Created, updated, and cross-referenced by the agent. Layer 3 — The Schema: CLAUDE.md defines structure, conventions, and tag taxonomy.

Scaling Pattern (learned from 12K+ source ingestion)

• Metadata-first approach: Don't extract PDF content (hits 5-min timeouts on large files). Instead, read structured metadata (titles, topics, sizes) and create wiki source pages.
• Proven at scale: 22K conference metadata records → 12K unique source pages, skipping 10K+ duplicates via checkpoint file.
• Batch size: 100 records per batch, progress reported every batch.
• Checkpoint resume: .checkpoint.jsonl tracks processed records by unique ID.
• Index management: Updates index.md after each batch, not after every record.
• Git considerations: Wiki pages must be force-added (git add -f) even if .gitignore excludes the wikis directory. Wiki content is the compounding artifact.
• Workspace-hub hook gotcha: knowledge/wikis/<domain>/CLAUDE.md files are wiki schema/config files generated by llm-wiki init, not harness adapter files. If the repo hook .claude/hooks/check-claude-md-limits.sh applies the 20-line harness limit to all CLAUDE.md paths, commits touching wiki CLAUDE.md can fail with a false positive. The minimal safe fix is to exclude ^knowledge/wikis/ from that hook's staged-file filter so harness limits still apply to real adapter files while wiki schema files remain editable.

Resuming an Existing Wiki (CRITICAL — do this every session)

When the user has an existing wiki, always orient yourself before doing anything:

① Read CLAUDE.md (or SCHEMA.md) — understand the domain, conventions, and tag taxonomy. ② Read index.md — learn what pages exist and their summaries. ③ Scan recent log.md — read the last 20-30 entries to understand recent activity.

WIKI="${wiki_path:-$HOME/wiki}"
# Orientation reads at session start
read_file "$WIKI/SCHEMA.md"
read_file "$WIKI/index.md"
read_file "$WIKI/log.md" offset=<last 30 lines>

Only after orientation should you ingest, query, or lint. This prevents:

• Creating duplicate pages for entities that already exist
• Missing cross-references to existing content
• Contradicting the schema's conventions
• Repeating work already logged

For large wikis (100+ pages), also run a quick search_files for the topic at hand before creating anything new.

Initializing a New Wiki (via CLI)

uv run scripts/knowledge/llm_wiki.py init <domain>

This scaffolds the full three-layer structure, creates CLAUDE.md with domain-specific schema, initializes index.md and log.md, and creates the raw/ and wiki/ subdirectories.

After scaffolding:

1. Add some sources: ingest <file> --wiki <domain>
2. For bulk: batch-ingest metadata.jsonl --wiki <domain> --batch-size 100
3. Check health: lint --wiki <domain>

Core Operations

1. Ingest

When the user provides a source (URL, file, paste), integrate it into the wiki:

① Capture the raw source:

• URL → use web_extract to get markdown, save to raw/articles/
• PDF → use web_extract (handles PDFs), save to raw/papers/
• Pasted text → save to appropriate raw/ subdirectory
• Name the file descriptively: raw/articles/karpathy-llm-wiki-2026.md

② Discuss takeaways with the user — what's interesting, what matters for the domain. (Skip this in automated/cron contexts — proceed directly.)

③ Check what already exists — search index.md and use search_files to find existing pages for mentioned entities/concepts. This is the difference between a growing wiki and a pile of duplicates.

④ Write or update wiki pages:

• New entities/concepts: Create pages only if they meet the Page Thresholds in SCHEMA.md (2+ source mentions, or central to one source)
• Existing pages: Add new information, update facts, bump updated date. When new info contradicts existing content, follow the Update Policy.
• Cross-reference: Every new or updated page must link to at least 2 other pages via [[wikilinks]]. Check that existing pages link back.
• Tags: Only use tags from the taxonomy in SCHEMA.md

⑤ Update navigation:

• Add new pages to index.md under the correct section, alphabetically
• Update the "Total pages" count and "Last updated" date in index header
• Append to log.md: ## [YYYY-MM-DD] ingest | Source Title
• List every file created or updated in the log entry

⑥ Report what changed — list every file created or updated to the user.

A single source can trigger updates across 5-15 wiki pages. This is normal and desired — it's the compounding effect.

2. Query

When the user asks a question about the wiki's domain:

① Read index.md to identify relevant pages. ② For wikis with 100+ pages, also search_files across all .md files for key terms — the index alone may miss relevant content. ③ Read the relevant pages using read_file. ④ Synthesize an answer from the compiled knowledge. Cite the wiki pages you drew from: "Based on [[page-a]] and [[page-b]]..." ⑤ File valuable answers back — if the answer is a substantial comparison, deep dive, or novel synthesis, create a page in queries/ or comparisons/. Don't file trivial lookups — only answers that would be painful to re-derive. ⑥ Update log.md with the query and whether it was filed.

3. Lint

When the user asks to lint, health-check, or audit the wiki:

① Orphan pages: Find pages with no inbound [[wikilinks]] from other pages.

# Use execute_code for this — programmatic scan across all wiki pages
# Scan all .md files in entities/, concepts/, comparisons/, queries/
# Extract all [[wikilinks]] — build inbound link map
# Pages with zero inbound links are orphans

② Broken wikilinks: Find [[links]] that point to pages that don't exist.

③ Index completeness: Every wiki page should appear in index.md. Compare the filesystem against index entries.

④ Frontmatter validation: Every wiki page must have all required fields (title, created, updated, type, tags, sources). Tags must be in the taxonomy.

⑤ Stale content: Pages whose updated date is >90 days older than the most recent source that mentions the same entities.

⑥ Contradictions: Pages on the same topic with conflicting claims. Look for pages that share tags/entities but state different facts.

⑦ Page size: Flag pages over 200 lines — candidates for splitting.

⑧ Tag audit: List all tags in use, flag any not in the SCHEMA.md taxonomy.

⑨ Log rotation: If log.md exceeds 500 entries, rotate it.

⑩ Report findings with specific file paths and suggested actions, grouped by severity (broken links > orphans > stale content > style issues).

⑪ Append to log.md: ## [YYYY-MM-DD] lint | N issues found

4. Bulk Ingest (CLI-based)

For large-scale ingestion (100+ sources), use the llm-wiki batch-ingest CLI:

# Dry-run first to preview
uv run scripts/knowledge/llm_wiki.py batch-ingest metadata.jsonl --wiki <domain> --batch-size 100 --dry-run

# Then run for real (resume-safe via .checkpoint.jsonl)
uv run scripts/knowledge/llm_wiki.py batch-ingest metadata.jsonl --wiki <domain> --batch-size 100

The CLI handles:

• ✅ Checkpoint-based resume (safe to interrupt/restart)
• ✅ Progress reporting every batch — 100 records per batch at ~400 records per 10s
• ✅ Skip already-processed records (10K+ duplicates in a 22K run)
• ✅ Batch index.md/log.md updates (efficient)
• ✅ --dry-run mode to preview filenames and counts

5. Seed Migration (YAML → Wiki)

For structured YAML knowledge seeds in knowledge/seeds/:

1. Parse the YAML to extract entries
2. Group entries by domain/category
3. For each entry: create entity/concept/source page
4. Cross-link with existing wiki pages
5. Update index.md with all migrated entries
6. Append structured entries to log.md

Proven pattern: 18 mooring failure entries → 4 wiki pages (source + 2 concepts + 1 entity) 10 law cases + 6 conventions → 20 wiki pages with cross-references

6. Private client wiki draft → review submission

When working in a private client wiki such as llm-wiki-<client> and the user asks to draft a note/report "for review":

1. Save the draft under the existing client-wiki convention, usually reports/YYYY-MM-DD-<slug>.md unless the wiki schema says otherwise.
2. Keep the draft clearly marked with private/synthetic/client-specific frontmatter as applicable (visibility, client, synthetic, source_pages).
3. If the user later says the draft "looks good" and asks to "submit" it so they can approve it, do not treat that as external sending or final client publication. Instead, mark the artifact as submitted for internal/client review, for example review_status: submitted_for_<client>_review.
4. Add the artifact to the local review queue/index if one exists (for the simple private-wiki layout, add it to reports/README.md under a "Submitted for review" section).
5. Read back the touched files to verify the submission marker and queue entry landed before claiming it is submitted.
6. If the wiki uses PR-only/Deckhand guardrails, avoid direct push claims unless the approved workflow explicitly authorizes GitHub write actions; local review-queue submission is still useful and auditable.

Working with the Wiki

Searching

# Find pages by content
search_files "transformer" path="$WIKI" file_glob="*.md"

# Find pages by filename
search_files "*.md" target="files" path="$WIKI"

# Find pages by tag
search_files "tags:.*alignment" path="$WIKI" file_glob="*.md"

# Recent activity
read_file "$WIKI/log.md" offset=<last 20 lines>

Seed Migration Pattern

For structured YAML seeds (like knowledge/seeds/naval-architecture-resources.yaml):

1. Parse YAML categories (textbooks, hydrostatics, portals, ship plans, etc.)
2. Create concept pages for each major topic domain (5-6 pages)
3. Create source pages for each individual resource (17-36 pages per seed)
4. Update index.md with structured tables
5. Update log.md with migration entry

This approach is much faster than full PDF extraction and creates structured wiki pages that can be enhanced later with LLM content.

Cross-Wiki Linking

When managing multiple domain wikis, look for natural connections:

• Deepwater Horizon → marine-engineering (lng-carrier-mooring) + maritime-law (OPA 90)
• Mooring failures → marine-eng + naval-arch stability concepts
• Classification rules → naval-arch + maritime-law liability conventions

Obsidian Integration

The wiki directory works as an Obsidian vault out of the box:

• [[wikilinks]] render as clickable links
• Graph View visualizes the knowledge network
• YAML frontmatter powers Dataview queries
• The raw/assets/ folder holds images referenced via ![[image.png]]

For best results:

• Set Obsidian's attachment folder to raw/assets/
• Enable "Wikilinks" in Obsidian settings (usually on by default)
• Install Dataview plugin for queries like TABLE tags FROM "entities" WHERE contains(tags, "company")

Pitfalls

• Automated wiki health cron gotchas — For workspace-hub scripts/knowledge/wiki_health_cron.py runs, verify the domain list before trusting the report. Hidden/operational directories under knowledge/wikis/ (for example .planning/) are not domain wikis and should be skipped. Directories that contain only CLAUDE.md or otherwise lack wiki/ should be reported as critical missing scaffold/index/log issues, not allowed to crash the cron before reports are written. Health reports under knowledge/wikis/health-reports/ may be ignored by .gitignore; use git add -f knowledge/wikis/health-reports/health-YYYY-MM-DD.{json,md} when committing them. For scheduled reports, compare against the previous JSON report to identify new/resolved issues, stage only the generated report files, and verify HEAD/origin/main after push; see references/wiki-health-cron-reporting.md.
• Never modify files in raw/ — sources are immutable. Corrections go in wiki pages.
• Always orient first — read SCHEMA + index + recent log before any operation in a new session. Skipping this causes duplicates and missed cross-references.
• Always update index.md and log.md — skipping this makes the wiki degrade. These are the navigational backbone.
• Don't create pages for passing mentions — follow the Page Thresholds in SCHEMA.md. A name appearing once in a footnote doesn't warrant an entity page.
• Don't create pages without cross-references — isolated pages are invisible. Every page must link to at least 2 other pages.
• Frontmatter is required — it enables search, filtering, and staleness detection.
• Tags must come from the taxonomy — freeform tags decay into noise. Add new tags to SCHEMA.md first, then use them.
• Keep pages scannable — a wiki page should be readable in 30 seconds. Split pages over 200 lines. Move detailed analysis to dedicated deep-dive pages.
• Ask before mass-updating — if an ingest would touch 10+ existing pages, confirm the scope with the user first.
• Rotate the log — when log.md exceeds 500 entries, rename it log-YYYY.md and start fresh. The agent should check log size during lint.
• Handle contradictions explicitly — don't silently overwrite. Note both claims with dates, mark in frontmatter, flag for user review.
• PDF extraction timeout — Large PDFs hit 5-min sandbox timeouts. Use metadata-first approach for speed. Full extraction = enhancement when async workers exist.
• Git force-add required — Wiki dirs may be in .gitignore. Use git add -f to commit content. Wiki content is the compounding artifact and must be tracked.
• Batch size trade-off — 100 records/batch balances speed with index update frequency. Smaller = more frequent updates, slower. Larger = fewer index commits, risk more on crash.
• Low link density early — Newly created wikis naturally have low link density. This resolves as cross-references grow during normal ingest operations.
• YAML seed migration is fast — Converting structured YAML to wiki pages is 10x faster than PDF extraction. Use this pattern whenever seeds exist.
• Bulk local-corpus indexing is link-only by default — When a large local corpus is already materialized outside git (for example under /mnt/ace/<repo>/<domain>/...), do not copy raw files into knowledge/wikis/<domain>/raw/ or create one source page per file. Treat the parent path as source-of-record, generate deterministic inventory/classification artifacts under .planning/intel/<corpus>-to-llm-wiki/, create one bucket-level batch-ingest record per logical corpus bucket, and defer deep extraction to a separate follow-up issue. Keep a full JSONL inventory for drilldown, a domain classification TSV, per-domain batch JSONL files, a deep-extraction candidate TSV, and a final ingest report.
• Curated deep extraction follows metadata-first indexing — When the user later approves deep extraction for selected high-value buckets from an external/local corpus, keep the extraction bounded to a few small/high-signal corpora rather than expanding the whole data lake. Create an approved follow-up issue/plan marker, write deterministic extraction scripts under .planning/intel/<corpus>-deep-extraction/, and keep raw source files link-only under their /mnt/ace/... source-of-record paths. Useful first-pass extractors: pdftotext -layout for PDFs; openpyxl.load_workbook(..., keep_vba=True, data_only=False) for workbook structure, labels, and formula samples without copying XLSM data into wiki; simple content sniffing for misleading extensions; and small deterministic parsers for GIS/CAD metadata (projection WKT, ASCII-grid DEM headers/stats, DXF entity counts). Create concise source summary pages plus concept/workflow pages, then update index.md and log.md so new pages are not orphaned. Validation should include llm_wiki.py status and lint for every touched domain plus the relevant pytest suite; document pre-existing lint warnings separately from warnings introduced by the new extraction.
• Validate and commit ignored domain wikis carefully — For workspace-hub multi-wiki work, initialize missing domains with uv run scripts/knowledge/llm_wiki.py init <domain>, run batch-ingest dry-run then real-run, backfill sources: frontmatter on newly created source pages with the external source-of-record path, and run status/lint for every touched wiki. New domain directories may be ignored by /knowledge/wikis/*; use git add -f only for intended wiki metadata artifacts. search_files may not surface ignored/untracked pages, so validate newly created domain pages with direct find/path checks. Track .checkpoint.jsonl so re-runs skip processed bucket records, check staged blob sizes before commit, and make sure no raw bulk data is staged. For deep-extraction follow-ups, also force-add newly created ignored concept/source/workflow pages and validation logs only after checking staged file sizes and confirming unrelated dirty files remain unstaged.
• Engineering implementation knowledge packs — When an engineering-critical implementation is blocked on plan approval and the user asks to preserve raw references before coding, treat the wiki update as a bounded pre-implementation knowledge pack: read the domain CLAUDE.md/index.md/log.md, use subagents or parallel extraction for high-value PDFs/wiki pages, create concise concept/source/comparison pages that preserve formulas, sign conventions, input/output scope, limitations, and future-extension guardrails, then update the issue plan and GitHub issue with the exact wiki anchors. Keep raw files at /mnt/ace/... as source-of-record; do not copy standards/textbooks into git. Example session pattern: references/yaw-moment-raw-reference-ingestion.md.
• Public spinout completion work uses metadata-only scorecards and portals first — When improving a public llm-wiki spinout from a private/control-plane workspace, start with deterministic repo-local scoring before touching content: inventory all domains, count curated vs source-summary pages, check navigation/frontmatter/index/orphan signals, and generate a human+JSON scorecard under docs/reports/. For very large source-heavy domains, add a faceted wiki/portal.md and index link rather than bulk-editing stale overview pages that may overlap approval-gated work. Generated public artifacts must not preserve private mount paths, raw archive paths, vendor PDF text, or secrets; scan the exact staged files for private path strings and token patterns, inspect diffs, validate JSON/links/whitespace, then commit/push and post body-file evidence comments to the driving GitHub issues. See references/public-spinout-scorecard-portal-closeout.md.
• Practical completion roadmap from private corpus + issue history — When the user asks to make a public llm-wiki repo complete using /mnt/ace or another private/local corpus, synthesize a roadmap rather than bulk ingesting: current scorecard, live GitHub issue portfolio, sanitized source-family inventory, tier-1 repo links, explicit Data-bank/Software/Results/github-issues layers, public-safety boundaries, and narrow provider/Codex execution lanes. See references/practical-completion-roadmap-from-private-corpus.md.