LLM Wiki

LLM Wiki — Karpathy Knowledge Base Pattern

Experimental skill — iterating. Authored by Lewis Liu (lylewis@outlook.com) · Inspired by Karpathy's llm-wiki Gist

Core idea

Rather than depending on RAG that re-retrieves raw docs on every query, the LLM compiles raw sources into a persistent, cross-linked wiki. An optional supported RAG/hybrid adapter may select context, but canonical pages and provenance remain authoritative. Every ingest, query, lint, and audit pass makes the wiki richer. Knowledge compounds — and the human stays in the loop through a structured feedback channel instead of ad-hoc corrections that get lost.

• You own: sourcing raw material, asking good questions, steering direction, filing feedback on anything the AI got wrong.
• LLM owns: authored analysis, cross-referencing, filing, bookkeeping, and acting on your feedback. Project-history source pages are deterministic compiler output, not agent-authored summaries.

Every session starts by reading purpose.md, schema.md, and wiki/index.md.

One layout

There is one layout and no profile to pick. docs/specs/llm-wiki-app-independence-decision.md settled that, along with the rule underneath it: nothing here depends on any application. No install step, no daemon, no private state directory, no HTTP API, no third-party Python package. The scripts read and write plain Markdown with the standard library. A wiki produced here opens in Obsidian, in a text editor, and in anything else that reads Markdown, but requires none of them.

<wiki-root>/
├── llm-wiki-project.json  ← Binding: which project this wiki is the history of (optional)
├── purpose.md             ← Scope, goal, key questions, thesis
├── schema.md              ← Page types, naming, front matter, index and log format
├── audit/                 ← Human corrections, one file each — the human-to-agent channel
│   └── resolved/          ← Applied or rejected, with a resolution appended
├── raw/                   ← Source material (the LLM reads, never rewrites)
│   ├── sources/           ← Articles, papers, notes, ingested repository docs
│   ├── refs/              ← Pointer files for anything too large to copy
│   └── assets/            ← Images and binaries a page cites
└── wiki/                  ← LLM-generated knowledge (the LLM writes, you read)
    ├── index.md           ← Master catalog — every page, exactly once
    ├── log.md             ← Operation log, newest first
    ├── concepts/          ← Ideas, techniques, phenomena (split when >1200 words)
    ├── entities/          ← People, tools, papers, organisations
    ├── sources/           ← One page per ingested source
    ├── queries/           ← Answers to questions asked of the wiki
    ├── comparisons/       ← Side-by-side analysis of related entities
    ├── synthesis/         ← Cross-cutting conclusions
    └── timeline/          ← The temporal axis
        └── tickets/       ← One lifecycle record per ticket

purpose.md and schema.md are the two configuration files, and both are read at the start of every session. purpose.md says what the wiki is for; schema.md says how it is shaped. Read references/schema-guide.md for what goes in each.

audit/ sits at the root, outside wiki/, because it is the one channel that runs human to agent — everything else in the tree runs the other way.

Core principles

Four rules govern everything below. If a future instruction contradicts one, flag it to the user before acting.

1. Divide and conquer

A single concept page should never try to cover a complex topic end-to-end. Target: 400–1200 words per page. When a topic would blow past that:

• Create a subfolder: wiki/concepts/<topic>/
• Put a short index page at wiki/concepts/<topic>/index.md — definition, list of sub-pages, one-line summaries
• Put each aspect in its own file: wiki/concepts/<topic>/<aspect>.md
• In wiki/index.md, show the hierarchy through indented bullets

Example layout (from a real wiki):

wiki/concepts/claude-code/
├── index.md                  (overview + links to sub-pages)
├── architecture.md
├── agent-framework.md
├── bridge-system.md
├── query-engine.md
├── skills-plugins.md
├── state-management.md
└── tool-system.md

One fat file covering all seven aspects would be unreadable and unlinkable. Seven focused files plus an index page give you navigation, selective reading, clean backlinks, and small audit targets.

2. Mermaid for diagrams, KaTeX for formulas

These authoring rules do not rewrite literal project-source payloads or their examples.

• Any flow, sequence, hierarchy, or state diagram must be written in mermaid — never ASCII art. ASCII boxes rot fast and are impossible to annotate.

  ```mermaid
  flowchart LR
      A[raw/sources/article] --> B[wiki/sources page]
      B --> C[concept page]
      C --> D[index.md]
  ```
  

• Any formula must be written in KaTeX: inline or block.

Both render in Obsidian with default settings.

3. Raw file policy

Small text-based sources (md, txt, small PDFs, small images) → copy into raw/sources/. Images and binaries a page cites → raw/assets/.

Large binaries (videos, model weights, installers, datasets, PDFs over 10 MB) → do not copy. Instead:

• Create a pointer file at raw/refs/<slug>.md with:

  ---
  kind: ref
  external_path: /Volumes/external/models/llama-3-70b/
  size: ~140 GB
  ---
  

  followed by a short description of what it is and why it matters to this wiki.
• Wiki pages cite [[raw/refs/<slug>]] exactly like any other source.

This keeps the wiki portable, and keeps it small enough to be tracked in Git if you want it tracked. Whether the wiki or the project's docs/ are tracked at all is a per-project choice, and no operation here assumes either way.

4. Audit is the human feedback surface

The wiki is AI-written; it will be wrong sometimes. The raw sources are human-written; they will contradict each other. audit/ is how humans correct both without losing the corrections in chat history.

• One correction is one file in audit/, with YAML front matter (anchor, target, severity) and a Markdown body. A human writes it in any editor.
• The AI must periodically run the audit op — never silently ignore audit/*.md.
• When a correction is applied, the file moves to audit/resolved/ with a # Resolution section appended and an entry recorded in wiki/log.md.
• Nothing in audit/ is ever deleted. A rejected correction is archived with its rejection rationale, which is the valuable part.

See references/audit-guide.md for the file format and the anchor strategy.

The five operations

Every action on the wiki is one of these six, and each mutation appends one entry to wiki/log.md.

1. compile

(Re)structure wiki content from existing raw/ material — splitting oversized pages, merging near-duplicates, rebuilding index.md.

When to run: after a big ingest batch, when a page has outgrown 1200 words, when index.md no longer reflects reality, or when the user says "clean up the wiki".

Steps:

1. Read purpose.md, schema.md, wiki/index.md, and every file in the target subtree.
2. For each page over ~1200 words: plan a split into concepts/<topic>/ with an index plus sub-pages. Confirm the plan with the user before writing.
3. For each pair of near-duplicate pages: propose a merge. Confirm, then rewrite.
4. Regenerate wiki/index.md so every page is listed exactly once.
5. Log: - HH:MM compile — <files touched, splits, merges>

2. ingest

Add an external article or raw source. One source typically touches 5–15 wiki pages. For configured repository documents, use scripts/ingest_docs.py instead: preserve their complete text through the project-source projection contract, not the summary-oriented article steps below.

Steps:

1. Save the source to raw/sources/<slug>.md — or raw/refs/<slug>.md as a pointer if it is large (see the raw file policy).
2. Read the source in full.
3. Create wiki/sources/<slug>.md (200–400 words — key takeaways, not a rewrite; see references/article-guide.md).
4. Create or update the relevant pages in wiki/concepts/. Respect divide-and-conquer: split rather than cram.
5. Create or update pages in wiki/entities/ for any new people, tools, papers, or organisations referenced.
6. Update wiki/index.md so the new pages appear under the right heading.
7. Log: - HH:MM ingest — <slug>, <one line> (N pages)

3. query

Answer a question grounded in the wiki, not in general knowledge.

Steps:

1. Probe the supported query mode. Use RAG/hybrid retrieval only when the active skill contract and selected bound wiki configuration both name the same supported adapter, and its tool or command is already usable within the request's privacy, network, dependency, and authority boundary. An optional recipe, installed package or binary, MCP declaration, cache/index, benchmark, or roadmap is not availability. Do not install, download, start, configure, or authenticate a retrieval component; the mode probe and fallback selection are non-mutating.
2. If that binding is valid, use it to select context and state rag/hybrid:<adapter-id> plus its non-secret binding identifier. Otherwise state compiled-markdown plus a concrete reason such as no-supported-rag-binding, then read wiki/index.md and scan for relevant pages by category.
3. Read the identified canonical pages in full; follow one level of explicit wikilinks. Treat chunks, scores, vectors, embeddings, graph projections, and caches only as derived context.
4. If the wiki does not have enough material, say so and suggest what to ingest next instead of inventing an answer.
5. Synthesize the answer, citing pages inline with [[page-slug]]; follow their provenance to primary sources before making material factual claims.
6. For read-only research, return the answer without modifying the wiki. Only when durable filing is requested and authorized, save to wiki/queries/<YYYY-MM-DD>-<question-slug>.md, include the query mode in the body unless the active schema defines a field, and list it in wiki/index.md.
7. If the filed answer is durable — a comparison, an analysis, a new synthesis — promote a cleaned-up version to wiki/comparisons/ or wiki/synthesis/.
8. For a durable filing, log - HH:MM query — <question-slug>, plus a separate - HH:MM promote — ... line if promoted.

4. lint

Health check, sixteen passes:

python3 scripts/lint_wiki.py <wiki-root>

Structural passes, over the wiki alone:

Drift passes, over the relationship between a page and the artefact it came from. These need a project binding; without one they report that they do not apply rather than reporting green.

semantic-coverage independently checks current configured source identity, kind and digest, required topic records and visible navigation, exact payload reconstruction, part ownership and order, and the 32 KiB page bound. Current metadata or arbitrary prose is insufficient. Present legacy metadata-only pages fail; regenerate through ingest_docs.py. Correct source, binding or ownership errors first: lint never repairs content. Missing-source tombstones are counted separately without a semantic freshness/coverage claim. With no binding, project-drift and semantic-coverage report informational not-applicable results; an invalid existing binding is an error, not an exemption.

Three properties of this design are worth knowing before reading its output.

Every pass reports rather than skips. A missing directory is an issue in layout, not a silent success in the pass that would have read it. A pass that cannot fail is worse than no pass, because it reports green and is believed.

Severity is not decoration. An error is a corruption or a broken reference and sets the exit code. A warning is a real signal you may reasonably defer. info is the normal steady state — un-ingested-artefact fires on everything added since the last ingest. Reporting all three at one volume teaches the reader to skip the output, so lint exits 0 when there are no errors even if warnings remain.

No pass assumes Git. A project's docs/ may be untracked and the host may not be a repository; both are supported. A missing history is not drift, so provenance-validity checks that a date's rung is coherent and never that it is a good one. An mtime date is a valid date.

Each pass prints the repair to propose. Propose it, confirm with the user, then apply. Log: - HH:MM lint — N errors, M warnings, K fixed.

5. audit

Apply human feedback from audit/.

Steps:

1. Run python3 scripts/audit_review.py <wiki-root> --open for a list grouped by target file.
2. For each open correction, read the file. Use the anchor_before / anchor_text / anchor_after window to locate the exact range in the target — line numbers drift.
3. Decide: accept (apply it), partially accept (apply what holds, note the rest), reject (say why — the feedback may rest on a misreading of scope or a contradictory source), or defer (add it to purpose.md under key questions and leave the correction in place).
4. Append a # Resolution section to the correction:

   # Resolution
   
   2026-04-10 · accepted.
   Fixed the file count (was "~1,900", corrected to "~1,800" per commit abc123).
   Updated: concepts/claude-code/architecture.md lines 47–48.
   

5. Move the file from audit/ to audit/resolved/. Filename unchanged.
6. Log per resolved correction: - HH:MM audit — resolved 20260409-143022-a1b2, <one line>
7. Never delete a correction. Rejected ones go to resolved/ too.

See references/audit-guide.md for the full format.

6. sync-project

Compile one existing project-bound wiki through the fail-closed wiki-sync-v1 boundary:

python3 scripts/sync_project.py <project-root> --wiki-root <wiki-root> --json

The explicit root is required for an external wiki. Without it, discovery checks only the project root and its direct children. No compatible wiki returns skipped/absent; it never scaffolds one. auto_sync: disabled in llm-wiki-project.json is a durable skip, while a missing auto_sync value means enabled for backward compatibility.

Ingest, timeline rebuild, generated-scope validation, and the full lint run happen in a staging copy. Only regular non-executable UTF-8 wiki/**/*.md may change. The root index is mixed-ownership: compilation may replace only its exact project, session, and timeline marker blocks; every byte outside them is human-owned. Missing, duplicate, malformed, nested, or conflicting boundaries return failed/compile before protected-wiki application. An external or internal-untracked wiki is updated directly after compare-and-swap. An internal tracked wiki returns a frozen wiki-sync-v1 candidate; this skill does not commit, deliver, or merge it. The result always carries one normalized status/reason, a fresh WikiSyncRef, origin provenance, changed paths, and deterministic validation evidence capped at implementation-complete.

The post-integration owner may pass --source-root <detached-worktree> --expected-source-head <sha> so compilation reads the exact integrated docs while discovery, tracking classification, and publication remain bound to the canonical project root. The alternate checkout must belong to the same Git common directory and match the expected head; otherwise synchronization fails closed without touching the wiki.

Project history

A wiki bound to a project — through llm-wiki-project.json at its root — additionally compiles that project's own docs/ and the agent sessions that worked on it. The scripts are self-documenting; run each with --help.

For project-source compilation or inspection, read semantic projection: every normalized source character is retained, pages are at most 32 KiB including framing, missing headings are reported truthfully, and unchanged replay writes nothing. This is separate from authored articles and session digests. semantic-coverage checks the projection against those configured sources; structural/freshness passes alone do not attest complete semantic payloads.

Three rules hold across all of them, and all are worth knowing before reading any page they produce:

• A page is named from the artefact's identity, never from its path. Moving a ticket into done/ updates its page; it does not mint a second one.
• Every date carries the rung that produced it, from git-rename down to unknown. A date with no witness renders as the word unknown with its reason, never as a plausible value.
• The compiler does not own the complete root index. It updates only exact generated ownership blocks; headings, links, and prose outside those blocks are preserved byte-for-byte.

wiki/index.md format

index.md is a mixed-ownership catalog. Scaffold writes one exact marker pair for each compiler owner: project-sources, session-sources, and timeline. Direct ingest and sync-project update those blocks in place and never infer ownership from a heading. An unchanged compile writes no index bytes.

A pre-WS-08 index with no markers is intentionally not migrated automatically. Adoption is a separate, explicit transaction: freeze the legacy file's SHA-256, review exact byte offsets for all three generated regions, then invoke the dedicated command with one ordered span per owner:

python3 scripts/adopt_root_catalog.py wiki/index.md \
  --expected-sha256 <64-lowercase-hex> \
  --span project-sources:<start>:<end> \
  --span session-sources:<start>:<end> \
  --span timeline:<start>:<end> \
  --json

Offsets are byte offsets into the exact unmarked UTF-8 file and must be complete, ordered, non-overlapping line boundaries. The command never infers spans from headings. It validates the digest, map, UTF-8, path type, resulting parser state, and marker-removal round trip before an atomic mode-preserving replacement. Exact replay writes nothing. Run full staged sync and lint, inspect the diff, and only then publish through the repository's normal candidate boundary. If ownership is uncertain, stop instead of marking an arbitrary section.

# Index — <Topic>

> One-sentence scope of the wiki.

## Navigation

[[#Concepts]] · [[#Entities]] · [[#Sources]] · [[#Queries]] · [[#Synthesis]] · [[#Timeline]]

## Concepts

### <Category A>
- [[concepts/foo]] — one-line summary
- [[concepts/bar/index|bar]] — (folder-split) one-line summary
    - [[concepts/bar/aspect-1]] — ...

## Entities
- [[entities/andrej-karpathy]] — AI researcher, author of the llm-wiki pattern

## Sources
- 2026-04-09 — [[sources/llm-wiki-gist]] — Karpathy's original Gist

## Timeline
- [[timeline/index]] — when things happened, and how each date is known

## Open Questions
- Q1: ...

Rules:

• Every wiki page appears exactly once in index.md. lint enforces it.
• Folder-split concepts show hierarchy through indented bullets.
• index.md is a catalog, not a citation. A page also needs a real inbound link from another page.
• Catalogs nest. wiki/index.md lists [[timeline/index]], and wiki/timeline/index.md lists the pages beneath it. index-drift accepts a page catalogued at any level, and reports a catalog that no other catalog links — because everything under an unreachable catalog is unreachable too.
• Every index.md, at any depth, and wiki/log.md are machinery rather than pages: none is catalogued, and none is expected to have inbound links.

wiki/log.md format

One file, newest first. See references/log-guide.md for the full convention.

# Log — <Topic>

## 2026-04-10

- 14:30 audit — resolved 20260409-143022-a1b2, corrected the file count
- 09:15 ingest — llm-wiki-gist, Karpathy's original (7 pages)

## 2026-04-09

- 16:02 scaffold — created the wiki tree

• H1 is the title; each H2 is one ISO date; dates run newest first.
• Each entry is - HH:MM <op> <description>.
• Ops: compile, ingest, query, lint, audit, promote, split, scaffold, ingest-docs, timeline, sessions, sync-project.
• Quick scan of recent history: grep -m 20 '^- ' wiki/log.md.

The log is one file rather than one file per day because the operations that append to it are agent-driven and serialized, so the concurrent-write problem a per-day directory solves does not arise. It lives inside wiki/ so it is part of the browsable wiki rather than a sidecar.

Tooling

Starting a new wiki

python3 scripts/scaffold.py <wiki-root> "<Topic Title>"
python3 scripts/scaffold.py <wiki-root> "<Topic Title>" --project-root <path-to-project>

Creates the tree above, with purpose.md, schema.md, wiki/index.md, wiki/log.md and audit/README.md filled from templates. Pass --project-root to record which project the wiki is the history of; every project-history operation needs that binding. A freshly scaffolded wiki passes lint with zero errors or warnings; an unbound wiki reports that project drift and semantic coverage do not apply. Errors on a fresh scaffold mean the scaffold and lint have drifted apart, not that the user must repair generated scaffolding.

After scaffolding:

1. Fill in purpose.md — scope, key questions, thesis.
2. Adjust schema.md if this wiki needs page types the default set does not cover.
3. Start ingesting.
4. Run lint periodically.
5. Run audit whenever corrections accumulate.

Interpreter. The commands above say python3, which is the convention across this repository. On Windows python3 may resolve to a Microsoft Store alias that prints an install prompt instead of running Python; use python there. The scripts need nothing beyond CPython 3.10 or later.

Use cases

• Research deep-dive — reading papers and articles on a topic over weeks; the wiki evolves with your understanding, and the audit trail keeps AI mistakes from silently accumulating
• Project history — a repository's specs, tickets and agent sessions compiled into a dated account of what was decided and when, with every date carrying its provenance
• Personal wiki — notes and ideas compiled into a personal encyclopedia; comment on anything you disagree with later, the AI corrects it
• Team knowledge base — fed by threads, meeting notes and docs; corrections filed as files
• Reading companion — filing each chapter as you go builds a companion wiki by the end

References

• references/schema-guide.md — What goes in purpose.md and schema.md
• references/article-guide.md — How to write a good page (length, wikilinks, mermaid, math, divide-and-conquer)
• references/log-guide.md — The wiki/log.md convention
• references/audit-guide.md — Correction format, anchor strategy, processing workflow
• references/tooling-tips.md — Obsidian setup, capturing sources, optional semantic search