Knowledge Base Builder

Knowledge Base Builder

Standing Mandates

• Organize for the user's likely lookup jobs, not for decorative hierarchy or a mechanical mirror of the source tree.
• Keep Markdown notes and catalog lookup as the operating model while the corpus remains practical to maintain and query selectively; hand off to RAG when it does not.
• Keep folders shallow, place each note at one canonical path, and use controlled metadata for cross-cutting retrieval.
• Preserve stable note IDs and exact source references so path changes do not break catalog lookup or later RAG citations.
• For non-trivial vaults, emit _knowledge/catalog.jsonl; keep note bodies, not catalog summaries, as the authoritative corpus.
• Preserve a coherent existing topology unless restructuring is requested or it demonstrably blocks retrieval.
• Preserve evidence-based lessons from meaningful build or update work in _knowledge/improvement-notes.md so the next builder pass can improve retrieval without rediscovering the same friction.
• Treat answerability as a completion gate, separate from structural cleanliness. A non-trivial vault is incomplete while any declared competency question is partial or unanswerable.
• Treat the claim, not the entity, as the atomic unit. Preserve relation claims as first-class notes when a comparison, equivalence, or sequence is the knowledge users need.

Quick Intake

Before building, ask a lightweight "what is this?" intake when the source purpose is not already obvious from the user's request or repository context. Keep it short:

• What kind of material is this? codebase, docs, notes, tickets, research, mixed?
• Who will browse the vault, and what should they be able to understand or do?
• Should the output optimize for onboarding, architecture navigation, research synthesis, operations, or long-term knowledge maintenance?
• Does an existing folder or naming convention need to be preserved?

If the user gives a partial answer, proceed with reasonable defaults and record assumptions in vault-plan.md or open-questions.md. Do not block on exhaustive taxonomy decisions before reading representative sources.

Process

1. Define the vault boundary. Identify the source directories, document sets, or pasted materials to include. Ask only when the boundary is ambiguous or destructive file placement is possible; otherwise create a clear output folder such as knowledge-base/ or use the user's requested vault path.
2. Define likely lookup jobs and competency questions. Capture the names, aliases, operator terms, source symbols, entities, domains, and question types an AI or human reader will use. For a non-trivial vault, create _knowledge/questions.jsonl from these jobs before treating extraction as complete; do not replace a difficult failed question with an easier one.
3. Inspect existing topology and memory. Before placing notes, inspect existing folders, naming rules, indexes, MOCs, links, and _knowledge/improvement-notes.md when present. Preserve a coherent existing convention unless the user requested restructuring or it demonstrably blocks retrieval.
4. Choose a folder strategy. Use the shallowest hierarchy that predictably narrows candidate notes. Prefer a flat notes/ folder for small or single-domain vaults and notes/<domain>/ for stable multi-domain corpora. Keep cross-cutting classifications in frontmatter rather than duplicating notes across folders.
5. Create a vault plan for non-trivial corpora. For multi-folder repos, mixed document sets, or long-lived vaults, write vault-plan.md before creating notes. Include audience, included/excluded sources, lookup jobs, folder placement rules, note types, naming rules, source coverage target, navigation surfaces, and the conditions for later RAG conversion.
6. Inventory the sources. Read representative files before designing the structure. For codebases, map modules, public entry points, configuration, runtime flows, tests, and docs. For mixed materials, group by domain, project, decision, people/process, and open questions.
7. Choose note types. Prefer a small set of reusable note shapes:
   • Concept notes for domain ideas and vocabulary.
   • Code notes for modules, APIs, services, commands, schemas, and data flows.
   • Decision notes for tradeoffs, ADR-like choices, constraints, and rejected alternatives.
   • Workflow notes for procedures, runbooks, onboarding paths, and repeated tasks.
   • Relation notes for durable contrasts, equivalences, and sequences across multiple participants.
   • MOC notes for navigation hubs that connect related notes.
8. Write atomic claims. Keep each note centered on one durable claim, idea, component, relation, or decision. Split notes when the title needs "and" only when the conjunction joins independent claims, unrelated readers would want only half, or backlinks would point to different concepts. A single note may span multiple entities when its atomic claim is the relation between them. Relation notes require all participants and evidence for every side.
9. Add retrieval metadata and vocabulary bridges. Give each note a stable id, explicit title, type, domain, controlled tags, useful aliases, important entities, status, and sources. Keep same-referent aliases in aliases; use user_terms, source_symbols, and lookup_layers for operator/UI-to-code/database bridges. Treat sources as the canonical provenance field.
10. Link and ground deliberately. Use Obsidian-style wikilinks ([[Note Title]]) for durable concepts and relative Markdown links for files that should open directly from the repository. Include exact source references and mark uncertain inferences as Open Questions instead of presenting them as facts.
11. Create navigation and lookup surfaces. Add index.md for readers, focused MOCs where they improve traversal, and _knowledge/catalog.jsonl for fast AI candidate discovery. The catalog should map stable IDs and retrieval metadata to current paths without copying entire note bodies.
12. Prepare the RAG handoff when relevant. Keep notes as the source-grounded corpus layer. Record whether headings are suitable chunk boundaries, whether metadata and permissions can propagate to chunks, and which sources are stale, conflicting, or missing before routing to knowledge:rag-corpus-builder.
13. Run the answerability gate. Execute every declared competency question through knowledge:knowledge-query, write _knowledge/question-results.jsonl, and summarize complete/partial/unanswerable counts in _knowledge/coverage.md. A question passes only when the required notes and direct evidence support every material part. Run the validator from references/answerability-contract.md; any partial or unanswerable result leaves the build incomplete and becomes the next extraction or correction task.
14. Check structural and claim quality. Inspect folder placement, catalog coverage, vocabulary lookup, orphan notes, duplicate IDs or titles, missing source references, dead links, oversized notes, and unsupported synthesis. For relation notes, compare each participant against its own cited anchor before accepting the relation claim.
15. Capture improvement memory. After meaningful build or update work, append only observed retrieval friction, structural decisions, failed competency questions, remaining manual steps, and evidence-backed next improvements to _knowledge/improvement-notes.md. Use the format in references/retrieval-layout.md; preserve prior entries and do not turn it into a generic activity log.

Output Template

A useful knowledge-base delivery includes:

Folder Strategy

Folders reduce the search space; metadata handles overlapping classifications. Before creating a non-trivial folder layout, catalog, or RAG handoff, read references/retrieval-layout.md. It defines placement choices, move invariants, catalog records, and metadata propagation.

Note Templates

Each note should generally use this base shape:

---
id: payment-authorization
title: Payment Authorization
type: concept
domain: billing
status: verified
confidence: direct
tags:
  - payment
  - authorization
aliases:
  - 결제 승인
  - payment approval
entities:
  - PaymentService
  - PaymentGateway
sources:
  - path/to/source.ext
related:
  - retry-policy
---

# Note Title

One-paragraph summary of the durable idea.

## Key Points

- Specific, source-grounded point.
- Link to related notes with [[Useful Wikilinks]].

## Related

- [[Related Note]]

## Sources

- `path/to/source.ext`

Adapt the sections by note type:

Linking Conventions

• Use stable, human-readable note titles: Payment Authorization Flow, not payment-auth-flow-notes-final.
• Add aliases in frontmatter only when they materially improve search or preserve existing terminology.
• Prefer links between notes over duplicating the same explanation in multiple places.
• Preserve repository paths exactly in source references.
• When a note is generated from code, include the key symbols, files, commands, or tests that justify the summary.
• Keep important notes reachable from index.md or a MOC within two clicks.
• Avoid orphan notes unless they are intentionally listed in open-questions.md as isolated findings.

Provenance Levels

Use confidence in frontmatter to make source strength explicit:

For inferred, uncertain, or stale-risk, add a short explanation in Open Questions or the note body. Do not present uncertain synthesis as fact.

Frontmatter for Retrieval Exports

Keep the base note frontmatter above simple and stable so catalog, graph, and later RAG exports can reuse it without reclassification.

Useful fields:

AI Lookup Catalog

For non-trivial vaults, emit one record per note in _knowledge/catalog.jsonl using the contracts in references/retrieval-layout.md and references/answerability-contract.md. Keep summaries short enough for candidate selection, paths current, IDs stable, and terms grounded in language users actually search. Include relation-side evidence and vocabulary bridge fields where applicable. knowledge:knowledge-query should search the catalog first and open only a small candidate set.

RAG Handoff

Use knowledge:rag-corpus-builder when the user requests embeddings, a vector index, chunk-level retrieval, or the vault has outgrown efficient targeted note lookup.

Follow the handoff contract in references/retrieval-layout.md: note bodies remain corpus input, catalog/frontmatter provide identity and metadata, semantic headings become preferred chunk boundaries, and unresolved quality or access issues remain visible in the ingestion report.

Quality Bar

• The vault can be browsed from index.md without knowing the source tree.
• Non-trivial vaults include vault-plan.md and follow its naming and note-type rules.
• A reader can predict a note's folder from the documented placement rule, and cross-cutting topics do not create duplicate notes.
• _knowledge/catalog.jsonl resolves every catalog entry to one current note path and enables title, alias, tag, domain, and entity lookup.
• Operator/UI-to-code/database lookup jobs have explicit user_terms, source_symbols, and declared bridge layers; aliases remains same-referent vocabulary.
• Relation notes represent contrasts, equivalences, and sequences with at least two participants and evidence for every side.
• Every declared competency question has one current result. All results are complete, cite direct evidence, include the required note IDs, and report no missing parts.
• _knowledge/coverage.md reports numeric answerability coverage, and the answerability validator exits successfully before completion is claimed.
• Important components, concepts, decisions, and workflows have at least one inbound link.
• Important notes are reachable from index.md or a MOC within two clicks.
• Every non-obvious claim is traceable to a source or labeled as an inference.
• Frontmatter is consistent enough to support later rag-corpus-builder or knowledge-graph-builder work.
• RAG-ready vaults preserve stable IDs, semantic headings, exact source references, and metadata that can propagate without reclassification.
• Improvement memory records concrete evidence and actionable next changes without becoming catalog or RAG content itself.
• Notes are useful independently but improve when followed through links.
• The result avoids documentation theater: no generic summaries, no invented architecture, no empty MOCs, and no link spam.

What Claude Does / What You Do

Related Skills

• develop:code-documenter - use when the main goal is inline/API/code documentation rather than a linked knowledge base.
• knowledge:rag-corpus-builder - use when the goal is retrieval-ready chunks, metadata, citations, and eval queries.
• knowledge:ontology-builder - use when note types, domains, aliases, or relation meanings need shared semantic control across vault, graph, and RAG outputs.
• knowledge:knowledge-graph-builder - use when the goal is entity/relationship schema and graph-ready records.
• knowledge:knowledge-query - use when querying an existing vault, graph, RAG corpus, or mixed knowledge asset.
• develop:documentation-strategy - use when planning a documentation system before producing notes.
• write:doc-coauthoring - use when collaboratively drafting a specific document for readers.