Wiki Ingest

Wiki Ingest

Purpose

Convert raw, unstructured sources into structured wiki pages stored in MemPalace. Each page has YAML frontmatter, mandatory Counter-Arguments section, bidirectional [[wikilinks]], and explored: false until human verification.

This is one of 4 wiki skills (ingest, query, lint, explore) that together implement the Brandvoice pattern on top of MemPalace.

When To Invoke

• New files appear in raw/ that don't have a corresponding wiki entry
• User says "ingest", "add this to my wiki", or drops a file into raw/
• Nightly scheduled cycle (configured via /mxm-tasks)
• After any web research session (research agents auto-trigger)

Inputs

• Source files in raw/:
  • raw/clippings/ — web clips, article dumps
  • raw/ideas/ — freeform notes, brain dumps
  • raw/sources/ — PDFs, papers, transcripts
  • raw/x-archive/ — social exports

Workflow

1. Scan raw/ for files not yet referenced by any MemPalace drawer (compare against mempalace_search results filtered by source path).
2. For each unprocessed source:
   • Detect type: tweet, article, PDF, transcript, note, bookmark, paper
   • If URL: fetch full content via WebFetch
   • Extract: key claims, concepts, entities, dates, counter-arguments, citations
   • Apply 80% relevance filter — skip if signal is weak (vague, redundant, off-topic)
3. Create source summary page in MemPalace:
   • Wing: wing_knowledge
   • Room: sources
   • Drawer: page body with YAML frontmatter (see schema below)
4. Create or UPDATE concept/entity pages (never overwrite — merge):
   • Concepts → wing wing_knowledge, room concepts
   • Entities (people, tools, companies) → wing wing_knowledge, room entities
5. Add bidirectional [[wikilinks]] between all related pages (use mempalace_kg_add to record relationships in the graph layer).
6. Set explored: false on all new/updated pages.
7. Update wiki index: append TLDR entries to MemPalace drawer wing_knowledge/index/index-md.
8. Append to wiki log: timestamp, source name, pages created/updated → MemPalace drawer wing_knowledge/log/log-md.

Page Schema (MemPalace drawer frontmatter)

---
title: ""
created: YYYY-MM-DD
updated: YYYY-MM-DD
tags: []
sources: []                  # paths to raw/ files
confidence: high | medium | low
explored: false              # AI-generated → human verification needed
startup: null                # set if scoped to a specific startup
---

Mandatory Page Sections (in order)

1. TLDR — 2–3 sentence summary
2. Main Content — H2/H3 structured body
3. Counter-Arguments — data gaps, opposing evidence, caveats (NEVER skip)
4. Related — [[wikilinks]] to other pages (≥2 inbound for new pages)
5. Sources — links back to raw/ files

Quality Rules

• Apply 80% relevance filter — skip sources with weak signal
• Every page MUST have Counter-Arguments section
• Never duplicate pages — merge into existing pages when topic overlaps
• Confidence defaults to medium unless source is official/verified
• Naming: lowercase kebab-case (agent-memory-patterns)
• One concept per page — split any page over 500 lines

MemPalace Integration

mempalace_add_drawer({
  wing: "wing_knowledge",
  room: "sources" | "concepts" | "entities" | "syntheses",
  content: "<full-page-with-frontmatter>",
  source_file: "raw/sources/<original>.pdf",
  added_by: "wiki-ingest"
})

mempalace_kg_add({
  subject: "<entity-or-concept>",
  predicate: "mentioned_in" | "related_to" | "contradicts" | "extends",
  object: "<related-entity-or-concept>",
  source_closet: "<drawer-id>"
})

Output Format

Wiki Ingest Report:
Sources processed: [N]
Skipped (low signal): [N]
Pages created:
  - sources/[slug] (confidence: medium, explored: false)
  - concepts/[slug] (confidence: medium, explored: false)
  - entities/[slug] (confidence: high, explored: false)
KG triples added: [N]
Index entries appended: [N]
Log appended: 1 entry @ [ISO timestamp]
Next: review pending pages with `/mxm-wiki explore`
Confidence: 🟢 HIGH (relevance ≥80%, all pages have Counter-Arguments)

Handoff

• Pages created → notify wiki-explore for the next verification pass
• Contradictions detected during ingest → escalate to wiki-lint
• Pages tagged with regulated topic (GDPR/HIPAA/etc.) → notify compliance-officer for review
• New concepts that should propagate to brand voice → notify brand-guardian

Maxim Behavioral Layer

• Confidence tagging: 🟢 HIGH when relevance ≥80% and all pages structured correctly | 🟡 MEDIUM when partial structure | 🔴 LOW when relevance filter rejected most sources
• Frameworks: LLM Wikid pattern (sources → structured wiki with bidirectional links + explored: false default + Counter-Arguments)
• Ethics gate: never ingest sources containing PII without data-privacy-officer review

Source References

• FINAL_RELEASE_CHANGES/Brandvoice-wiki-example.md — original pattern
• documents/reference/AGENTS.md — agent usage guide for the wiki layer
• .claude/skills/memory-palace/SKILL.md — MemPalace integration patterns

Copyright (c) 2026 iSystematic Inc. Maxim product. BSL 1.1.