Forge Document Intelligence

FORGE DOCUMENT INTELLIGENCE — EMD Stack Skill

DITEMPA BUKAN DIBERI — Intelligence is forged, not given. OCR is sensory perception, not reasoning. Image → 555-ASI-VISION (gate) → structured text → 333-AGI (reason).

0. ARCHITECTURE (v2.0 — 2026-07-31)

The DeepSeek-OCR eureka fundamentally reorients our architecture:

BEFORE (v1.0):
  PDF → PyMuPDF → Tesseract → Text → Agent (reasoning on untrusted text)
  ❌ No injection scan on OCR output
  ❌ No constitutional gate between pixels and reasoning
  ❌ Text extraction = lossy compression with no provenance

AFTER (v2.0):
  PDF → Router → Engine (Qwen2.5-VL / Unlimited-OCR / Tesseract)
       → 555-ASI-VISION GATE (F12 INJECTION + F9 ANTI-HANTU + F2 TRUTH)
       → 555→333 Contract (OBS · DER · CONFIDENCE · F9 · F12)
       → 333-AGI (reason on GATED text only)
  ✅ Mandatory F12 injection scan on ALL OCR output
  ✅ F2 epistemic labels on every element
  ✅ SHA-256 provenance per element
  ✅ 555→333 contract — structured, auditable, gated

The gate is the architecture. Adversarial text hidden in images IS a real vector. OCR output is untrusted text — it came from pixels, not from a human typing.

Engine Selection

marker / marker_single (2026-08-12 OBS): locally installed layout-preserving PDF→Markdown converter — the zero-API local workhorse for the common case (tables, headings, lists, equations → markdown) before reaching for VLM APIs. marker_single file.pdf → markdown output. Standalone CLI, not an aaa_ocr perception backend. When an external tool's pitch is "PDF→Markdown with structure preservation" (e.g. MarkPDFDown), the answer is usually marker + governance gate, not a new install — see references/markpdfdown-v2m-evaluation.md for the capability-vs-packaging map and the do-not-install verdict.

Unified CLI: aaa_ocr — single entry point for all AAA agents.

aaa_ocr doc.pdf                           # auto-route
aaa_ocr doc.pdf --engine qwen25_vl        # force engine  
aaa_ocr doc.pdf --financial               # bbox required
aaa_ocr --status                           # engine matrix

1. What This Skill Is

This skill wraps the EMD (Encode-Metabolize-Decode) document intelligence stack as a first-class federation capability. It encodes the 3-layer architecture (Perception → Provenance → Purpose), the visual re-grounding anti-hallucination protocol, 888_HOLD gate placement, and Malaysian document risk patterns into one governed skill.

The core insight: olmOCR, forge_document_ingest, and the constitutional pipeline are NOT competing tools — they are three layers of one stack, each reducing a different type of entropy. This skill makes that stack available to every AAA citizen agent.

2. The Three Layers (v2.0 — DeepSeek-OCR Eureka)

Key constraint: af-forge has NO GPU. DeepSeek-OCR (200K+ pages/day on A100) is future. Vision-native engines use remote APIs. The gate runs locally.

DeepSeek-OCR Eureka (arxiv:2510.18234): Vision tokens are compressed context — 256 vision tokens carry 2500+ text tokens at 97% precision. OCR is sensory perception, not reasoning. This CONFIRMS the architecture: 555-ASI-VISION perceives, 333-AGI reasons.

Configurable Perception Backend

The perception layer is swap-able via config, not hardcoded. Declare in skill metadata:

perception_backend: qwen25_vl | unlimited_gradio | tesseract | deepseek_ocr

Unified CLI: aaa_ocr — single entry point for all AAA agents.

aaa_ocr doc.pdf                    # auto-route based on doc characteristics
aaa_ocr doc.pdf --financial        # force bbox-capable engine
aaa_ocr --status                   # engine matrix

Swapping rule: Change perception_backend in skill config → pipeline routes accordingly. The gate layer (555-ASI-VISION) and provenance layer (forge_document_ingest) stay unchanged. Vision-native engines share the same contract output.

3. The EMD Stack

┌──────────────────────────────────────────────────────┐
│  LAYER 1: PERCEPTION (Ingestion Boundary)            │
│  ────────────────────────────────────────────────    │
│  Input: PDF/PNG/JPEG                                 │
│  Engine: Qwen2.5-VL via Bailian API (no GPU needed)  │
│  Output: Markdown + page images (PRESERVED)          │
│  Consumer: GEOX, WEALTH, WELL, AAA                   │
│  Floor: F2 TRUTH (label as OBS, confidence <0.90)    │
│  Cost: ~$0.02/page via Bailian                        │
│  NOTE: af-forge has NO GPU. Use remote API only.     │
└──────────────────┬───────────────────────────────────┘
                   │  Markdown + page images
                   ▼
┌──────────────────────────────────────────────────────┐
│  LAYER 2: METABOLISM (Provenance Layer)              │
│  ────────────────────────────────────────────────    │
│  Input: Markdown + page images                       │
│  Engine: forge_document_ingest (ALREADY EXISTS)      │
│  Output: Structured JSON + bbox + SHA-256 + chunks   │
│  Key: bbox → original page pixels (NOT to Markdown)  │
│  Floor: F11 AUDIT (provenance hash)                  │
│  Cost: ~$0 (CPU, local)                              │
│  Modes: analyze, extract, chunk, compare              │
└──────────────────┬───────────────────────────────────┘
                   │  JSON + bbox + provenance
                   ▼
┌──────────────────────────────────────────────────────┐
│  LAYER 3: PURPOSE (Governed Action Layer)            │
│  ────────────────────────────────────────────────    │
│  Input: JSON + bbox + provenance                     │
│  Engine: Constitutional pipeline                     │
│         (000→111→333→666→888→999)                    │
│  Action: Route by domain, verify claims,             │
│         re-ground against original image if HIGH     │
│         stakes (money, legal, medical)               │
│  Floor: F1 AMANAH + F2 TRUTH + F13 SOVEREIGN         │
│  888_HOLD: Any extraction feeding WEALTH capital     │
│  computation or VAULT999 seal                        │
│  Cost: governance compute                            │
└──────────────────────────────────────────────────────┘

Pipeline flow: PDF → Qwen2.5-VL (Encode) → forge_document_ingest (Metabolize) → Constitutional pipeline (Decode)

4. Visual Re-Grounding Protocol — The Critical Anti-Hallucination Mechanism

The Error Propagation Problem

VLM extracts: "Total: RM 1,250,000"   (original said 12,500,000 — missed a digit)
    ↓ Markdown looks clean
Agentic layer trusts it: confidence=HIGH
    ↓
WEALTH computes NPV on wrong number
    ↓
VAULT999 seals wrong number with high confidence

Clean-looking OCR with subtle errors is MORE dangerous than obviously bad OCR — because the governance pipeline trusts structured output. A messy Tesseract extraction triggers suspicion; a clean VLM extraction with a missed digit sails through every gate.

The Fix

The Agentic layer (Layer 3) must re-verify against the original page image via bbox coordinates, NOT against the intermediate Markdown.

forge_document_ingest already stores bbox coordinates per element — this is the anchor. The bbox maps extracted text back to the exact pixels on the original page image.

When to Re-Ground

How to Re-Ground

1. Identify the key data point in the structured JSON (e.g., {"text": "RM 12,500,000", "bbox": [120, 340, 280, 360], "page": 3})
2. Load the original page image for that page
3. Crop the bbox region from the original image
4. Present the cropped region alongside the extracted text for verification
5. If mismatch → flag as RE_GROUNDING_FAILED, trigger 888_HOLD

What to Do If Re-Grounding Fails

• Flag the element as RE_GROUNDING_FAILED in the evidence table
• Trigger 888_HOLD — do NOT pass the unverified value downstream
• Present the discrepancy to Arif with both the extracted text and the original image crop
• Log the failure to forge_work/ with SHA-256 of both versions

5. 888_HOLD Gates

Gate sequence: Perception QC → (pass) → Metabolism → Re-grounding → (pass) → 888_HOLD → (Arif decides) → VAULT999 seal

6. Domain Routing

Routing rule: When in doubt, route to the organ whose domain matches the document's subject matter, not the document's format.

7. Malaysian Document Risks

Additional Malaysian context: Many critical documents (land titles, grant letters, Syariah court orders) combine multiple risk factors — old scan + rubber stamp + multi-language + handwriting. These require the full re-grounding protocol, not just VLM extraction.

8. Cost-Aware Extraction Strategy

Zen marginal cost:

Tesseract:  $0/M pages   → HIGH entropy   → POISONED RAG
olmOCR:     $176/M pages  → LOW entropy    → CLEAN RAG
Agentic:    +governance   → LOWEST entropy → GOVERNED RAG

You pay for OCR once. You pay for bad OCR forever.

The expensive layer is perception. Everything after is governance, and governance is cheap when perception is clean.

Decision rule: If the document has tables, multi-columns, stamps, handwriting, or feeds a high-stakes decision → use VLM. If it's a clean single-column printout → Tesseract is fine.

9. Integration Points

This skill connects to the federation at these points:

Key invariant: forge_document_ingest is the metabolism engine. This skill does NOT replace it — it wraps the full EMD stack around it, adding the perception layer (VLM) and the purpose layer (constitutional governance) that forge_document_ingest alone does not provide.

10. Quick Reference — Decision Tree

DOCUMENT ARRIVES
    │
    ├── What format?
    │   ├── PDF → check if scanned or digital-born
    │   ├── Image (PNG/JPEG) → always VLM path
    │   └── Digital-born PDF → may skip VLM, extract text directly
    │
    ├── What complexity?
    │   ├── Single-column, text-only, clean → Tesseract (ocr=true in forge_document_ingest)
    │   ├── Tables, multi-column, stamps, handwriting → VLM (Qwen2.5-VL via Bailian)
    │   └── Old scan (pre-1990) → VLM + mandatory re-grounding
    │
    ├── What stakes?
    │   ├── Low (reference, knowledge base) → Extract → Metabolize → Store
    │   ├── Medium (operational data) → Extract → Metabolize → Domain route
    │   └── High (money, legal, medical) → Extract → Metabolize → RE-GROUND → 888_HOLD → Seal
    │
    └── What domain?
        ├── Legal/financial → WEALTH
        ├── Geological → GEOX
        ├── Medical → WELL
        └── General → AAA

11. Anti-Patterns

12. Constitutional Constraints

• F1 AMANAH: All document processing is reversible (read-only ingestion). Original files are never modified. forge_document_ingest creates new structured output without touching the source.
• F2 TRUTH: All VLM extractions labeled OBS with confidence <0.90. Never claim certainty without re-grounding against original image. The Markdown is a convenience view, not evidence.
• F4 CLARITY: Clean output — structured JSON with bbox, provenance hash, and page references. Never dump raw Markdown without structure.
• F6 MARUAH: Document intelligence serves the sovereign. Personal documents (medical, legal) are handled with dignity — no unnecessary exposure of sensitive content.
• F11 AUDIT: Every extraction leaves a provenance hash (SHA-256) and bbox trail. The hash chain from original file → VLM output → structured JSON → domain action is traceable end-to-end.
• F13 SOVEREIGN: 888_HOLD before any document-derived claim enters VAULT999. Arif decides trustworthiness of OCR-derived data, not the pipeline.

13. Telemetry

{
  "skill_name": "forge-document-intelligence",
  "version": "1.0.0",
  "trigger_phrase": "{{trigger_phrase}}",
  "selected_reason": "{{selected_reason}}",
  "document_processed": null,
  "pages_processed": 0,
  "vlm_extraction_used": false,
  "tesseract_used": false,
  "re_grounding_triggered": false,
  "re_grounding_failed": false,
  "888_hold_triggered": false,
  "domain_routed_to": null,
  "provenance_hash": null,
  "latency_ms": 0,
  "token_in": 0,
  "token_out": 0,
  "cost_usd": 0
}

Session Provenance

• Origin session: SEAL-76129e84d1e6415c (FORGE GLM-5.2, 2026-07-02)
• Analysis file: /root/A-FORGE/forge_work/2026-07-02/olmocr-agentic-ocr-analysis.md
• Init prompt: /root/A-FORGE/forge_work/2026-07-02/AAA-INIT-PROMPT-document-intelligence.md
• Hermes input: Confirmed architectural split (perception vs reasoning), flagged hallucination chaining risk and Malaysian document unknowns
• Qwen-arifOS input: Confirmed perception vs reasoning paradigm split, referenced LandingAI ADE and LlamaIndex
• Meta-mesa gap: P1 — Document intelligence was entirely missing domain (0 of 36 skills covered it)

References

• MarkPDFDown V2M tool evaluation: references/markpdfdown-v2m-evaluation.md
• olmOCR: https://github.com/allenai/olmocr
• olmOCR paper: https://olmocr.allenai.org/papers/olmocr.pdf
• forge_document_ingest: A-FORGE MCP tool (already exists, modes: analyze/extract/chunk/compare)
• Meta-mesa skill atlas: /root/.agents/skills/meta-mesa-skill-atlas/SKILL.md
• AGENTS.md: /root/AGENTS.md (heptalogy + constitutional floors)
• A-FORGE AGENTS.md: /root/A-FORGE/AGENTS.md

Forged: 2026-07-02 by FORGE (000Ω) for F13 SOVEREIGN Session: SEAL-76129e84d1e6415c Gap filled: P1 — Document intelligence (meta-mesa §3B → now covered) DITEMPA BUKAN DIBERI — You pay for OCR once. You pay for bad OCR forever.

APPENDIX: Live RAG Pipeline (2026-07-24)

The EMD stack now has a live RAG pipeline in A-FORGE:

forge_document_ingest(chunk) → Ollama bge-m3 embed → Qdrant rag_federation_docs → rag query tool

Pipeline scripts: /root/A-FORGE/rag/

• embed_store.py — Ingest chunks → embed via bge-m3 → store in Qdrant
• query.py — Embed query → search Qdrant → return ranked results (+ optional FLAME synthesis)

Collection: rag_federation_docs (Qdrant, 1024-dim, Cosine distance) Embedding model: bge-m3:latest (Ollama, 567M params, 1024-dim output)

Quick test:

python3 /root/A-FORGE/rag/query.py "your question" --top-k 5 --synthesize