andurilcode Deep Document Processor
Transform large documents into token-efficient context artifacts optimized for downstream agent reasoning. Goal: maximum decision-relevant signal per token, not human-readable summaries.
Naive approaches fail predictably:
- Full inclusion: blows budget, buries signal
- Naive truncation: loses structure, misses late insights
- Single-pass summary: misses cross-references, flattens hierarchy
The Protocol: Four Passes
Pass 1 — Structural Survey (≤30 seconds)
Extract the document's skeleton before reading content. Creates a mental map guiding attention in later passes.
Extract:
- Document type and purpose (paper? spec? report? guide?)
- Table of contents / section headings / file tree
- Total length estimate (pages, sections, or tokens)
- Key entities mentioned (people, systems, concepts)
- Publication date and freshness signal
Output format:
STRUCTURE:
- Type: [document type]
- Sections: [numbered list of headings/sections]
- Length: [estimate]
- Key entities: [comma-separated]
- Freshness: [date or "undated"]
Pass 2 — Selective Extraction (The 20% Rule)
Read each section and extract ONLY content meeting one of these criteria:
- Decision-relevant: Would change a downstream choice or recommendation
- Constraint-defining: Sets boundaries on what's possible or allowed
- Counter-intuitive: Contradicts likely assumptions an agent might hold
- Dependency-creating: Other facts depend on this being known
Hard constraint: extracted content ≤20% of original tokens. If extracting more, stop and re-evaluate decision-relevant vs. merely interesting.
Extraction format — telegram-style, not prose:
EXTRACT [Section Name]:
- [fact/constraint/insight in ≤15 words]
- [fact/constraint/insight in ≤15 words]
- COUNTER-INTUITIVE: [thing that contradicts common assumption]
Pass 3 — Cross-Reference and Conflict Scan
Review your Pass 2 extracts and identify:
- Internal contradictions: Does section 3 contradict section 7?
- Cross-references: Does understanding X require knowing Y from another section?
- Missing context: Are there terms, acronyms, or concepts used without definition?
- Implicit assumptions: What does the document assume the reader already knows?
Output format:
CROSS-REFS:
- [Section X] depends on [Section Y]: [why]
- CONFLICT: [Section A] says X, but [Section B] implies Y
- UNDEFINED: [term/concept] used but never defined
- ASSUMES: [implicit knowledge required]
Pass 4 — Context Artifact Assembly
Combine passes 1-3 into a single context artifact for agent consumption.
Template:
=== CONTEXT: [Document Title] ===
Type: [type] | Freshness: [date] | Compression: [X% of original]
STRUCTURE MAP:
[Pass 1 skeleton — 2-3 lines max]
KEY EXTRACTS:
[Pass 2 content, organized by relevance not document order]
DEPENDENCIES & CONFLICTS:
[Pass 3 findings — only if non-empty]
AGENT NOTES:
- [Any warnings about document quality, bias, or missing info]
- [Suggested follow-up if context is insufficient]
===
Quality Checks
- Compression ratio: ≤20% of input tokens. If not, cut more.
- Standalone test: Could an agent with ONLY this artifact make the same decisions as with the full document? If not, what's missing?
- No filler: Every line passes "would removing this change a decision?". If no — delete.
- Telegram density: cut articles, hedging, qualifiers ruthlessly.
Anti-Patterns
- Summarizing for humans: not a book report. No flowing prose.
- Preserving document order: organize by relevance, not appearance.
- "Interesting but irrelevant": if it doesn't affect decisions, cut it.
- Over-extracting definitions: only define terms the downstream agent won't know.
- Hedging: "The document seems to suggest..." — No. State what it says. Flag uncertainty explicitly if needed, but don't hedge the extraction itself.