MinerU Learning Path

MinerU Learning Path

A progressive guide to mastering MinerU - from understanding its purpose to building production-ready document processing pipelines.

Level 1: Overview & Motivation

What Problem Does MinerU Solve?

The Core Problem: Traditional PDF extraction produces semantically incoherent text that's unsuitable for Large Language Model (LLM) training and processing.

When you extract text from a typical PDF with standard tools, you get:

• Text in the wrong order (columns mixed up, headers scattered)
• Mathematical formulas as gibberish or question marks
• Tables as unstructured text soup
• Headers, footers, and page numbers mixed into content
• Images with no metadata or context

MinerU transforms this chaos into clean, structured, LLM-ready data.

Origin Story

MinerU was born during the pre-training process of InternLM (a large language model). The team needed to extract high-quality training data from scientific papers, which are notoriously difficult:

• Complex multi-column layouts
• Mathematical formulas with special symbols
• Tables with intricate structures
• Figures with captions
• Mixed languages

Existing tools failed to preserve semantic coherence and reading order. So they built MinerU.

What Existed Before?

Traditional PDF Extractors:

• pdfminer.six: Gets raw text but no structure, wrong order
• PyPDF2: Basic extraction, struggles with complex layouts
• Tesseract OCR: Good for scanned docs but no layout understanding
• Commercial tools (Mathpix, Adobe): Expensive, closed-source, limited API access

Problems:

• No understanding of document structure
• Can't handle scientific notation
• Mixed reading order in multi-column layouts
• Poor table extraction
• No formula recognition

Why is MinerU Better?

1. Layout Understanding: Recognizes document structure (headings, paragraphs, lists, tables)
2. Reading Order Preservation: Outputs text in human reading order
3. Formula Recognition: Converts mathematical formulas to LaTeX
4. Table Extraction: Transforms tables into HTML format
5. Multilingual OCR: Supports 109 languages
6. Multiple Backends: Choose speed vs accuracy tradeoffs
7. Open Source: Free, customizable, community-driven
8. LLM-Ready Output: Markdown and JSON formats optimized for AI

Who Uses MinerU?

Researchers & Academics:

• Extracting training data from scientific papers
• Mining research literature
• Building academic knowledge bases

AI/ML Developers:

• Preparing corpora for LLM training
• Building RAG (Retrieval-Augmented Generation) systems
• Creating custom datasets

Enterprise & Business:

• Processing legal documents
• Extracting financial reports
• Converting technical documentation
• Building searchable knowledge bases

Data Scientists:

• Document intelligence systems
• Information extraction pipelines
• Multi-language document processing

For What Use Cases?

✅ Perfect for:

• Scientific papers with formulas and tables
• Multi-column layouts (journals, magazines)
• Technical documentation
• Business reports with structured data
• Scanned documents (with OCR)
• Mixed-language documents
• Documents for LLM training/RAG systems

❌ NOT recommended for:

• Comic books and art albums
• Vertical text (not supported)
• Handwritten notes (poor accuracy)
• Primary school textbooks/exercises
• Documents requiring code block extraction
• Real-time processing (relatively slow)
• Simple text-only PDFs (overkill)

When Should You NOT Use MinerU?

Use simpler tools if:

• You have simple, single-column text PDFs → Use pdftotext or PyPDF2
• You need instant processing → Use lightweight extractors
• You're on low-resource hardware → MinerU needs 16GB+ RAM
• You need code block extraction → Not yet supported
• Document has vertical text → Not supported

Consider alternatives if:

• You need commercial support → Use Adobe, Mathpix
• You want faster processing → Try Marker (but GPL license)
• You need better image handling → Marker excels here
• Budget allows paid APIs → Doc2X, Mathpix offer convenience

Key Takeaway

MinerU is your go-to tool when document structure and semantic coherence matter more than raw speed. If you're building AI systems that need to understand documents (not just extract text), MinerU is likely your best open-source choice.

Level 2: Installation & Hello World

Prerequisites

Hardware Requirements:

• RAM: 16GB minimum, 32GB+ recommended
• Disk Space: 20GB+ (SSD preferred)
• GPU (optional): 10GB+ VRAM for VLM backends (CUDA, MPS, or NPU)

Software Requirements:

• Python: 3.10 - 3.13
  • ⚠️ Windows only supports 3.10-3.12 (Ray dependency limitation)
• Operating Systems:
  • Linux: 2019+ distributions
  • Windows: Python 3.10-3.12 only
  • macOS: 14.0 or later

Installation Steps

Method 1: pip/uv (Recommended)

# Upgrade pip first
pip install --upgrade pip

# Install uv (modern fast Python package installer)
pip install uv

# Install MinerU with all dependencies
uv pip install -U "mineru[all]"

What does this install?

• MinerU core library
• All backend dependencies (pipeline, VLM, hybrid)
• OCR models and dependencies
• FastAPI and Gradio interfaces

Method 2: From Source (For Contributors)

# Clone the repository
git clone https://github.com/opendatalab/MinerU.git
cd MinerU

# Install in editable mode
uv pip install -e .[all]

Method 3: Docker (For Environment Isolation)

# Pull the official Docker image
docker pull opendatalab/mineru:latest

# Run container with GPU support
docker run --gpus all -v $(pwd)/data:/app/data opendatalab/mineru:latest

When to use Docker?

• Environment compatibility issues
• Multiple Python versions on your system
• Production deployment
• Avoiding dependency conflicts

Verification

After installation, verify MinerU is working:

# Check version
mineru --version

# Should output: MinerU version 2.7.1 (or latest)

Hello World: Your First PDF Extraction

Let's extract content from a PDF and see MinerU in action.

Step 1: Prepare a Sample PDF

Create a simple test PDF or use one from the demo folder:

# Download a sample PDF
curl -O https://raw.githubusercontent.com/opendatalab/MinerU/master/demo/pdfs/demo1.pdf

Or use any PDF you have available.

Step 2: Basic Command-Line Extraction

# Extract PDF to output folder
mineru -p demo1.pdf -o output

# For CPU-only systems (no GPU)
mineru -p demo1.pdf -o output -b pipeline

What happens?

1. MinerU analyzes the PDF layout
2. Detects if OCR is needed
3. Extracts text, formulas, tables, and images
4. Generates output in multiple formats

Step 3: Examine the Output

# Navigate to output folder
cd output

# List files
ls -la

You should see:

demo1/
├── auto/
│   ├── demo1.md              # Markdown version
│   ├── middle.json           # Intermediate format
│   ├── layout.png            # Layout visualization
│   └── images/               # Extracted images

Step 4: View the Markdown

# View the extracted markdown
cat demo1/auto/demo1.md

Expected output: Clean, structured markdown with:

• Proper heading hierarchy
• Paragraphs in reading order
• Tables in HTML format
• Formulas in LaTeX notation
• Image references

Hello World: Python API

Create a file hello_mineru.py:

from pathlib import Path
from mineru.cli.client import parse_doc

# Input PDF path
pdf_path = Path("demo1.pdf")

# Output directory
output_dir = Path("output")

# Parse the document
parse_doc(
    path_list=[pdf_path],
    output_dir=output_dir,
    backend="hybrid-auto-engine",  # Default: best balance
    lang="en"                       # Language for OCR
)

print(f"✅ Extraction complete! Check {output_dir}")

Run it:

python hello_mineru.py

What this does:

1. Imports the main parsing function
2. Specifies input PDF and output directory
3. Uses hybrid backend (best accuracy/speed balance)
4. Sets language to English for OCR
5. Generates all output formats

Troubleshooting

Problem: ImportError: libGL.so.1: cannot open shared object file

# Solution (Ubuntu/Debian)
sudo apt-get install libgl1-mesa-glx

Problem: Missing CJK (Chinese/Japanese/Korean) fonts on Linux

# Solution: Install Noto fonts
sudo apt install fonts-noto-core fonts-noto-cjk
fc-cache -fv

Problem: Out of memory errors

# Solution: Use pipeline backend (lower memory)
mineru -p input.pdf -o output -b pipeline

Problem: Slow processing

• Use GPU if available (automatically detected)
• Reduce resolution in config for faster processing
• Consider using pipeline backend for speed

Success Criteria

✅ You're ready to move on if:

• MinerU installed without errors
• Verification command shows version
• Sample PDF extracted successfully
• Output folder contains markdown and JSON
• Python API script runs without errors

Next Step Preview

In Level 3, we'll explore MinerU's core concepts:

• Processing backends (pipeline, VLM, hybrid)
• Layout understanding and reading order
• Intelligent content recognition
• Multilingual OCR
• Flexible deployment architecture

Level 3: Core Concepts

Now that MinerU is running, let's understand the fundamental concepts that make it powerful.

Concept 1: Multiple Processing Backends

MinerU offers three distinct backends, each with different accuracy/speed tradeoffs.

Pipeline Backend

How it works: Traditional model-based processing with explicit stages:

1. Layout detection (YOLO-based)
2. OCR for text extraction
3. Formula recognition (MFR)
4. Table structure parsing
5. Reading order determination

Characteristics:

• ⚡ Fast: Optimized for speed
• 📊 Accuracy: 82+ on OmniDocBench
• 💾 Resource-friendly: Lower memory usage
• 🎯 No hallucinations: Rule-based, deterministic

Best for:

• CPU-only systems
• Batch processing many documents
• When speed matters more than perfection
• Simple to moderately complex layouts

Example:

from pathlib import Path
from mineru.cli.client import parse_doc

parse_doc(
    path_list=[Path("document.pdf")],
    output_dir="output",
    backend="pipeline",  # Use pipeline backend
    lang="en"
)

VLM Backend

How it works: Uses MinerU2.5-2509-1.2B vision-language model with two-stage inference:

1. Global layout understanding (structure, regions)
2. Fine-grained content recognition (text, formulas, tables)

Characteristics:

• 🎯 High accuracy: 90+ on benchmarks
• 🧠 Context-aware: Understands document semantics
• 💪 Handles complexity: Best for intricate layouts
• 🔥 GPU required: 10GB+ VRAM needed

Best for:

• Maximum accuracy requirements
• Complex scientific papers
• Documents with unusual layouts
• GPU-equipped systems

Example:

parse_doc(
    path_list=[Path("complex_paper.pdf")],
    output_dir="output",
    backend="vlm-auto-engine",  # Use VLM backend locally
    lang="en"
)

Hybrid Backend (Default in v2.7+)

How it works: Combines advantages of both approaches:

• Direct text extraction for text PDFs
• VLM for complex elements (formulas, tables)
• Multi-language OCR (109 languages)
• Optional inline formula recognition

Characteristics:

• ⚖️ Balanced: Best of both worlds
• 🎯 Accuracy: 90+ on benchmarks
• 🔄 Adaptive: Chooses right tool for each element
• 🚀 Default choice: Recommended for most users

Best for:

• General-purpose document processing
• Unknown document types
• Production systems
• When you want "just works" experience

Example:

parse_doc(
    path_list=[Path("mixed_document.pdf")],
    output_dir="output",
    backend="hybrid-auto-engine",  # Default: recommended
    lang="en"
)

Backend Selection Decision Tree

Do you have a GPU with 10GB+ VRAM?
├─ Yes → Use "vlm-auto-engine" (highest accuracy)
└─ No
   ├─ Do you need maximum speed?
   │  └─ Yes → Use "pipeline" (fastest)
   └─ Do you want best overall results?
      └─ Yes → Use "hybrid-auto-engine" (default, balanced)

Common Mistake: Using VLM backend without enough VRAM

# ❌ This will fail on CPU or low-VRAM GPU
parse_doc(..., backend="vlm-auto-engine")  # Requires 10GB+ VRAM

# ✅ Use hybrid instead
parse_doc(..., backend="hybrid-auto-engine")  # Works on CPU

Concept 2: Layout Understanding and Reading Order Preservation

MinerU doesn't just extract text—it understands document structure.

What is Layout Understanding?

Layout detection identifies:

• Document regions (text blocks, images, tables, formulas)
• Hierarchical structure (headings, subheadings, paragraphs)
• Reading flow (column order, page flow)
• Semantic elements (captions, footnotes, headers/footers)

Example: Multi-column journal paper

┌─────────────────────────────┐
│  Title: Neural Networks     │
│  Author: Jane Doe          │
├──────────────┬──────────────┤
│  Column 1    │  Column 2    │
│  Text flows  │  Text flows  │
│  top to      │  top to      │
│  bottom      │  bottom      │
│              │              │
│  [Figure 1]  │  [Table 1]   │
│  Caption     │  Caption     │
└──────────────┴──────────────┘

Traditional extractors produce:

Title: Neural Networks Column 1 Column 2 Author: Jane Doe Text flows Text flows
top to top to [Figure 1] bottom bottom Caption [Table 1] Caption

❌ Completely garbled!

MinerU produces:

# Neural Networks

**Author**: Jane Doe

Text flows top to bottom [Column 1 content]

Text flows top to bottom [Column 2 content]

![Figure 1](images/figure1.png)
*Figure 1: Caption*

| Table 1 |
|---------|
| Data    |
*Table 1: Caption*

✅ Perfect reading order!

How Reading Order Works

1. Layout detection identifies all elements

2. Region classification labels each (text/image/table/formula)

3. Reading order determination sorts elements:

   • Top-to-bottom within columns
   • Left-to-right for columns
   • Proper handling of spans and floats

4. Noise removal filters out:

   • Headers and footers
   • Page numbers
   • Decorative elements
   • Watermarks

Example: Preserve structure

from pathlib import Path
from mineru.cli.client import parse_doc

# Parse with structure preservation
parse_doc(
    path_list=[Path("journal.pdf")],
    output_dir="output",
    backend="hybrid-auto-engine"
)

# Result maintains semantic coherence
# - Headings marked with #, ##, ###
# - Lists properly formatted
# - Paragraphs in logical order
# - Tables and images inline at correct positions

Common Mistake: Expecting perfect order for extremely complex layouts

# ❌ Very complex layouts may still have minor ordering issues
# Documents with:
# - Irregular grids
# - Text wrapping around multiple images
# - Rotated text boxes
# - Overlapping elements

# ✅ Best practice: Review output for critical documents
# Use layout visualization to verify

Verify Layout Detection

Generate layout visualization to see what MinerU detected:

from mineru.cli.client import do_parse

do_parse(
    pdf_path="document.pdf",
    output_dir="output",
    f_draw_layout_bbox=True  # Enable layout visualization
)

# Check output/document/auto/layout.png
# Shows bounding boxes for detected elements

Use cases:

• Debugging extraction issues
• Understanding model behavior
• Quality assurance for critical documents

Concept 3: Intelligent Content Recognition

MinerU recognizes and converts different content types intelligently.

Formula Recognition

Problem: Mathematical formulas in PDFs are either:

• Embedded as images (unextractable)
• Encoded with special fonts (garbled text)

Solution: MinerU converts formulas to LaTeX notation.

Example:

PDF shows: E = mc²

MinerU extracts:

$E = mc^2$

For block formulas:

$$
\int_{-\infty}^{\infty} e^{-x^2} dx = \sqrt{\pi}
$$

Code example:

from mineru.cli.client import do_parse

do_parse(
    pdf_path="scientific_paper.pdf",
    output_dir="output",
    formula_enable=True  # Enable formula recognition
)

# Output markdown will contain LaTeX formulas
# Can be rendered in Markdown viewers, Jupyter, LaTeX

Common Mistake: Expecting perfect handwritten formula recognition

# ❌ Handwritten formulas have poor accuracy
# MinerU is optimized for typeset formulas

# ✅ Best for:
# - Journal articles (LaTeX-generated)
# - Textbooks (professional typesetting)
# - Technical reports

Table Conversion

Problem: Tables in PDFs lose structure when extracted as plain text.

Solution: MinerU converts tables to HTML format (structured).

Example:

PDF table:

┌──────────┬──────────┐
│  Name    │  Score   │
├──────────┼──────────┤
│  Alice   │  95      │
│  Bob     │  87      │
└──────────┴──────────┘

MinerU extracts:

<table>
  <thead>
    <tr><th>Name</th><th>Score</th></tr>
  </thead>
  <tbody>
    <tr><td>Alice</td><td>95</td></tr>
    <tr><td>Bob</td><td>87</td></tr>
  </tbody>
</table>

In markdown:

| Name  | Score |
|-------|-------|
| Alice | 95    |
| Bob   | 87    |

Code example:

from mineru.cli.client import do_parse

do_parse(
    pdf_path="report.pdf",
    output_dir="output",
    table_enable=True  # Enable table recognition
)

Complex tables: MinerU handles:

• Merged cells (spanning rows/columns)
• Nested headers
• Multi-line cells
• Tables spanning pages

Known limitation: Very complex tables may have row/column errors.

Image Extraction

What MinerU does:

• Extracts images from PDFs
• Saves with unique filenames
• Preserves metadata (captions)
• Links images in markdown

Example:

![Figure 1: Neural network architecture](images/abc123.png)

*Figure 1: Neural network architecture showing input, hidden, and output layers.*

Code example:

# Images extracted automatically
parse_doc(
    path_list=[Path("paper.pdf")],
    output_dir="output"
)

# Check output/paper/auto/images/ folder
# All images saved with MD5 hashes as filenames

Automatic OCR Detection

Problem: Some PDFs contain scanned images (no text layer).

Solution: MinerU automatically detects and enables OCR.

How it works:

1. Analyzes if PDF has extractable text
2. If not (scanned PDF), activates OCR
3. If garbled text detected, switches to OCR mode

Example:

parse_doc(
    path_list=[Path("scanned_document.pdf")],
    output_dir="output",
    method="auto"  # Auto-detect if OCR needed (default)
)

# Alternatives:
# method="txt"  → Force text extraction (faster, no OCR)
# method="ocr"  → Force OCR (even if text exists)

Common Mistake: Not specifying language for OCR

# ❌ May produce poor results for non-English
parse_doc(..., lang="en")  # Wrong for Chinese document

# ✅ Specify correct language
parse_doc(..., lang="ch")  # Correct for Chinese

Concept 4: Multilingual OCR Support

MinerU supports 109 languages for OCR processing.

Supported Language Groups

• Latin script: English, Spanish, French, German, Portuguese, etc.
• CJK: Chinese, Japanese, Korean
• Cyrillic: Russian, Ukrainian, Bulgarian
• Devanagari: Hindi, Marathi, Nepali
• Arabic script: Arabic, Urdu, Persian
• Other: Thai, Tamil, Telugu, Kannada, Greek, and more

Language Codes

Common codes:

• en - English
• ch - Chinese (Simplified)
• ch_server - Chinese (Server model, higher accuracy)
• chinese_cht - Chinese (Traditional)
• korean - Korean
• japan - Japanese
• latin - Latin-based scripts (general)
• arabic - Arabic
• cyrillic - Cyrillic scripts
• devanagari - Hindi and related scripts

Using Languages

Single language:

parse_doc(
    path_list=[Path("chinese_doc.pdf")],
    output_dir="output",
    lang="ch"  # Specify language
)

Auto-detection (for mixed languages):

parse_doc(
    path_list=[Path("multilang_doc.pdf")],
    output_dir="output",
    lang="ch"  # Primary language, auto-detects others
)

Multiple documents with different languages:

documents = [
    (Path("english.pdf"), "en"),
    (Path("chinese.pdf"), "ch"),
    (Path("japanese.pdf"), "japan")
]

for pdf_path, language in documents:
    parse_doc(
        path_list=[pdf_path],
        output_dir="output",
        lang=language
    )

Language Performance Notes

Best performance:

• English (en)
• Chinese Simplified (ch_server for highest accuracy)
• European languages (Latin group)

Moderate performance:

• Japanese, Korean
• Cyrillic scripts
• Thai, Devanagari

Challenging:

• Arabic script (easily confused characters)
• Diacritical marks in Latin scripts
• Mixed-script documents

Common Mistake: Using wrong language model

# ❌ Using English for Chinese text
parse_doc(..., lang="en")  # Poor results for Chinese

# ✅ Use appropriate language
parse_doc(..., lang="ch")  # Much better results

Tip: If text is garbled, try:

1. Checking language setting
2. Forcing OCR mode (method="ocr")
3. Installing CJK fonts on Linux systems

Concept 5: Flexible Deployment Architecture

MinerU offers multiple ways to deploy and scale.

Local Processing (Default)

Single-machine processing: Everything runs on your machine.

parse_doc(
    path_list=[Path("doc.pdf")],
    output_dir="output",
    backend="hybrid-auto-engine"  # Local processing
)

Pros:

• No network latency
• Data privacy (nothing leaves your machine)
• No API costs

Cons:

• Resource-intensive (uses your RAM/GPU)
• Scales only with your hardware

Client-Server Architecture

Scenario: Lightweight CPU clients + powerful GPU server.

Setup:

Server (GPU machine):

# Start VLM inference server
mineru-vllm-server \
  --host 0.0.0.0 \
  --port 8001 \
  --model opendatalab/MinerU2.5-2509-1.2B-hf

Client (CPU machine):

parse_doc(
    path_list=[Path("doc.pdf")],
    output_dir="output",
    backend="vlm-http-client",  # Use remote server
    server_url="http://gpu-server:8001"  # Server address
)

Benefits:

• CPU-only clients can use GPU processing
• Centralized resource management
• Scale by adding more servers
• Multiple clients share one GPU server

Use cases:

• Team with one GPU machine
• Cloud deployment with GPU instances
• Batch processing from multiple sources

API Server

FastAPI REST API for HTTP-based processing.

Start server:

mineru-api --host 0.0.0.0 --port 8000

Access API docs: http://127.0.0.1:8000/docs

Use from any language:

Python:

import requests

response = requests.post(
    "http://localhost:8000/parse",
    json={
        "pdf_url": "https://example.com/document.pdf",
        "backend": "hybrid-auto-engine"
    },
    headers={"Authorization": "Bearer YOUR_TOKEN"}
)

result = response.json()

cURL:

curl -X POST http://localhost:8000/parse \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"pdf_url":"https://example.com/doc.pdf"}'

Benefits:

• Language-agnostic (HTTP interface)
• Stateless, scalable
• Easy integration with existing systems

Web Interface (Gradio)

Interactive web UI for manual processing.

mineru-gradio --server-name 0.0.0.0 --server-port 7860

Access: http://127.0.0.1:7860

Features:

• Upload PDFs through browser
• Visual feedback and previews
• Download results directly
• No coding required

Use cases:

• One-off document processing
• Demos and presentations
• Non-technical users

Cloud Services

Official cloud: https://mineru.net

• 2000 pages/day at highest priority
• No installation needed
• API access with token

Limitations:

• 200MB max file size
• 600 pages max per file
• Beta service (limits may change)

Deployment Decision Tree

What's your use case?

├─ Quick local processing
│  └─ Use: Command-line or Python API locally

├─ Team with shared GPU server
│  └─ Use: Client-server architecture

├─ Web service / production API
│  └─ Use: FastAPI server + load balancer

├─ Non-technical users
│  └─ Use: Gradio web interface

└─ No local resources
   └─ Use: Official cloud service at mineru.net

Common Mistake: Running VLM backend on CPU

# ❌ VLM backend requires GPU (10GB+ VRAM)
parse_doc(..., backend="vlm-auto-engine")  # Fails on CPU

# ✅ Options:
# 1. Use hybrid backend (works on CPU)
parse_doc(..., backend="hybrid-auto-engine")

# 2. Use remote GPU server
parse_doc(..., backend="vlm-http-client", server_url="...")

# 3. Use pipeline backend (CPU-friendly)
parse_doc(..., backend="pipeline")

Concepts Summary

You've mastered the concepts! Next: Apply them in practical patterns.

Level 4: Practical Patterns

Now let's build real-world document processing pipelines using MinerU's concepts.

Pattern 1: Simple Document Extraction

Scenario: Extract content from a single PDF for reading or analysis.

Code:

from pathlib import Path
from mineru.cli.client import parse_doc

def extract_simple_pdf(pdf_path: str, output_dir: str = "output"):
    """
    Extract content from a PDF into markdown format.

    Args:
        pdf_path: Path to input PDF
        output_dir: Where to save results
    """
    parse_doc(
        path_list=[Path(pdf_path)],
        output_dir=output_dir,
        backend="hybrid-auto-engine",  # Best balance
        lang="en"  # Adjust for your language
    )

    # Output location
    pdf_name = Path(pdf_path).stem
    markdown_file = Path(output_dir) / pdf_name / "auto" / f"{pdf_name}.md"

    print(f"✅ Extraction complete!")
    print(f"📄 Markdown: {markdown_file}")

    return markdown_file

# Usage
extract_simple_pdf("research_paper.pdf")

Expected output:

✅ Extraction complete!
📄 Markdown: output/research_paper/auto/research_paper.md

When to use:

• Single document processing
• Manual review needed
• One-off conversions

Pattern 2: Batch Processing Multiple PDFs

Scenario: Process an entire folder of PDFs.

Code:

from pathlib import Path
from mineru.cli.client import parse_doc
from typing import List

def batch_extract_pdfs(input_folder: str, output_dir: str = "batch_output"):
    """
    Process all PDFs in a folder.

    Args:
        input_folder: Folder containing PDFs
        output_dir: Where to save results
    """
    # Find all PDFs
    pdf_files = list(Path(input_folder).glob("*.pdf"))

    if not pdf_files:
        print(f"❌ No PDFs found in {input_folder}")
        return

    print(f"📚 Found {len(pdf_files)} PDFs to process")

    # Process all at once
    parse_doc(
        path_list=pdf_files,
        output_dir=output_dir,
        backend="hybrid-auto-engine",
        lang="en"
    )

    print(f"✅ Processed {len(pdf_files)} documents")
    print(f"📁 Results in: {output_dir}")

# Usage
batch_extract_pdfs("./pdfs_to_process", "output")

For better control (process one-by-one):

def batch_extract_with_progress(input_folder: str, output_dir: str = "output"):
    """Process PDFs one by one with progress tracking."""
    from tqdm import tqdm  # Progress bar

    pdf_files = list(Path(input_folder).glob("*.pdf"))

    for pdf_file in tqdm(pdf_files, desc="Processing PDFs"):
        try:
            parse_doc(
                path_list=[pdf_file],
                output_dir=output_dir,
                backend="hybrid-auto-engine",
                lang="en"
            )
            print(f"✅ {pdf_file.name}")
        except Exception as e:
            print(f"❌ {pdf_file.name}: {e}")

# Usage
batch_extract_with_progress("./papers")

When to use:

• Processing document archives
• Building datasets
• Automated pipelines

Pattern 3: Language-Specific Processing

Scenario: Handle documents in different languages correctly.

Code:

from pathlib import Path
from mineru.cli.client import parse_doc
import fast_langdetect  # Optional: auto-detect language

# Language mapping
LANGUAGE_CODES = {
    "en": "en",
    "zh-cn": "ch",
    "zh-tw": "chinese_cht",
    "ja": "japan",
    "ko": "korean",
    "es": "latin",
    "fr": "latin",
    "de": "latin",
    "ar": "arabic",
    "ru": "cyrillic"
}

def extract_multilingual_pdf(pdf_path: str, language: str = None):
    """
    Extract PDF with language-specific OCR.

    Args:
        pdf_path: Path to PDF
        language: ISO language code (e.g., 'en', 'zh-cn')
                  If None, attempts auto-detection
    """
    # Auto-detect if not specified
    if language is None:
        # Simple heuristic: check filename or use default
        language = "en"  # Default to English
        print(f"⚠️  No language specified, using: {language}")

    # Map to MinerU language code
    mineru_lang = LANGUAGE_CODES.get(language, "en")

    print(f"🌍 Processing with language: {mineru_lang}")

    parse_doc(
        path_list=[Path(pdf_path)],
        output_dir="output",
        backend="hybrid-auto-engine",
        lang=mineru_lang
    )

# Usage examples
extract_multilingual_pdf("paper_english.pdf", "en")
extract_multilingual_pdf("论文_中文.pdf", "zh-cn")
extract_multilingual_pdf("논문_한국어.pdf", "ko")

Advanced: Batch with language detection:

def batch_extract_multilingual(pdf_language_pairs: List[tuple]):
    """
    Process multiple PDFs with different languages.

    Args:
        pdf_language_pairs: List of (pdf_path, language_code) tuples
    """
    for pdf_path, lang_code in pdf_language_pairs:
        mineru_lang = LANGUAGE_CODES.get(lang_code, "en")

        print(f"📄 Processing {Path(pdf_path).name} ({mineru_lang})")

        parse_doc(
            path_list=[Path(pdf_path)],
            output_dir="output",
            lang=mineru_lang
        )

# Usage
documents = [
    ("report_en.pdf", "en"),
    ("报告_中文.pdf", "zh-cn"),
    ("レポート.pdf", "ja"),
    ("доклад.pdf", "ru")
]

batch_extract_multilingual(documents)

When to use:

• International document collections
• Multi-language research
• Global content processing

Pattern 4: Scientific Paper Processing

Scenario: Extract structured data from academic papers (formulas, tables, figures).

Code:

from pathlib import Path
from mineru.cli.client import parse_doc
import json

def extract_scientific_paper(
    pdf_path: str,
    output_dir: str = "papers_output",
    visualize: bool = True
):
    """
    Extract scientific paper with formula and table recognition.

    Args:
        pdf_path: Path to research paper PDF
        output_dir: Output directory
        visualize: Generate layout visualization
    """
    from mineru.cli.client import do_parse

    print(f"🔬 Processing scientific paper: {Path(pdf_path).name}")

    do_parse(
        pdf_path=pdf_path,
        output_dir=output_dir,
        backend="vlm-auto-engine",  # Highest accuracy for papers
        lang="en",
        formula_enable=True,  # Enable formula recognition
        table_enable=True,    # Enable table recognition
        f_dump_md=True,       # Generate markdown
        f_dump_middle_json=True,  # Generate intermediate JSON
        f_draw_layout_bbox=visualize  # Layout visualization
    )

    pdf_name = Path(pdf_path).stem
    output_path = Path(output_dir) / pdf_name / "auto"

    # Load structured data
    middle_json = output_path / "middle.json"
    if middle_json.exists():
        with open(middle_json) as f:
            data = json.load(f)

        # Extract statistics
        num_pages = len(data.get("pages", []))
        formulas = [elem for page in data.get("pages", [])
                   for elem in page.get("elements", [])
                   if elem.get("type") == "formula"]
        tables = [elem for page in data.get("pages", [])
                 for elem in page.get("elements", [])
                 if elem.get("type") == "table"]

        print(f"📊 Statistics:")
        print(f"   Pages: {num_pages}")
        print(f"   Formulas: {len(formulas)}")
        print(f"   Tables: {len(tables)}")

    print(f"✅ Complete! Check {output_path}")

# Usage
extract_scientific_paper(
    "quantum_physics_paper.pdf",
    visualize=True
)

Access extracted formulas:

def extract_formulas_from_paper(pdf_path: str):
    """Extract all LaTeX formulas from a paper."""
    # First process the paper
    extract_scientific_paper(pdf_path)

    # Read markdown output
    pdf_name = Path(pdf_path).stem
    md_file = Path("papers_output") / pdf_name / "auto" / f"{pdf_name}.md"

    with open(md_file) as f:
        content = f.read()

    # Extract inline formulas: $...$
    import re
    inline_formulas = re.findall(r'\$([^\$]+)\$', content)

    # Extract block formulas: $$...$$
    block_formulas = re.findall(r'\$\$([^\$]+)\$\$', content)

    print(f"Found {len(inline_formulas)} inline formulas")
    print(f"Found {len(block_formulas)} block formulas")

    return inline_formulas, block_formulas

# Usage
inline, block = extract_formulas_from_paper("paper.pdf")
for formula in block[:5]:  # Show first 5
    print(f"  {formula.strip()}")

When to use:

• Academic research mining
• Building training datasets from papers
• Extracting mathematical content
• Citation analysis pipelines

Pattern 5: Building a RAG System Data Pipeline

Scenario: Prepare document collection for a Retrieval-Augmented Generation (RAG) system.

Code:

from pathlib import Path
from mineru.cli.client import parse_doc
import json
from typing import List, Dict

def prepare_rag_dataset(
    pdf_folder: str,
    output_dir: str = "rag_data",
    chunk_size: int = 1000
) -> List[Dict]:
    """
    Process PDFs and prepare for RAG system ingestion.

    Args:
        pdf_folder: Folder with PDF documents
        output_dir: Where to save processed data
        chunk_size: Target size for text chunks (characters)

    Returns:
        List of document chunks with metadata
    """
    pdf_files = list(Path(pdf_folder).glob("*.pdf"))

    print(f"📚 Processing {len(pdf_files)} documents for RAG")

    # Process all PDFs
    parse_doc(
        path_list=pdf_files,
        output_dir=output_dir,
        backend="hybrid-auto-engine",
        lang="en"
    )

    # Extract and chunk content
    chunks = []

    for pdf_file in pdf_files:
        pdf_name = pdf_file.stem
        md_file = Path(output_dir) / pdf_name / "auto" / f"{pdf_name}.md"

        if not md_file.exists():
            print(f"⚠️  Skipping {pdf_name} (no markdown output)")
            continue

        with open(md_file) as f:
            content = f.read()

        # Simple chunking by paragraphs
        paragraphs = content.split('\n\n')
        current_chunk = ""

        for para in paragraphs:
            if len(current_chunk) + len(para) > chunk_size:
                # Save current chunk
                if current_chunk:
                    chunks.append({
                        "text": current_chunk.strip(),
                        "source": pdf_name,
                        "source_file": str(pdf_file)
                    })
                current_chunk = para
            else:
                current_chunk += "\n\n" + para

        # Add remaining chunk
        if current_chunk:
            chunks.append({
                "text": current_chunk.strip(),
                "source": pdf_name,
                "source_file": str(pdf_file)
            })

    # Save chunked dataset
    output_file = Path(output_dir) / "rag_chunks.json"
    with open(output_file, 'w') as f:
        json.dump(chunks, f, indent=2)

    print(f"✅ Created {len(chunks)} chunks")
    print(f"💾 Saved to: {output_file}")

    return chunks

# Usage
chunks = prepare_rag_dataset("./knowledge_base", chunk_size=1000)

# Sample output
print(f"\nSample chunk:")
print(f"Source: {chunks[0]['source']}")
print(f"Text: {chunks[0]['text'][:200]}...")

Integrate with vector database:

def ingest_into_vectordb(chunks: List[Dict], collection_name: str = "documents"):
    """
    Ingest chunks into a vector database (example with ChromaDB).

    Note: Requires `pip install chromadb sentence-transformers`
    """
    import chromadb
    from chromadb.utils import embedding_functions

    # Initialize client
    client = chromadb.Client()

    # Create collection with embedding function
    sentence_transformer_ef = embedding_functions.SentenceTransformerEmbeddingFunction(
        model_name="all-MiniLM-L6-v2"
    )

    collection = client.create_collection(
        name=collection_name,
        embedding_function=sentence_transformer_ef
    )

    # Add documents
    collection.add(
        documents=[chunk["text"] for chunk in chunks],
        metadatas=[{"source": chunk["source"]} for chunk in chunks],
        ids=[f"chunk_{i}" for i in range(len(chunks))]
    )

    print(f"✅ Ingested {len(chunks)} chunks into ChromaDB")

# Full pipeline
chunks = prepare_rag_dataset("./docs")
ingest_into_vectordb(chunks)

When to use:

• Building knowledge bases for LLMs
• Question-answering systems
• Document search and retrieval
• AI-powered research assistants

Pattern 6: API Server for Production

Scenario: Deploy MinerU as a production REST API service.

Setup API server:

# server.py
from fastapi import FastAPI, UploadFile, File, HTTPException
from pathlib import Path
import tempfile
import shutil
from mineru.cli.client import parse_doc

app = FastAPI(title="MinerU API", version="1.0")

@app.post("/parse")
async def parse_pdf(
    file: UploadFile = File(...),
    backend: str = "hybrid-auto-engine",
    language: str = "en"
):
    """
    Parse uploaded PDF and return markdown content.
    """
    # Validate file type
    if not file.filename.endswith('.pdf'):
        raise HTTPException(400, "Only PDF files are supported")

    # Save uploaded file temporarily
    with tempfile.NamedTemporaryFile(delete=False, suffix=".pdf") as tmp:
        shutil.copyfileobj(file.file, tmp)
        tmp_path = tmp.name

    # Create output directory
    output_dir = tempfile.mkdtemp()

    try:
        # Process PDF
        parse_doc(
            path_list=[Path(tmp_path)],
            output_dir=output_dir,
            backend=backend,
            lang=language
        )

        # Read markdown output
        pdf_name = Path(tmp_path).stem
        md_file = Path(output_dir) / pdf_name / "auto" / f"{pdf_name}.md"

        with open(md_file) as f:
            markdown_content = f.read()

        return {
            "status": "success",
            "filename": file.filename,
            "markdown": markdown_content
        }

    except Exception as e:
        raise HTTPException(500, f"Processing failed: {str(e)}")

    finally:
        # Cleanup
        Path(tmp_path).unlink()
        shutil.rmtree(output_dir)

@app.get("/health")
def health_check():
    """Health check endpoint."""
    return {"status": "healthy"}

# Run with: uvicorn server:app --host 0.0.0.0 --port 8000

Client usage:

# client.py
import requests

def parse_pdf_via_api(pdf_p

…(truncated)