# MinerU Learning Path

> A progressive guide to mastering MinerU - from understanding its purpose to building production-ready document processing pipelines.

- Skill: `tools-only/mineru-learning-path` (Agent Skill, multi-file: 3 files)
- Install (CLI): `npx skillmds@latest add tools-only/mineru-learning-path`
- Raw SKILL.md: https://api.skillmd.com/api/skills/tools-only/mineru-learning-path/raw
- Safety review: pending (external: skill-scanner PASS, skillspector CAUTION)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: tools-only (https://skillmd.com/u/tools-only)
- Updated: 2026-09-29
- Page: https://skillmd.com/skills/tools-only/mineru-learning-path

---

# MinerU Learning Path

A progressive guide to mastering MinerU - from understanding its purpose to building production-ready document processing pipelines.

---

## Level 1: Overview & Motivation

### What Problem Does MinerU Solve?

**The Core Problem**: Traditional PDF extraction produces semantically incoherent text that's unsuitable for Large Language Model (LLM) training and processing.

When you extract text from a typical PDF with standard tools, you get:
- Text in the wrong order (columns mixed up, headers scattered)
- Mathematical formulas as gibberish or question marks
- Tables as unstructured text soup
- Headers, footers, and page numbers mixed into content
- Images with no metadata or context

**MinerU transforms this chaos into clean, structured, LLM-ready data.**

### Origin Story

MinerU was born during the pre-training process of **InternLM** (a large language model). The team needed to extract high-quality training data from scientific papers, which are notoriously difficult:
- Complex multi-column layouts
- Mathematical formulas with special symbols
- Tables with intricate structures
- Figures with captions
- Mixed languages

Existing tools failed to preserve semantic coherence and reading order. So they built MinerU.

### What Existed Before?

**Traditional PDF Extractors**:
- **pdfminer.six**: Gets raw text but no structure, wrong order
- **PyPDF2**: Basic extraction, struggles with complex layouts
- **Tesseract OCR**: Good for scanned docs but no layout understanding
- **Commercial tools** (Mathpix, Adobe): Expensive, closed-source, limited API access

**Problems**:
- No understanding of document structure
- Can't handle scientific notation
- Mixed reading order in multi-column layouts
- Poor table extraction
- No formula recognition

### Why is MinerU Better?

1. **Layout Understanding**: Recognizes document structure (headings, paragraphs, lists, tables)
2. **Reading Order Preservation**: Outputs text in human reading order
3. **Formula Recognition**: Converts mathematical formulas to LaTeX
4. **Table Extraction**: Transforms tables into HTML format
5. **Multilingual OCR**: Supports 109 languages
6. **Multiple Backends**: Choose speed vs accuracy tradeoffs
7. **Open Source**: Free, customizable, community-driven
8. **LLM-Ready Output**: Markdown and JSON formats optimized for AI

### Who Uses MinerU?

**Researchers & Academics**:
- Extracting training data from scientific papers
- Mining research literature
- Building academic knowledge bases

**AI/ML Developers**:
- Preparing corpora for LLM training
- Building RAG (Retrieval-Augmented Generation) systems
- Creating custom datasets

**Enterprise & Business**:
- Processing legal documents
- Extracting financial reports
- Converting technical documentation
- Building searchable knowledge bases

**Data Scientists**:
- Document intelligence systems
- Information extraction pipelines
- Multi-language document processing

### For What Use Cases?

✅ **Perfect for**:
- Scientific papers with formulas and tables
- Multi-column layouts (journals, magazines)
- Technical documentation
- Business reports with structured data
- Scanned documents (with OCR)
- Mixed-language documents
- Documents for LLM training/RAG systems

❌ **NOT recommended for**:
- Comic books and art albums
- Vertical text (not supported)
- Handwritten notes (poor accuracy)
- Primary school textbooks/exercises
- Documents requiring code block extraction
- Real-time processing (relatively slow)
- Simple text-only PDFs (overkill)

### When Should You NOT Use MinerU?

**Use simpler tools if**:
- You have simple, single-column text PDFs → Use `pdftotext` or `PyPDF2`
- You need instant processing → Use lightweight extractors
- You're on low-resource hardware → MinerU needs 16GB+ RAM
- You need code block extraction → Not yet supported
- Document has vertical text → Not supported

**Consider alternatives if**:
- You need commercial support → Use Adobe, Mathpix
- You want faster processing → Try Marker (but GPL license)
- You need better image handling → Marker excels here
- Budget allows paid APIs → Doc2X, Mathpix offer convenience

### Key Takeaway

**MinerU is your go-to tool when document structure and semantic coherence matter more than raw speed.** If you're building AI systems that need to understand documents (not just extract text), MinerU is likely your best open-source choice.

---

## Level 2: Installation & Hello World

### Prerequisites

**Hardware Requirements**:
- **RAM**: 16GB minimum, 32GB+ recommended
- **Disk Space**: 20GB+ (SSD preferred)
- **GPU** (optional): 10GB+ VRAM for VLM backends (CUDA, MPS, or NPU)

**Software Requirements**:
- **Python**: 3.10 - 3.13
  - ⚠️ Windows only supports 3.10-3.12 (Ray dependency limitation)
- **Operating Systems**:
  - Linux: 2019+ distributions
  - Windows: Python 3.10-3.12 only
  - macOS: 14.0 or later

### Installation Steps

#### Method 1: pip/uv (Recommended)

```bash
# Upgrade pip first
pip install --upgrade pip

# Install uv (modern fast Python package installer)
pip install uv

# Install MinerU with all dependencies
uv pip install -U "mineru[all]"
```

**What does this install?**
- MinerU core library
- All backend dependencies (pipeline, VLM, hybrid)
- OCR models and dependencies
- FastAPI and Gradio interfaces

#### Method 2: From Source (For Contributors)

```bash
# Clone the repository
git clone https://github.com/opendatalab/MinerU.git
cd MinerU

# Install in editable mode
uv pip install -e .[all]
```

#### Method 3: Docker (For Environment Isolation)

```bash
# Pull the official Docker image
docker pull opendatalab/mineru:latest

# Run container with GPU support
docker run --gpus all -v $(pwd)/data:/app/data opendatalab/mineru:latest
```

**When to use Docker?**
- Environment compatibility issues
- Multiple Python versions on your system
- Production deployment
- Avoiding dependency conflicts

### Verification

After installation, verify MinerU is working:

```bash
# Check version
mineru --version

# Should output: MinerU version 2.7.1 (or latest)
```

### Hello World: Your First PDF Extraction

Let's extract content from a PDF and see MinerU in action.

#### Step 1: Prepare a Sample PDF

Create a simple test PDF or use one from the demo folder:

```bash
# Download a sample PDF
curl -O https://raw.githubusercontent.com/opendatalab/MinerU/master/demo/pdfs/demo1.pdf
```

Or use any PDF you have available.

#### Step 2: Basic Command-Line Extraction

```bash
# Extract PDF to output folder
mineru -p demo1.pdf -o output

# For CPU-only systems (no GPU)
mineru -p demo1.pdf -o output -b pipeline
```

**What happens?**
1. MinerU analyzes the PDF layout
2. Detects if OCR is needed
3. Extracts text, formulas, tables, and images
4. Generates output in multiple formats

#### Step 3: Examine the Output

```bash
# Navigate to output folder
cd output

# List files
ls -la
```

**You should see**:
```
demo1/
├── auto/
│   ├── demo1.md              # Markdown version
│   ├── middle.json           # Intermediate format
│   ├── layout.png            # Layout visualization
│   └── images/               # Extracted images
```

#### Step 4: View the Markdown

```bash
# View the extracted markdown
cat demo1/auto/demo1.md
```

**Expected output**: Clean, structured markdown with:
- Proper heading hierarchy
- Paragraphs in reading order
- Tables in HTML format
- Formulas in LaTeX notation
- Image references

### Hello World: Python API

Create a file `hello_mineru.py`:

```python
from pathlib import Path
from mineru.cli.client import parse_doc

# Input PDF path
pdf_path = Path("demo1.pdf")

# Output directory
output_dir = Path("output")

# Parse the document
parse_doc(
    path_list=[pdf_path],
    output_dir=output_dir,
    backend="hybrid-auto-engine",  # Default: best balance
    lang="en"                       # Language for OCR
)

print(f"✅ Extraction complete! Check {output_dir}")
```

Run it:

```bash
python hello_mineru.py
```

**What this does**:
1. Imports the main parsing function
2. Specifies input PDF and output directory
3. Uses hybrid backend (best accuracy/speed balance)
4. Sets language to English for OCR
5. Generates all output formats

### Troubleshooting

**Problem**: `ImportError: libGL.so.1: cannot open shared object file`
```bash
# Solution (Ubuntu/Debian)
sudo apt-get install libgl1-mesa-glx
```

**Problem**: Missing CJK (Chinese/Japanese/Korean) fonts on Linux
```bash
# Solution: Install Noto fonts
sudo apt install fonts-noto-core fonts-noto-cjk
fc-cache -fv
```

**Problem**: Out of memory errors
```bash
# Solution: Use pipeline backend (lower memory)
mineru -p input.pdf -o output -b pipeline
```

**Problem**: Slow processing
- Use GPU if available (automatically detected)
- Reduce resolution in config for faster processing
- Consider using pipeline backend for speed

### Success Criteria

✅ You're ready to move on if:
- MinerU installed without errors
- Verification command shows version
- Sample PDF extracted successfully
- Output folder contains markdown and JSON
- Python API script runs without errors

### Next Step Preview

In Level 3, we'll explore MinerU's core concepts:
- Processing backends (pipeline, VLM, hybrid)
- Layout understanding and reading order
- Intelligent content recognition
- Multilingual OCR
- Flexible deployment architecture

---

## Level 3: Core Concepts

Now that MinerU is running, let's understand the fundamental concepts that make it powerful.

### Concept 1: Multiple Processing Backends

MinerU offers **three distinct backends**, each with different accuracy/speed tradeoffs.

#### Pipeline Backend

**How it works**: Traditional model-based processing with explicit stages:
1. Layout detection (YOLO-based)
2. OCR for text extraction
3. Formula recognition (MFR)
4. Table structure parsing
5. Reading order determination

**Characteristics**:
- ⚡ **Fast**: Optimized for speed
- 📊 **Accuracy**: 82+ on OmniDocBench
- 💾 **Resource-friendly**: Lower memory usage
- 🎯 **No hallucinations**: Rule-based, deterministic

**Best for**:
- CPU-only systems
- Batch processing many documents
- When speed matters more than perfection
- Simple to moderately complex layouts

**Example**:
```python
from pathlib import Path
from mineru.cli.client import parse_doc

parse_doc(
    path_list=[Path("document.pdf")],
    output_dir="output",
    backend="pipeline",  # Use pipeline backend
    lang="en"
)
```

#### VLM Backend

**How it works**: Uses **MinerU2.5-2509-1.2B** vision-language model with two-stage inference:
1. Global layout understanding (structure, regions)
2. Fine-grained content recognition (text, formulas, tables)

**Characteristics**:
- 🎯 **High accuracy**: 90+ on benchmarks
- 🧠 **Context-aware**: Understands document semantics
- 💪 **Handles complexity**: Best for intricate layouts
- 🔥 **GPU required**: 10GB+ VRAM needed

**Best for**:
- Maximum accuracy requirements
- Complex scientific papers
- Documents with unusual layouts
- GPU-equipped systems

**Example**:
```python
parse_doc(
    path_list=[Path("complex_paper.pdf")],
    output_dir="output",
    backend="vlm-auto-engine",  # Use VLM backend locally
    lang="en"
)
```

#### Hybrid Backend (Default in v2.7+)

**How it works**: Combines advantages of both approaches:
- Direct text extraction for text PDFs
- VLM for complex elements (formulas, tables)
- Multi-language OCR (109 languages)
- Optional inline formula recognition

**Characteristics**:
- ⚖️ **Balanced**: Best of both worlds
- 🎯 **Accuracy**: 90+ on benchmarks
- 🔄 **Adaptive**: Chooses right tool for each element
- 🚀 **Default choice**: Recommended for most users

**Best for**:
- General-purpose document processing
- Unknown document types
- Production systems
- When you want "just works" experience

**Example**:
```python
parse_doc(
    path_list=[Path("mixed_document.pdf")],
    output_dir="output",
    backend="hybrid-auto-engine",  # Default: recommended
    lang="en"
)
```

#### Backend Selection Decision Tree

```
Do you have a GPU with 10GB+ VRAM?
├─ Yes → Use "vlm-auto-engine" (highest accuracy)
└─ No
   ├─ Do you need maximum speed?
   │  └─ Yes → Use "pipeline" (fastest)
   └─ Do you want best overall results?
      └─ Yes → Use "hybrid-auto-engine" (default, balanced)
```

**Common Mistake**: Using VLM backend without enough VRAM
```python
# ❌ This will fail on CPU or low-VRAM GPU
parse_doc(..., backend="vlm-auto-engine")  # Requires 10GB+ VRAM

# ✅ Use hybrid instead
parse_doc(..., backend="hybrid-auto-engine")  # Works on CPU
```

---

### Concept 2: Layout Understanding and Reading Order Preservation

MinerU doesn't just extract text—it **understands document structure**.

#### What is Layout Understanding?

**Layout detection** identifies:
- Document regions (text blocks, images, tables, formulas)
- Hierarchical structure (headings, subheadings, paragraphs)
- Reading flow (column order, page flow)
- Semantic elements (captions, footnotes, headers/footers)

**Example**: Multi-column journal paper

```
┌─────────────────────────────┐
│  Title: Neural Networks     │
│  Author: Jane Doe          │
├──────────────┬──────────────┤
│  Column 1    │  Column 2    │
│  Text flows  │  Text flows  │
│  top to      │  top to      │
│  bottom      │  bottom      │
│              │              │
│  [Figure 1]  │  [Table 1]   │
│  Caption     │  Caption     │
└──────────────┴──────────────┘
```

**Traditional extractors produce**:
```
Title: Neural Networks Column 1 Column 2 Author: Jane Doe Text flows Text flows
top to top to [Figure 1] bottom bottom Caption [Table 1] Caption
```
❌ Completely garbled!

**MinerU produces**:
```markdown
# Neural Networks

**Author**: Jane Doe

Text flows top to bottom [Column 1 content]

Text flows top to bottom [Column 2 content]

![Figure 1](images/figure1.png)
*Figure 1: Caption*

| Table 1 |
|---------|
| Data    |
*Table 1: Caption*
```
✅ Perfect reading order!

#### How Reading Order Works

1. **Layout detection** identifies all elements
2. **Region classification** labels each (text/image/table/formula)
3. **Reading order determination** sorts elements:
   - Top-to-bottom within columns
   - Left-to-right for columns
   - Proper handling of spans and floats

4. **Noise removal** filters out:
   - Headers and footers
   - Page numbers
   - Decorative elements
   - Watermarks

**Example**: Preserve structure

```python
from pathlib import Path
from mineru.cli.client import parse_doc

# Parse with structure preservation
parse_doc(
    path_list=[Path("journal.pdf")],
    output_dir="output",
    backend="hybrid-auto-engine"
)

# Result maintains semantic coherence
# - Headings marked with #, ##, ###
# - Lists properly formatted
# - Paragraphs in logical order
# - Tables and images inline at correct positions
```

**Common Mistake**: Expecting perfect order for extremely complex layouts
```python
# ❌ Very complex layouts may still have minor ordering issues
# Documents with:
# - Irregular grids
# - Text wrapping around multiple images
# - Rotated text boxes
# - Overlapping elements

# ✅ Best practice: Review output for critical documents
# Use layout visualization to verify
```

#### Verify Layout Detection

Generate layout visualization to see what MinerU detected:

```python
from mineru.cli.client import do_parse

do_parse(
    pdf_path="document.pdf",
    output_dir="output",
    f_draw_layout_bbox=True  # Enable layout visualization
)

# Check output/document/auto/layout.png
# Shows bounding boxes for detected elements
```

**Use cases**:
- Debugging extraction issues
- Understanding model behavior
- Quality assurance for critical documents

---

### Concept 3: Intelligent Content Recognition

MinerU recognizes and converts different content types intelligently.

#### Formula Recognition

**Problem**: Mathematical formulas in PDFs are either:
- Embedded as images (unextractable)
- Encoded with special fonts (garbled text)

**Solution**: MinerU converts formulas to **LaTeX notation**.

**Example**:

PDF shows: E = mc²

MinerU extracts:
```markdown
$E = mc^2$
```

For block formulas:
```markdown
$$
\int_{-\infty}^{\infty} e^{-x^2} dx = \sqrt{\pi}
$$
```

**Code example**:
```python
from mineru.cli.client import do_parse

do_parse(
    pdf_path="scientific_paper.pdf",
    output_dir="output",
    formula_enable=True  # Enable formula recognition
)

# Output markdown will contain LaTeX formulas
# Can be rendered in Markdown viewers, Jupyter, LaTeX
```

**Common Mistake**: Expecting perfect handwritten formula recognition
```python
# ❌ Handwritten formulas have poor accuracy
# MinerU is optimized for typeset formulas

# ✅ Best for:
# - Journal articles (LaTeX-generated)
# - Textbooks (professional typesetting)
# - Technical reports
```

#### Table Conversion

**Problem**: Tables in PDFs lose structure when extracted as plain text.

**Solution**: MinerU converts tables to **HTML format** (structured).

**Example**:

PDF table:
```
┌──────────┬──────────┐
│  Name    │  Score   │
├──────────┼──────────┤
│  Alice   │  95      │
│  Bob     │  87      │
└──────────┴──────────┘
```

MinerU extracts:
```html
<table>
  <thead>
    <tr><th>Name</th><th>Score</th></tr>
  </thead>
  <tbody>
    <tr><td>Alice</td><td>95</td></tr>
    <tr><td>Bob</td><td>87</td></tr>
  </tbody>
</table>
```

In markdown:
```markdown
| Name  | Score |
|-------|-------|
| Alice | 95    |
| Bob   | 87    |
```

**Code example**:
```python
from mineru.cli.client import do_parse

do_parse(
    pdf_path="report.pdf",
    output_dir="output",
    table_enable=True  # Enable table recognition
)
```

**Complex tables**: MinerU handles:
- Merged cells (spanning rows/columns)
- Nested headers
- Multi-line cells
- Tables spanning pages

**Known limitation**: Very complex tables may have row/column errors.

#### Image Extraction

**What MinerU does**:
- Extracts images from PDFs
- Saves with unique filenames
- Preserves metadata (captions)
- Links images in markdown

**Example**:
```markdown
![Figure 1: Neural network architecture](images/abc123.png)

*Figure 1: Neural network architecture showing input, hidden, and output layers.*
```

**Code example**:
```python
# Images extracted automatically
parse_doc(
    path_list=[Path("paper.pdf")],
    output_dir="output"
)

# Check output/paper/auto/images/ folder
# All images saved with MD5 hashes as filenames
```

#### Automatic OCR Detection

**Problem**: Some PDFs contain scanned images (no text layer).

**Solution**: MinerU automatically detects and enables OCR.

**How it works**:
1. Analyzes if PDF has extractable text
2. If not (scanned PDF), activates OCR
3. If garbled text detected, switches to OCR mode

**Example**:
```python
parse_doc(
    path_list=[Path("scanned_document.pdf")],
    output_dir="output",
    method="auto"  # Auto-detect if OCR needed (default)
)

# Alternatives:
# method="txt"  → Force text extraction (faster, no OCR)
# method="ocr"  → Force OCR (even if text exists)
```

**Common Mistake**: Not specifying language for OCR
```python
# ❌ May produce poor results for non-English
parse_doc(..., lang="en")  # Wrong for Chinese document

# ✅ Specify correct language
parse_doc(..., lang="ch")  # Correct for Chinese
```

---

### Concept 4: Multilingual OCR Support

MinerU supports **109 languages** for OCR processing.

#### Supported Language Groups

- **Latin script**: English, Spanish, French, German, Portuguese, etc.
- **CJK**: Chinese, Japanese, Korean
- **Cyrillic**: Russian, Ukrainian, Bulgarian
- **Devanagari**: Hindi, Marathi, Nepali
- **Arabic script**: Arabic, Urdu, Persian
- **Other**: Thai, Tamil, Telugu, Kannada, Greek, and more

#### Language Codes

Common codes:
- `en` - English
- `ch` - Chinese (Simplified)
- `ch_server` - Chinese (Server model, higher accuracy)
- `chinese_cht` - Chinese (Traditional)
- `korean` - Korean
- `japan` - Japanese
- `latin` - Latin-based scripts (general)
- `arabic` - Arabic
- `cyrillic` - Cyrillic scripts
- `devanagari` - Hindi and related scripts

#### Using Languages

**Single language**:
```python
parse_doc(
    path_list=[Path("chinese_doc.pdf")],
    output_dir="output",
    lang="ch"  # Specify language
)
```

**Auto-detection** (for mixed languages):
```python
parse_doc(
    path_list=[Path("multilang_doc.pdf")],
    output_dir="output",
    lang="ch"  # Primary language, auto-detects others
)
```

**Multiple documents with different languages**:
```python
documents = [
    (Path("english.pdf"), "en"),
    (Path("chinese.pdf"), "ch"),
    (Path("japanese.pdf"), "japan")
]

for pdf_path, language in documents:
    parse_doc(
        path_list=[pdf_path],
        output_dir="output",
        lang=language
    )
```

#### Language Performance Notes

**Best performance**:
- English (`en`)
- Chinese Simplified (`ch_server` for highest accuracy)
- European languages (Latin group)

**Moderate performance**:
- Japanese, Korean
- Cyrillic scripts
- Thai, Devanagari

**Challenging**:
- Arabic script (easily confused characters)
- Diacritical marks in Latin scripts
- Mixed-script documents

**Common Mistake**: Using wrong language model
```python
# ❌ Using English for Chinese text
parse_doc(..., lang="en")  # Poor results for Chinese

# ✅ Use appropriate language
parse_doc(..., lang="ch")  # Much better results
```

**Tip**: If text is garbled, try:
1. Checking language setting
2. Forcing OCR mode (`method="ocr"`)
3. Installing CJK fonts on Linux systems

---

### Concept 5: Flexible Deployment Architecture

MinerU offers multiple ways to deploy and scale.

#### Local Processing (Default)

**Single-machine processing**: Everything runs on your machine.

```python
parse_doc(
    path_list=[Path("doc.pdf")],
    output_dir="output",
    backend="hybrid-auto-engine"  # Local processing
)
```

**Pros**:
- No network latency
- Data privacy (nothing leaves your machine)
- No API costs

**Cons**:
- Resource-intensive (uses your RAM/GPU)
- Scales only with your hardware

#### Client-Server Architecture

**Scenario**: Lightweight CPU clients + powerful GPU server.

**Setup**:

**Server** (GPU machine):
```bash
# Start VLM inference server
mineru-vllm-server \
  --host 0.0.0.0 \
  --port 8001 \
  --model opendatalab/MinerU2.5-2509-1.2B-hf
```

**Client** (CPU machine):
```python
parse_doc(
    path_list=[Path("doc.pdf")],
    output_dir="output",
    backend="vlm-http-client",  # Use remote server
    server_url="http://gpu-server:8001"  # Server address
)
```

**Benefits**:
- CPU-only clients can use GPU processing
- Centralized resource management
- Scale by adding more servers
- Multiple clients share one GPU server

**Use cases**:
- Team with one GPU machine
- Cloud deployment with GPU instances
- Batch processing from multiple sources

#### API Server

**FastAPI REST API** for HTTP-based processing.

**Start server**:
```bash
mineru-api --host 0.0.0.0 --port 8000
```

**Access API docs**: http://127.0.0.1:8000/docs

**Use from any language**:

Python:
```python
import requests

response = requests.post(
    "http://localhost:8000/parse",
    json={
        "pdf_url": "https://example.com/document.pdf",
        "backend": "hybrid-auto-engine"
    },
    headers={"Authorization": "Bearer YOUR_TOKEN"}
)

result = response.json()
```

cURL:
```bash
curl -X POST http://localhost:8000/parse \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"pdf_url":"https://example.com/doc.pdf"}'
```

**Benefits**:
- Language-agnostic (HTTP interface)
- Stateless, scalable
- Easy integration with existing systems

#### Web Interface (Gradio)

**Interactive web UI** for manual processing.

```bash
mineru-gradio --server-name 0.0.0.0 --server-port 7860
```

**Access**: http://127.0.0.1:7860

**Features**:
- Upload PDFs through browser
- Visual feedback and previews
- Download results directly
- No coding required

**Use cases**:
- One-off document processing
- Demos and presentations
- Non-technical users

#### Cloud Services

**Official cloud**: https://mineru.net
- 2000 pages/day at highest priority
- No installation needed
- API access with token

**Limitations**:
- 200MB max file size
- 600 pages max per file
- Beta service (limits may change)

#### Deployment Decision Tree

```
What's your use case?

├─ Quick local processing
│  └─ Use: Command-line or Python API locally

├─ Team with shared GPU server
│  └─ Use: Client-server architecture

├─ Web service / production API
│  └─ Use: FastAPI server + load balancer

├─ Non-technical users
│  └─ Use: Gradio web interface

└─ No local resources
   └─ Use: Official cloud service at mineru.net
```

**Common Mistake**: Running VLM backend on CPU
```python
# ❌ VLM backend requires GPU (10GB+ VRAM)
parse_doc(..., backend="vlm-auto-engine")  # Fails on CPU

# ✅ Options:
# 1. Use hybrid backend (works on CPU)
parse_doc(..., backend="hybrid-auto-engine")

# 2. Use remote GPU server
parse_doc(..., backend="vlm-http-client", server_url="...")

# 3. Use pipeline backend (CPU-friendly)
parse_doc(..., backend="pipeline")
```

---

### Concepts Summary

| Concept | Key Idea | Common Mistake |
|---------|----------|----------------|
| **Multiple Backends** | Choose speed vs accuracy | Using VLM without GPU |
| **Layout Understanding** | Structure + reading order | Expecting perfection on complex docs |
| **Content Recognition** | Formulas, tables, images | Not enabling features explicitly |
| **Multilingual OCR** | 109 languages supported | Wrong language code |
| **Flexible Deployment** | Local, server, cloud | Not choosing right architecture |

**You've mastered the concepts!** Next: Apply them in practical patterns.

---

## Level 4: Practical Patterns

Now let's build real-world document processing pipelines using MinerU's concepts.

### Pattern 1: Simple Document Extraction

**Scenario**: Extract content from a single PDF for reading or analysis.

**Code**:

```python
from pathlib import Path
from mineru.cli.client import parse_doc

def extract_simple_pdf(pdf_path: str, output_dir: str = "output"):
    """
    Extract content from a PDF into markdown format.

    Args:
        pdf_path: Path to input PDF
        output_dir: Where to save results
    """
    parse_doc(
        path_list=[Path(pdf_path)],
        output_dir=output_dir,
        backend="hybrid-auto-engine",  # Best balance
        lang="en"  # Adjust for your language
    )

    # Output location
    pdf_name = Path(pdf_path).stem
    markdown_file = Path(output_dir) / pdf_name / "auto" / f"{pdf_name}.md"

    print(f"✅ Extraction complete!")
    print(f"📄 Markdown: {markdown_file}")

    return markdown_file

# Usage
extract_simple_pdf("research_paper.pdf")
```

**Expected output**:
```
✅ Extraction complete!
📄 Markdown: output/research_paper/auto/research_paper.md
```

**When to use**:
- Single document processing
- Manual review needed
- One-off conversions

---

### Pattern 2: Batch Processing Multiple PDFs

**Scenario**: Process an entire folder of PDFs.

**Code**:

```python
from pathlib import Path
from mineru.cli.client import parse_doc
from typing import List

def batch_extract_pdfs(input_folder: str, output_dir: str = "batch_output"):
    """
    Process all PDFs in a folder.

    Args:
        input_folder: Folder containing PDFs
        output_dir: Where to save results
    """
    # Find all PDFs
    pdf_files = list(Path(input_folder).glob("*.pdf"))

    if not pdf_files:
        print(f"❌ No PDFs found in {input_folder}")
        return

    print(f"📚 Found {len(pdf_files)} PDFs to process")

    # Process all at once
    parse_doc(
        path_list=pdf_files,
        output_dir=output_dir,
        backend="hybrid-auto-engine",
        lang="en"
    )

    print(f"✅ Processed {len(pdf_files)} documents")
    print(f"📁 Results in: {output_dir}")

# Usage
batch_extract_pdfs("./pdfs_to_process", "output")
```

**For better control** (process one-by-one):

```python
def batch_extract_with_progress(input_folder: str, output_dir: str = "output"):
    """Process PDFs one by one with progress tracking."""
    from tqdm import tqdm  # Progress bar

    pdf_files = list(Path(input_folder).glob("*.pdf"))

    for pdf_file in tqdm(pdf_files, desc="Processing PDFs"):
        try:
            parse_doc(
                path_list=[pdf_file],
                output_dir=output_dir,
                backend="hybrid-auto-engine",
                lang="en"
            )
            print(f"✅ {pdf_file.name}")
        except Exception as e:
            print(f"❌ {pdf_file.name}: {e}")

# Usage
batch_extract_with_progress("./papers")
```

**When to use**:
- Processing document archives
- Building datasets
- Automated pipelines

---

### Pattern 3: Language-Specific Processing

**Scenario**: Handle documents in different languages correctly.

**Code**:

```python
from pathlib import Path
from mineru.cli.client import parse_doc
import fast_langdetect  # Optional: auto-detect language

# Language mapping
LANGUAGE_CODES = {
    "en": "en",
    "zh-cn": "ch",
    "zh-tw": "chinese_cht",
    "ja": "japan",
    "ko": "korean",
    "es": "latin",
    "fr": "latin",
    "de": "latin",
    "ar": "arabic",
    "ru": "cyrillic"
}

def extract_multilingual_pdf(pdf_path: str, language: str = None):
    """
    Extract PDF with language-specific OCR.

    Args:
        pdf_path: Path to PDF
        language: ISO language code (e.g., 'en', 'zh-cn')
                  If None, attempts auto-detection
    """
    # Auto-detect if not specified
    if language is None:
        # Simple heuristic: check filename or use default
        language = "en"  # Default to English
        print(f"⚠️  No language specified, using: {language}")

    # Map to MinerU language code
    mineru_lang = LANGUAGE_CODES.get(language, "en")

    print(f"🌍 Processing with language: {mineru_lang}")

    parse_doc(
        path_list=[Path(pdf_path)],
        output_dir="output",
        backend="hybrid-auto-engine",
        lang=mineru_lang
    )

# Usage examples
extract_multilingual_pdf("paper_english.pdf", "en")
extract_multilingual_pdf("论文_中文.pdf", "zh-cn")
extract_multilingual_pdf("논문_한국어.pdf", "ko")
```

**Advanced: Batch with language detection**:

```python
def batch_extract_multilingual(pdf_language_pairs: List[tuple]):
    """
    Process multiple PDFs with different languages.

    Args:
        pdf_language_pairs: List of (pdf_path, language_code) tuples
    """
    for pdf_path, lang_code in pdf_language_pairs:
        mineru_lang = LANGUAGE_CODES.get(lang_code, "en")

        print(f"📄 Processing {Path(pdf_path).name} ({mineru_lang})")

        parse_doc(
            path_list=[Path(pdf_path)],
            output_dir="output",
            lang=mineru_lang
        )

# Usage
documents = [
    ("report_en.pdf", "en"),
    ("报告_中文.pdf", "zh-cn"),
    ("レポート.pdf", "ja"),
    ("доклад.pdf", "ru")
]

batch_extract_multilingual(documents)
```

**When to use**:
- International document collections
- Multi-language research
- Global content processing

---

### Pattern 4: Scientific Paper Processing

**Scenario**: Extract structured data from academic papers (formulas, tables, figures).

**Code**:

```python
from pathlib import Path
from mineru.cli.client import parse_doc
import json

def extract_scientific_paper(
    pdf_path: str,
    output_dir: str = "papers_output",
    visualize: bool = True
):
    """
    Extract scientific paper with formula and table recognition.

    Args:
        pdf_path: Path to research paper PDF
        output_dir: Output directory
        visualize: Generate layout visualization
    """
    from mineru.cli.client import do_parse

    print(f"🔬 Processing scientific paper: {Path(pdf_path).name}")

    do_parse(
        pdf_path=pdf_path,
        output_dir=output_dir,
        backend="vlm-auto-engine",  # Highest accuracy for papers
        lang="en",
        formula_enable=True,  # Enable formula recognition
        table_enable=True,    # Enable table recognition
        f_dump_md=True,       # Generate markdown
        f_dump_middle_json=True,  # Generate intermediate JSON
        f_draw_layout_bbox=visualize  # Layout visualization
    )

    pdf_name = Path(pdf_path).stem
    output_path = Path(output_dir) / pdf_name / "auto"

    # Load structured data
    middle_json = output_path / "middle.json"
    if middle_json.exists():
        with open(middle_json) as f:
            data = json.load(f)

        # Extract statistics
        num_pages = len(data.get("pages", []))
        formulas = [elem for page in data.get("pages", [])
                   for elem in page.get("elements", [])
                   if elem.get("type") == "formula"]
        tables = [elem for page in data.get("pages", [])
                 for elem in page.get("elements", [])
                 if elem.get("type") == "table"]

        print(f"📊 Statistics:")
        print(f"   Pages: {num_pages}")
        print(f"   Formulas: {len(formulas)}")
        print(f"   Tables: {len(tables)}")

    print(f"✅ Complete! Check {output_path}")

# Usage
extract_scientific_paper(
    "quantum_physics_paper.pdf",
    visualize=True
)
```

**Access extracted formulas**:

```python
def extract_formulas_from_paper(pdf_path: str):
    """Extract all LaTeX formulas from a paper."""
    # First process the paper
    extract_scientific_paper(pdf_path)

    # Read markdown output
    pdf_name = Path(pdf_path).stem
    md_file = Path("papers_output") / pdf_name / "auto" / f"{pdf_name}.md"

    with open(md_file) as f:
        content = f.read()

    # Extract inline formulas: $...$
    import re
    inline_formulas = re.findall(r'\$([^\$]+)\$', content)

    # Extract block formulas: $$...$$
    block_formulas = re.findall(r'\$\$([^\$]+)\$\$', content)

    print(f"Found {len(inline_formulas)} inline formulas")
    print(f"Found {len(block_formulas)} block formulas")

    return inline_formulas, block_formulas

# Usage
inline, block = extract_formulas_from_paper("paper.pdf")
for formula in block[:5]:  # Show first 5
    print(f"  {formula.strip()}")
```

**When to use**:
- Academic research mining
- Building training datasets from papers
- Extracting mathematical content
- Citation analysis pipelines

---

### Pattern 5: Building a RAG System Data Pipeline

**Scenario**: Prepare document collection for a Retrieval-Augmented Generation (RAG) system.

**Code**:

```python
from pathlib import Path
from mineru.cli.client import parse_doc
import json
from typing import List, Dict

def prepare_rag_dataset(
    pdf_folder: str,
    output_dir: str = "rag_data",
    chunk_size: int = 1000
) -> List[Dict]:
    """
    Process PDFs and prepare for RAG system ingestion.

    Args:
        pdf_folder: Folder with PDF documents
        output_dir: Where to save processed data
        chunk_size: Target size for text chunks (characters)

    Returns:
        List of document chunks with metadata
    """
    pdf_files = list(Path(pdf_folder).glob("*.pdf"))

    print(f"📚 Processing {len(pdf_files)} documents for RAG")

    # Process all PDFs
    parse_doc(
        path_list=pdf_files,
        output_dir=output_dir,
        backend="hybrid-auto-engine",
        lang="en"
    )

    # Extract and chunk content
    chunks = []

    for pdf_file in pdf_files:
        pdf_name = pdf_file.stem
        md_file = Path(output_dir) / pdf_name / "auto" / f"{pdf_name}.md"

        if not md_file.exists():
            print(f"⚠️  Skipping {pdf_name} (no markdown output)")
            continue

        with open(md_file) as f:
            content = f.read()

        # Simple chunking by paragraphs
        paragraphs = content.split('\n\n')
        current_chunk = ""

        for para in paragraphs:
            if len(current_chunk) + len(para) > chunk_size:
                # Save current chunk
                if current_chunk:
                    chunks.append({
                        "text": current_chunk.strip(),
                        "source": pdf_name,
                        "source_file": str(pdf_file)
                    })
                current_chunk = para
            else:
                current_chunk += "\n\n" + para

        # Add remaining chunk
        if current_chunk:
            chunks.append({
                "text": current_chunk.strip(),
                "source": pdf_name,
                "source_file": str(pdf_file)
            })

    # Save chunked dataset
    output_file = Path(output_dir) / "rag_chunks.json"
    with open(output_file, 'w') as f:
        json.dump(chunks, f, indent=2)

    print(f"✅ Created {len(chunks)} chunks")
    print(f"💾 Saved to: {output_file}")

    return chunks

# Usage
chunks = prepare_rag_dataset("./knowledge_base", chunk_size=1000)

# Sample output
print(f"\nSample chunk:")
print(f"Source: {chunks[0]['source']}")
print(f"Text: {chunks[0]['text'][:200]}...")
```

**Integrate with vector database**:

```python
def ingest_into_vectordb(chunks: List[Dict], collection_name: str = "documents"):
    """
    Ingest chunks into a vector database (example with ChromaDB).

    Note: Requires `pip install chromadb sentence-transformers`
    """
    import chromadb
    from chromadb.utils import embedding_functions

    # Initialize client
    client = chromadb.Client()

    # Create collection with embedding function
    sentence_transformer_ef = embedding_functions.SentenceTransformerEmbeddingFunction(
        model_name="all-MiniLM-L6-v2"
    )

    collection = client.create_collection(
        name=collection_name,
        embedding_function=sentence_transformer_ef
    )

    # Add documents
    collection.add(
        documents=[chunk["text"] for chunk in chunks],
        metadatas=[{"source": chunk["source"]} for chunk in chunks],
        ids=[f"chunk_{i}" for i in range(len(chunks))]
    )

    print(f"✅ Ingested {len(chunks)} chunks into ChromaDB")

# Full pipeline
chunks = prepare_rag_dataset("./docs")
ingest_into_vectordb(chunks)
```

**When to use**:
- Building knowledge bases for LLMs
- Question-answering systems
- Document search and retrieval
- AI-powered research assistants

---

### Pattern 6: API Server for Production

**Scenario**: Deploy MinerU as a production REST API service.

**Setup API server**:

```python
# server.py
from fastapi import FastAPI, UploadFile, File, HTTPException
from pathlib import Path
import tempfile
import shutil
from mineru.cli.client import parse_doc

app = FastAPI(title="MinerU API", version="1.0")

@app.post("/parse")
async def parse_pdf(
    file: UploadFile = File(...),
    backend: str = "hybrid-auto-engine",
    language: str = "en"
):
    """
    Parse uploaded PDF and return markdown content.
    """
    # Validate file type
    if not file.filename.endswith('.pdf'):
        raise HTTPException(400, "Only PDF files are supported")

    # Save uploaded file temporarily
    with tempfile.NamedTemporaryFile(delete=False, suffix=".pdf") as tmp:
        shutil.copyfileobj(file.file, tmp)
        tmp_path = tmp.name

    # Create output directory
    output_dir = tempfile.mkdtemp()

    try:
        # Process PDF
        parse_doc(
            path_list=[Path(tmp_path)],
            output_dir=output_dir,
            backend=backend,
            lang=language
        )

        # Read markdown output
        pdf_name = Path(tmp_path).stem
        md_file = Path(output_dir) / pdf_name / "auto" / f"{pdf_name}.md"

        with open(md_file) as f:
            markdown_content = f.read()

        return {
            "status": "success",
            "filename": file.filename,
            "markdown": markdown_content
        }

    except Exception as e:
        raise HTTPException(500, f"Processing failed: {str(e)}")

    finally:
        # Cleanup
        Path(tmp_path).unlink()
        shutil.rmtree(output_dir)

@app.get("/health")
def health_check():
    """Health check endpoint."""
    return {"status": "healthy"}

# Run with: uvicorn server:app --host 0.0.0.0 --port 8000
```

**Client usage**:

```python
# client.py
import requests

def parse_pdf_via_api(pdf_p

…(truncated)