166 Status 76beec6a

PDF Image Extraction (Task B1.5)

Status: ✅ Completed Date: October 21, 2025 Task: B1.5 - Add PDF image extraction (diagrams, screenshots)

Overview

Task B1.5 adds the ability to extract images (diagrams, screenshots, charts) from PDF documentation and save them as separate files. This is essential for preserving visual documentation elements in skills.

New Features

✅ 1. Image Extraction to Files

Extract embedded images from PDFs and save them to disk:

# Extract images along with text
python3 cli/pdf_extractor_poc.py manual.pdf --extract-images

# Specify output directory
python3 cli/pdf_extractor_poc.py manual.pdf --extract-images --image-dir assets/images/

# Filter small images (icons, bullets)
python3 cli/pdf_extractor_poc.py manual.pdf --extract-images --min-image-size 200

✅ 2. Size-Based Filtering

Automatically filter out small images (icons, bullets, decorations):

• Default threshold: 100x100 pixels
• Configurable: --min-image-size
• Purpose: Focus on meaningful diagrams and screenshots

✅ 3. Image Metadata

Each extracted image includes comprehensive metadata:

{
  "filename": "manual_page5_img1.png",
  "path": "output/manual_images/manual_page5_img1.png",
  "page_number": 5,
  "width": 800,
  "height": 600,
  "format": "png",
  "size_bytes": 45821,
  "xref": 42
}

✅ 4. Automatic Directory Creation

Images are automatically organized:

• Default: output/{pdf_name}_images/
• Naming: {pdf_name}_page{N}_img{M}.{ext}
• Formats: PNG, JPEG, GIF, BMP, etc.

Usage Examples

Basic Image Extraction

# Extract all images from PDF
python3 cli/pdf_extractor_poc.py tutorial.pdf --extract-images -v

Output:

📄 Extracting from: tutorial.pdf
   Pages: 50
   Metadata: {...}
   Image directory: output/tutorial_images

  Page 1: 2500 chars, 3 code blocks, 2 headings, 0 images
  Page 2: 1800 chars, 1 code blocks, 1 headings, 2 images
    Extracted image: tutorial_page2_img1.png (800x600)
    Extracted image: tutorial_page2_img2.jpeg (1024x768)
  ...

✅ Extraction complete:
   Images found: 45
   Images extracted: 32
   Image directory: output/tutorial_images

Custom Image Directory

# Save images to specific directory
python3 cli/pdf_extractor_poc.py manual.pdf --extract-images --image-dir docs/images/

Result: Images saved to docs/images/manual_page*_img*.{ext}

Filter Small Images

# Only extract images >= 200x200 pixels
python3 cli/pdf_extractor_poc.py guide.pdf --extract-images --min-image-size 200 -v

Verbose output shows filtering:

  Page 5: 3200 chars, 4 code blocks, 3 headings, 3 images
    Skipping small image: 32x32
    Skipping small image: 64x48
    Extracted image: guide_page5_img3.png (1200x800)

Complete Extraction Workflow

# Extract everything: text, code, images
python3 cli/pdf_extractor_poc.py documentation.pdf \
  --extract-images \
  --min-image-size 150 \
  --min-quality 6.0 \
  --chunk-size 20 \
  --output documentation.json \
  --verbose \
  --pretty

Output Format

Enhanced JSON Structure

The output now includes image extraction data:

{
  "source_file": "manual.pdf",
  "total_pages": 50,
  "total_images": 45,
  "total_extracted_images": 32,
  "image_directory": "output/manual_images",
  "extracted_images": [
    {
      "filename": "manual_page2_img1.png",
      "path": "output/manual_images/manual_page2_img1.png",
      "page_number": 2,
      "width": 800,
      "height": 600,
      "format": "png",
      "size_bytes": 45821,
      "xref": 42
    }
  ],
  "pages": [
    {
      "page_number": 1,
      "images_count": 3,
      "extracted_images": [
        {
          "filename": "manual_page1_img1.jpeg",
          "path": "output/manual_images/manual_page1_img1.jpeg",
          "width": 1024,
          "height": 768,
          "format": "jpeg",
          "size_bytes": 87543
        }
      ]
    }
  ]
}

File System Layout

output/
├── manual.json                          # Extraction results
└── manual_images/                       # Image directory
    ├── manual_page2_img1.png           # Page 2, Image 1
    ├── manual_page2_img2.jpeg          # Page 2, Image 2
    ├── manual_page5_img1.png           # Page 5, Image 1
    └── ...

Technical Implementation

Image Extraction Method

def extract_images_from_page(self, page, page_num):
    """Extract images from PDF page and save to disk"""

    extracted = []
    image_list = page.get_images()

    for img_index, img in enumerate(image_list):
        # Get image data from PDF
        xref = img[0]
        base_image = self.doc.extract_image(xref)

        image_bytes = base_image["image"]
        image_ext = base_image["ext"]
        width = base_image.get("width", 0)
        height = base_image.get("height", 0)

        # Filter small images
        if width < self.min_image_size or height < self.min_image_size:
            continue

        # Generate filename
        image_filename = f"{pdf_basename}_page{page_num+1}_img{img_index+1}.{image_ext}"
        image_path = Path(self.image_dir) / image_filename

        # Save image
        with open(image_path, "wb") as f:
            f.write(image_bytes)

        # Store metadata
        image_info = {
            'filename': image_filename,
            'path': str(image_path),
            'page_number': page_num + 1,
            'width': width,
            'height': height,
            'format': image_ext,
            'size_bytes': len(image_bytes),
        }

        extracted.append(image_info)

    return extracted

Performance

Extraction Speed

Note: Image extraction adds 10-20% overhead depending on image count and size.

Storage Requirements

• PNG images: ~10-500 KB each (diagrams)
• JPEG images: ~50-2000 KB each (screenshots)
• Typical documentation (100 pages): ~50-200 MB total

Supported Image Formats

PyMuPDF automatically handles format detection and extraction:

• ✅ PNG (lossless, best for diagrams)
• ✅ JPEG (lossy, best for photos)
• ✅ GIF (animated, rare in PDFs)
• ✅ BMP (uncompressed)
• ✅ TIFF (high quality)

Images are extracted in their original format.

Filtering Strategy

Why Filter Small Images?

PDFs often contain:

• Icons: 16x16, 32x32 (UI elements)
• Bullets: 8x8, 12x12 (decorative)
• Logos: 50x50, 100x100 (branding)

These are usually not useful for documentation skills.

Recommended Thresholds

Set with: --min-image-size N

Integration with Skill Seeker

Future Workflow (Task B1.6+)

When building PDF-based skills, images will be:

1. Extracted from PDF documentation
2. Organized into skill's assets/ directory
3. Referenced in SKILL.md and reference files
4. Packaged in final .zip file

Example:

# API Architecture

See diagram below for the complete API flow:

![API Flow](assets/images/api_flow.png)

The diagram shows...

Limitations

Current Limitations

1. No OCR

   • Cannot extract text from images
   • Code screenshots are not parsed
   • Future: Add OCR support for code in images

2. No Image Analysis

   • Cannot detect diagram types (flowchart, UML, etc.)
   • Cannot extract captions
   • Future: Add AI-based image classification

3. No Deduplication

   • Same image on multiple pages extracted multiple times
   • Future: Add image hash-based deduplication

4. Format Preservation

   • Images saved in original format (no conversion)
   • No optimization or compression

Known Issues

1. Vector Graphics

   • Some PDFs use vector graphics (not images)
   • These are not extracted (rendered as part of page)
   • Workaround: Use PDF-to-image tools first

2. Embedded vs Referenced

   • Only embedded images are extracted
   • External image references are not followed

3. Image Quality

   • Quality depends on PDF source
   • Low-res source = low-res output

Troubleshooting

No Images Extracted

Problem: total_extracted_images: 0 but PDF has visible images

Possible causes:

1. Images are vector graphics (not raster)
2. Images smaller than --min-image-size threshold
3. Images are page backgrounds (not embedded images)

Solution:

# Try with no size filter
python3 cli/pdf_extractor_poc.py input.pdf --extract-images --min-image-size 0 -v

Permission Errors

Problem: PermissionError: [Errno 13] Permission denied

Solution:

# Ensure output directory is writable
mkdir -p output/images
chmod 755 output/images

# Or specify different directory
python3 cli/pdf_extractor_poc.py input.pdf --extract-images --image-dir ~/my_images/

Disk Space

Problem: Running out of disk space

Solution:

# Check PDF size first
du -h input.pdf

# Estimate: ~100-200 MB per 100 pages with images
# Use higher min-image-size to extract fewer images
python3 cli/pdf_extractor_poc.py input.pdf --extract-images --min-image-size 300

Examples

Extract Diagram-Heavy Documentation

# Architecture documentation with many diagrams
python3 cli/pdf_extractor_poc.py architecture.pdf \
  --extract-images \
  --min-image-size 250 \
  --image-dir docs/diagrams/ \
  -v

Result: High-quality diagrams extracted, icons filtered out.

Tutorial with Screenshots

# Tutorial with step-by-step screenshots
python3 cli/pdf_extractor_poc.py tutorial.pdf \
  --extract-images \
  --min-image-size 400 \
  --image-dir tutorial_screenshots/ \
  -v

Result: Full screenshots extracted, UI icons ignored.

API Reference with Small Charts

# API docs with various image sizes
python3 cli/pdf_extractor_poc.py api_reference.pdf \
  --extract-images \
  --min-image-size 150 \
  -o api.json \
  --pretty

Result: Charts and graphs extracted, small icons filtered.

Command-Line Reference

Image Extraction Options

--extract-images
    Enable image extraction to files
    Default: disabled

--image-dir PATH
    Directory to save extracted images
    Default: output/{pdf_name}_images/

--min-image-size PIXELS
    Minimum image dimension (width or height)
    Filters out icons and small decorations
    Default: 100

Complete Example

python3 cli/pdf_extractor_poc.py manual.pdf \
  --extract-images \
  --image-dir assets/images/ \
  --min-image-size 200 \
  --min-quality 7.0 \
  --chunk-size 15 \
  --output manual.json \
  --verbose \
  --pretty

Comparison: Before vs After

Next Steps

Task B1.6: Full PDF Scraper CLI

The image extraction feature will be integrated into the full PDF scraper:

# Future: Full PDF scraper with images
python3 cli/pdf_scraper.py \
  --config configs/manual_pdf.json \
  --extract-images \
  --enhance-local

Task B1.7: MCP Tool Integration

Images will be available through MCP:

# Future: MCP tool
result = mcp.scrape_pdf(
    pdf_path="manual.pdf",
    extract_images=True,
    min_image_size=200
)

Conclusion

Task B1.5 successfully implements:

• ✅ Image extraction from PDF pages
• ✅ Automatic file saving with metadata
• ✅ Size-based filtering (configurable)
• ✅ Organized directory structure
• ✅ Multiple format support

Impact:

• Preserves visual documentation
• Essential for diagram-heavy docs
• Improves skill completeness

Performance: 10-20% overhead (acceptable)

Compatibility: Backward compatible (images optional)

Ready for B1.6: Full PDF scraper CLI tool

Task Completed: October 21, 2025 Next Task: B1.6 - Create pdf_scraper.py CLI tool