ryan-brosas PDF Extraction
Core Principle
Match the extraction library to the PDF's actual structure, and prove it on a small
sample before scaling up. Simple text → pdf-parse/pdfjs; structured data →
pdfplumber/camelot; complex layouts → vision model. Tables are not text, scanned
PDFs need OCR first, and structure must be preserved when it matters.
Iron Laws
When to Use
Extracting text from PDFs; tabular data; images; metadata; "convert this PDF to X"; batch processing; RAG.
When NOT to Use
Data exists in a non-PDF format (use that); copy-paste works; one-time, hand inspection.
Library Selection
| PDF type | Library |
|---|---|
| Simple text, English | pdf-parse, pdfplumber |
| Complex text, multi-column | pdfplumber (layout-aware) |
| Tables (financial, scientific) | camelot, tabula-py, pdfplumber |
| Scanned (image-based) | pytesseract (OCR), or vision model |
| Mixed (text + tables + images) | Vision model (GPT-4V, Claude) |
| Form fields | pdf-lib, pdfrw, pypdf |
When in doubt: pdfplumber for English text, vision model for weird layouts.
Workflow
- Inspect the PDF. Open it, look at the structure. Is it text or scanned? Tables? Forms? Images? Multi-column?
- Pick the library based on the structure.
- Test on a small sample. Extract 1-5 pages. Is the output what you expected?
- Scale up. Run on the full set. Watch for failures (encrypted, corrupted, etc.).
- Verify. Spot-check. Automated extraction lies silently, a missing column, a merged cell.
Common Patterns
# pdfplumber — text + tables
import pdfplumber
with pdfplumber.open("file.pdf") as pdf:
for page in pdf.pages:
text = page.extract_text()
tables = page.extract_tables()
# pytesseract — OCR
import pytesseract
from PIL import Image
text = pytesseract.image_to_string(Image.open("scan.png"))
Common Mistakes
pdftotext on a scanned PDF (no text to extract); pdf-parse on a multi-column PDF (text jumbled). flattening tables to text (loses structure). no verification (output silently wrong). choosing a library for a 500-page PDF without testing on 5 pages. OCR on text-based PDF (overkill, slower, less accurate). batch processing without error handling (one bad PDF kills the run).
Red Flags
No sample test before full run. no verification of output; pdftotext on scanned PDF. no error handling on batch. "the extraction worked on 1 page" assumption for the full 500. tables flattened to text. "I'll fix the output in post" (extraction should be correct). "the library says it supports tables" (test it on YOUR tables).
Anti-Patterns
Wrong library for the PDF (text vs scanned vs tables); no sample test; no verification; flattening tables (loses structure); OCR on text-based PDF; batch without error handling; "the library supports X" (test YOUR data); "I'll fix in post" (extraction should be right).
Verification
- Spot-check the extracted output against the source PDF, automated extraction lies silently (a missing column, a merged cell).
- Confirm the sample test (1–5 pages) produced the expected output before trusting the full run.
- For tables: verify row/column counts and cell alignment, not just that text came back.
- For batches: confirm error handling caught encrypted/corrupted files instead of one bad PDF killing the run.
References
N/A, no reference files; library selection is fully covered by the table in this file.