tools-only System Design & Architecture
Architecture Overview
What is the high-level system structure?
The pdf_extractor acts as a data source for the validation engine. It encapsulates all PDF-specific complexity.
graph LR
PDF[PDF File] --> Fitz[PyMuPDF/fitz]
Fitz --> Extractor[pdf_extractor.py]
Extractor --> Canonicalizer[text_canonicalizer.py]
Canonicalizer --> Output[Clean Text]
Data Models
What data do we need to manage?
- PDFContent: A simple container or string containing the extracted text.
- ExtractionMetadata: Page count, info (title, author).
API Design
How do components communicate?
Primary Interface
class PDFExtractor:__init__(file_path: Path)get_full_text() -> striter_pages() -> Iterator[str](Memory efficient extraction)get_page_text(page_num: int) -> strget_metadata() -> dict
Component Breakdown
What are the major building blocks?
- Fitz Wrapper: Handles opening and reading the PDF.
- Text Cleaner: Integrates with
text_canonicalizer. - Layout Manager: Uses
page.get_text("blocks")to respect reading order and preserve paragraph-like structures.
Design Decisions
Why did we choose this approach?
- PyMuPDF (fitz): Chosen for its speed and superior "block-based" extraction which helps preserve paragraphs better than line-based tools.
- Direct Canonicalization: We canonicalize at the source to ensure no "dirty" text ever enters the system.
Non-Functional Requirements
How should the system perform?
- Robustness: Handle encrypted PDFs (raise clear exception).
- Speed: Extract a 300-page book in < 5 seconds.