PDF Extractor

PDF Field Extraction Configuration Design

Design the extraction configuration for AI-based data extraction from insurance and mortgage PDFs. Covers document type identification, field definition, confidence thresholds, model selection, pre-processing requirements, and structured output schema.

Step 1: Document Type Identification

Before extracting fields, the system must identify what type of document is being processed. Route to the correct extraction model based on document type.

Document classification first: Run a classifier (see form-classifier skill) before the extraction model. The classifier returns a document type label that selects the correct field extraction configuration.

Document types and extraction priority:

Step 2: Field Definition by Document Type

Pay Stub

Annual income calculation:

Bi-weekly: ytd_gross / (pay_period_end_week_of_year / 2) × 26
Semi-monthly: ytd_gross / (pay_period_number) × 24
Monthly: gross_pay_this_period × 12
Hourly: hourly_rate × hours_per_week × 52 (use 2-year average if variable)

W-2

1040 Tax Return (Federal)

Bank Statement

Policy Declarations Page

Step 3: Extraction Confidence Thresholds

Field-specific thresholds:

Critical fields (wrong value has significant downstream impact) should have higher thresholds:

• Currency amounts: raise auto-accept to ≥ 0.93
• Dates: raise auto-accept to ≥ 0.93 (off-by-one errors on dates can affect compliance)
• SSN / EIN: raise auto-accept to ≥ 0.95; prefer human verification for all

Confidence aggregation:

• Document-level confidence = weighted average of field-level confidence scores (weight by field importance)
• If document-level confidence < 0.75: route entire document to human review, not field-by-field

Step 4: Model Selection

Model selection criteria:

• Use Azure Document Intelligence prebuilt models first (W-2, 1099, invoice) — highest accuracy for covered types
• For document types without a prebuilt model: train a custom Azure DI model using at least 50 labeled examples per document type (100+ preferred)
• AWS Textract as alternative if client is AWS-native and Azure DI is not preferred
• Never use general-purpose OCR (Tesseract) alone for structured financial documents — accuracy is insufficient for regulatory contexts

Step 5: Pre-Processing Requirements

Before sending to extraction model:

Quality gate: Any document failing minimum quality thresholds (DPI < 150, or confidence after pre-processing below floor) is routed to manual entry queue with specific quality failure message.

Step 6: Structured Output Schema

All extraction results output in consistent JSON format for downstream system consumption.

{
  "extraction_id": "uuid",
  "document_id": "uuid",
  "document_type": "pay_stub",
  "extraction_timestamp": "2024-01-15T14:32:00Z",
  "model_name": "azure-di-custom-pay-stub-v2",
  "model_version": "2.1.0",
  "document_confidence": 0.91,
  "page_count": 2,
  "fields": {
    "employer_name": {
      "value": "Acme Corporation",
      "confidence": 0.97,
      "bounding_box": {"page": 1, "x": 120, "y": 45, "width": 200, "height": 20},
      "status": "auto_accepted"
    },
    "gross_pay_this_period": {
      "value": 4250.00,
      "value_type": "currency",
      "confidence": 0.89,
      "bounding_box": {"page": 1, "x": 450, "y": 310, "width": 80, "height": 18},
      "status": "flagged_for_review"
    }
  },
  "missing_fields": ["employee_ssn_last4"],
  "validation_results": {
    "pay_period_end_vs_today": "within_60_days",
    "ytd_gross_ge_period_gross": "pass",
    "cross_field_consistency": "pass"
  },
  "routing": "auto_process",
  "review_reasons": ["gross_pay_confidence_below_threshold"]
}

Output Format

Deliver two artifacts:

1. Extraction Configuration Specification — For each document type in scope: field definition table (name, type, required, validation, extraction note), confidence thresholds, model selection, and pre-processing requirements

2. Output Schema Documentation — JSON schema for extraction results with field definitions, status enum values, routing logic, and validation rule definitions