# Pdfstudio Impl Annotation Pipeline

> Use when modifying annotation handling in open-pdf-studio: loading, rendering, editing, or saving annotations. Prevents the common mistake of breaking the coordinate transform chain or losing annotation data during the full pipeline. Covers the end-to-end flow: PDF file → PDF.js parsing → app model → canvas rendering → user editing → pdf-lib serialization → saved PDF file. Keywords: annotations, pipeline, coordinate transform, CropBox, canvas overlay, dual canvas, rendering, saver, loader, annotation-converter, stamp, freetext, annotations wrong position, add annotation, annotation workflow.

- Skill: `impertio-studio/pdfstudio-impl-annotation-pipeline` (Agent Skill, multi-file: 4 files)
- Install (CLI): `npx skillmds@latest add impertio-studio/pdfstudio-impl-annotation-pipeline`
- Raw SKILL.md: https://api.skillmd.com/api/skills/impertio-studio/pdfstudio-impl-annotation-pipeline/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: DevOps & Infra
- License: MIT
- Author: Impertio-Studio (https://skillmd.com/u/impertio-studio)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/impertio-studio/pdfstudio-impl-annotation-pipeline

---


# Annotation Pipeline: End-to-End Flow

## Purpose

This skill documents the complete annotation lifecycle in open-pdf-studio: from PDF file bytes through parsing, coordinate transformation, canvas rendering, user editing, and back to PDF file. EVERY annotation modification MUST respect this pipeline or data will be lost or mispositioned.

## Pipeline Overview

```
┌─────────────────────────────────────────────────────────────────┐
│  1. LOAD                                                         │
│  PDF bytes → PDF.js page.getAnnotations()                        │
│           → pdf-lib color extraction (parallel)                  │
│           → annotation-converter.js → app model                  │
│                                                                  │
│  2. RENDER                                                       │
│  renderer.js → PDF.js renders page to pdf-canvas                 │
│  rendering.js → Canvas2D draws annotations on annotation-canvas  │
│                                                                  │
│  3. EDIT                                                         │
│  tool-dispatcher.js → creates/modifies annotation objects        │
│  → pushes to state.annotations (SolidJS reactive)                │
│  → triggers rendering.js redrawAnnotations()                     │
│                                                                  │
│  4. SAVE                                                         │
│  saver.js → PDFDocument.load(originalBytes) via pdf-lib          │
│  → strips old annotations → writes new annotation dicts          │
│  → coordinate transform (app → PDF) → pdfDocLib.save()           │
└─────────────────────────────────────────────────────────────────┘
```

## Three Coordinate Systems

This is the most critical concept. Three coordinate systems interact, and confusion between them is the primary source of annotation positioning bugs.

| System | Origin | Units | Y-axis | Used By |
|--------|--------|-------|--------|---------|
| **PDF coordinates** | Bottom-left | Points (1/72 inch) | Up (ascending) | PDF file spec, pdf-lib |
| **Viewport coordinates** | Top-left | Scaled pixels | Down (descending) | PDF.js after `getViewport()` |
| **App coordinates** | Top-left | Unscaled pixels (scale=1) | Down (descending) | `state.annotations`, Canvas2D |

### Loading Transform (PDF → App)

```javascript
// js/pdf/loader/annotation-converter.js
// PDF.js viewport handles the coordinate transform
const viewport = page.getViewport({ scale: 1, rotation: 0 });

const convertPoint = (pdfX, pdfY) => {
  return viewport.convertToViewportPoint(pdfX, pdfY);
  // Flips Y-axis, applies CropBox offset, scales to 1:1
};

const convertRect = (pdfRect) => {
  return viewport.convertToViewportRectangle(pdfRect);
  // [x1,y1,x2,y2] in PDF → [left,top,right,bottom] in viewport
};
```

ALWAYS use `viewport.convertToViewportPoint()` for loading. NEVER manually flip Y-axis during load — PDF.js handles CropBox, rotation, and scaling internally.

### Saving Transform (App → PDF)

```javascript
// js/pdf/saver.js
const cropBox = page.getCropBox();
const viewLeft = cropBox.x;
const viewTop = cropBox.y + cropBox.height;

// App coordinates → PDF coordinates
const convertX = (canvasX) => canvasX + viewLeft;
const convertY = (canvasY) => viewTop - canvasY;  // Y-axis flip
```

ALWAYS use `page.getCropBox()` as the reference frame during save. NEVER use `page.getMediaBox()` — CropBox defines the visible area that PDF.js rendered.

### Why CropBox Matters

```
MediaBox: [0, 0, 612, 792]     ← Physical page bounds
CropBox:  [36, 36, 576, 756]   ← Visible area (with margins)
```

PDF.js renders only the CropBox area. App coordinates are relative to the CropBox origin. During save, annotations MUST be offset by CropBox.x and CropBox.y to land in the correct PDF position. If CropBox equals MediaBox (common), the offset is zero and the issue is invisible — but the code MUST handle non-zero offsets.

## Dual-Canvas System

```
┌──────────────────────────────────┐
│  annotation-canvas (top z-index) │  ← rendering.js draws here
│  pointer-events: varies          │     Pure Canvas2D, no PDF library
├──────────────────────────────────┤
│  pdf-canvas (bottom)             │  ← PDF.js renders here
│  pointer-events: none            │     annotationMode: 0 (disabled)
├──────────────────────────────────┤
│  text-layer (HTML overlay)       │  ← PDF.js text selection
│  link-layer (HTML overlay)       │  ← PDF.js clickable links
│  form-layer (HTML overlay)       │  ← PDF.js interactive form fields
└──────────────────────────────────┘
```

**Key rule**: PDF.js annotation rendering is ALWAYS disabled (`annotationMode: 0` in `renderer.js`). The app draws ALL annotations itself on the overlay canvas. This gives full control over appearance and interaction but means the app MUST handle every annotation type it wants to display.

## Stage 1: Loading Annotations

### Flow

```
PDF.js page.getAnnotations()
    │
    ├── Returns raw annotation objects with:
    │   - subtype (Highlight, Square, FreeText, Stamp, etc.)
    │   - rect [x1, y1, x2, y2] in PDF coordinates
    │   - color, borderStyle, contents, etc.
    │
    ▼
annotation-converter.js
    │
    ├── Coordinate transform (PDF → viewport at scale=1)
    ├── Map PDF annotation type → app annotation type
    ├── Extract text content, border width, opacity
    │
    ├── Color extraction (parallel, via pdf-lib)
    │   js/pdf/loader/color-extraction.js
    │   → extractAnnotationColors(pageNum, pdfLibDoc)
    │   → Map<rectKey, { fillColor, strokeColor, opacity }>
    │
    ├── Image extraction (for stamps)
    │   js/pdf/loader/image-extraction.js
    │   → extractStampImagesViaPdfJs(page, annotations)
    │   → Map<annotIndex, dataURL>
    │
    └── Output: app annotation objects → doc.annotations[]
```

### Color Extraction Bridge

PDF.js does NOT expose all annotation colors (particularly `IC` / interior color and appearance stream colors). The app uses pdf-lib to extract these in parallel:

```javascript
// js/pdf/loader/color-extraction.js
export async function extractAnnotationColors(pageNum, pdfLibDoc) {
  // Parses the page's /Annots array via pdf-lib
  // Returns Map<rectKey, { fillColor, strokeColor, opacity }>
}
```

Colors are matched to PDF.js annotations via a fuzzy rect-key lookup (within 8 points tolerance) because PDF.js may adjust annotation rects by border width.

### Staleness Protection

The loader uses a `loadId` counter to prevent stale async results from overwriting current state:

```javascript
const loadId = ++doc._annotationLoadId;
// ... async work (PDF.js parsing, color extraction, image extraction) ...
if (loadId !== doc._annotationLoadId) return;  // Document was reloaded, abort
```

ALWAYS check `loadId` after every `await` in annotation loading code. NEVER skip this check — it prevents race conditions when the user rapidly switches documents.

### Document Closure Check

```javascript
const isClosed = () => !state.documents.includes(doc);
```

ALWAYS check `isClosed()` after async operations. If the document was closed while annotations were loading, writing to its state would cause errors.

## Stage 2: Rendering Annotations

### rendering.js: Canvas2D Drawing

`js/annotations/rendering.js` draws all annotations on the overlay canvas using pure Canvas2D API. It does NOT use PDF.js or pdf-lib for rendering.

```javascript
export function redrawAnnotations(canvas, annotations, scale, pageIndex) {
  const ctx = canvas.getContext('2d');
  ctx.clearRect(0, 0, canvas.width, canvas.height);

  for (const annot of annotations) {
    // Scale app coordinates by current zoom level
    // Draw based on annotation type (rect, highlight, freetext, stamp, etc.)
    drawAnnotation(ctx, annot, scale);
  }
}
```

Rendering ALWAYS works with app coordinates scaled by the current zoom level. The scale factor converts unscaled app coordinates to actual canvas pixels.

### Annotation Types

The app supports approximately 20 annotation types (defined in `js/types/annotation.ts`):

| Category | Types |
|----------|-------|
| **Markup** | Highlight, Underline, StrikeOut, Squiggly |
| **Shape** | Square, Circle, Line, Arrow, Polygon, Polyline |
| **Text** | FreeText, StickyNote (Text) |
| **Media** | Stamp, Image |
| **Drawing** | Ink (freehand) |
| **Link** | Link |

Each type has specific rendering logic in `rendering.js` and specific serialization logic in `saver.js`.

## Stage 3: User Editing

### tool-dispatcher.js

The tool dispatcher routes mouse/touch events to the active tool handler:

```
User click/drag on annotation-canvas
    │
    ▼
tool-dispatcher.js
    │
    ├── Hand tool → pan/scroll
    ├── Select tool → select/move/resize annotation
    ├── Highlight tool → create Highlight annotation
    ├── Rectangle tool → create Square annotation
    ├── FreeText tool → create FreeText annotation
    ├── Stamp tool → create Stamp annotation
    ├── Ink tool → create Ink annotation
    └── ... (one handler per tool)
```

### Annotation Creation

```javascript
// js/annotations/factory.js
export function createAnnotation(type, pageIndex, rect, options = {}) {
  return {
    id: generateId(),
    type,
    pageIndex,
    rect,           // { x, y, width, height } in app coordinates
    color: options.color || defaultColor,
    opacity: options.opacity || 1.0,
    // ... type-specific properties
  };
}
```

New annotations are created in app coordinates (top-left origin, unscaled). They are pushed to `state.annotations` (SolidJS reactive array), which triggers `redrawAnnotations()` automatically.

## Stage 4: Saving Annotations

### saver.js Flow

```
state.annotations (app model)
    │
    ▼
PDFDocument.load(originalBytes)          ◄── pdf-lib, from originalBytesCache
    │
    ├── For each page:
    │   ├── Get existing /Annots array
    │   ├── Strip annotations of handled types
    │   │   (Highlight, Square, FreeText, etc.)
    │   │   Keep unhandled types untouched
    │   │
    │   ├── For each app annotation on this page:
    │   │   ├── Convert app coords → PDF coords (Y-flip + CropBox offset)
    │   │   ├── Create PDF annotation dict with:
    │   │   │   - /Type /Annot
    │   │   │   - /Subtype (mapped from app type)
    │   │   │   - /Rect [x1, y1, x2, y2]
    │   │   │   - /C (color), /CA (opacity)
    │   │   │   - Type-specific entries
    │   │   └── Add to page's /Annots array
    │   │
    │   └── Persist form field values
    │       (PDF.js AnnotationStorage → pdf-lib form fields)
    │
    └── pdfDocLib.save() → Uint8Array
```

### Strip-and-Rewrite Strategy

The saver does NOT merge annotations. It ALWAYS:

1. Strips all annotations of types the app handles
2. Rewrites them from the app's annotation model

This means annotations the app handles are ALWAYS round-tripped through the app model. Annotations of types the app does NOT handle are preserved as-is in the PDF.

NEVER partially update existing PDF annotation dicts. ALWAYS strip and rewrite. This prevents state drift between the app model and the PDF file.

### Form Field Persistence

Form field values edited via the PDF.js form layer are stored in `PDF.js AnnotationStorage`. During save, these values are transferred to the corresponding pdf-lib form fields:

```javascript
// saver.js (simplified)
const annotationStorage = pdfDoc.annotationStorage;
for (const [key, value] of annotationStorage) {
  // Find corresponding pdf-lib form field and set its value
}
```

## Key Files

| File | Purpose |
|------|---------|
| `js/pdf/loader.js` | Orchestrates PDF loading, manages both PDF.js and pdf-lib instances |
| `js/pdf/loader/annotation-converter.js` | Transforms PDF.js annotations to app model (coordinate conversion) |
| `js/pdf/loader/color-extraction.js` | Extracts colors via pdf-lib that PDF.js cannot provide |
| `js/pdf/loader/image-extraction.js` | Extracts stamp images via PDF.js rendering fallback |
| `js/pdf/renderer.js` | PDF.js page rendering, manages text/link/form layers |
| `js/pdf/saver.js` | Converts app annotations to PDF dicts via pdf-lib, handles save |
| `js/annotations/rendering.js` | Canvas2D drawing of all annotation types |
| `js/annotations/factory.js` | Creates new annotation objects with defaults |
| `js/types/annotation.ts` | TypeScript definitions for ~20 annotation types |
| `js/tools/tool-dispatcher.js` | Routes input events to active tool handlers |
| `js/core/state.ts` | Central state, `state.annotations` is the source of truth |

## Coordinate Transform Quick Reference

### Load (PDF → App)

```
PDF rect [x1, y1, x2, y2] (bottom-left origin)
    → viewport.convertToViewportRectangle()
    → App rect { x, y, width, height } (top-left origin, scale=1)
```

### Render (App → Canvas)

```
App rect { x, y, width, height } (scale=1)
    → multiply by current zoom scale
    → Canvas pixel coordinates
```

### Save (App → PDF)

```
App rect { x, y, width, height } (top-left origin)
    → pdfX = x + cropBox.x
    → pdfY = (cropBox.y + cropBox.height) - y    // Y-flip
    → PDF rect [x1, y1, x2, y2] (bottom-left origin)
```