Handwriting

Apple Notes Handwriting Reader

Apple Pencil drawings in Notes are invisible to AppleScript and every Notes MCP server — but each drawing is rendered to a FallbackImage.png on disk, and the local NoteStore.sqlite knows which drawings exist, when they changed, and which note they belong to. Claude reads those images directly (multimodal), which beats Apple's built-in handwriting OCR in accuracy, Korean included.

Workflow

1. Scan — run the bundled scanner (Python stdlib only, read-only, headless):

   uv run "${CLAUDE_PLUGIN_ROOT}/skills/handwriting/scripts/scan_handwriting.py"            # new since watermark (loop mode)
   uv run "${CLAUDE_PLUGIN_ROOT}/skills/handwriting/scripts/scan_handwriting.py" --all --limit 10   # browse recent (on-demand)
   

   Each output line is JSON: note_title, modified, modified_raw (the raw cursor value — keep it for the watermark step), image_path, image_bytes, handwriting_text (Apple's recognition text — a noisy hint), attachment_id. Loop mode emits oldest-first so a truncated batch is resumed next poll; --all browsing is newest-first.

2. Read — for each row, Read image_path (it is an image; Claude sees the strokes). Skip rows where image_bytes is under ~10 KB — those are blank or near-blank canvases; say so instead of inventing content.

3. Interpret — transcribe faithfully, then add a one-line reading of what the note is about. Trust the image over handwriting_text: Apple's recognizer produces errors like "Menory" for "Memory"; use its text only as a hint for ambiguous strokes or as a cheap pre-filter. Present per note: title, modified time, transcription, interpretation. When running as 마틴, emit the result over the Telegram reply channel as usual.

4. Advance the watermark — only in loop mode. After new handwriting has been successfully consumed (interpreted and delivered), run --update-watermark <max modified_raw of the consumed rows>. The value comes from the batch you actually consumed — never from a fresh scan — so notes that arrived in between are picked up next poll instead of being skipped. On-demand browsing must NOT advance the watermark. State lives at ~/.local/state/notes-handwriting/watermark.

Edge cases

• image_path null: the fallback render doesn't exist yet (drawing very fresh, or created on iPad and image not synced). Report the note by title and handwriting_text if present; it usually resolves on the next scan.
• Empty handwriting_text + tiny image (~7.5 KB blank render): either a genuinely empty canvas, or a placeholder whose real strokes haven't synced from the iPad yet (verified 2026-06: fresh iPad drawings can sit at a blank generation-3 render while older ones reach rich generation-14 renders). The stroke source of truth is a CloudKit asset, not the local DB (ZMERGEABLEDATA1 is empty for all drawings). Treat as pending: surface the note title, say the ink hasn't landed, and recheck on the next scan rather than concluding the note is empty.
• Locked notes: encrypted in the database and in fallback images — inaccessible by design; don't try to work around it.
• Schema drift: NoteStore.sqlite is a private schema. If the scanner errors after a macOS major update, check columns with PRAGMA table_info(ZICCLOUDSYNCINGOBJECT) — the load-bearing ones are ZTYPEUTI, ZHANDWRITINGSUMMARY, ZNOTE, ZTITLE1, ZMODIFICATIONDATE.
• Sync lag: the Mac sees handwriting only after iCloud syncs it from the iPad (NoteStore.sqlite mtime shows last sync). "I just wrote it" + no result usually means sync hasn't landed yet, not a bug.

Loop reuse contract

A future polling loop (cron / /loop) should call exactly: scan (no --all) → if rows: read + interpret + deliver → --update-watermark <max modified_raw of those rows>. No other state is needed; the watermark file is the single cursor shared by all consumers. The polling loop is the watermark's single writer (on-demand browsing never advances it); the monotonic check is not atomic across processes, so never run two consumers that both advance the cursor.