PDF To Epub

PDF → EPUB conversion

Converting a PDF into a genuinely reflowing ebook is reconstruction, not extraction. A PDF has no paragraphs — only positioned glyphs. Getting a result that's actually pleasant to read (paragraphs that flow, headings that are headings, figures that land where they belong, footnotes that work as links) takes deliberate, book-specific reverse-engineering of this book's particular typesetting, followed by real verification. A one-shot pdftotext | pandoc produces a wall of hard-wrapped text with running headers baked into the body — worse than the PDF it came from.

There are two distinct tasks this skill covers — figure out which one applies before doing anything else:

• Converting from a PDF (the common case) → the workflow below.
• Repairing a pre-existing EPUB that's already reflowing but fails to validate or import somewhere → read references/epub_repair.md instead; it's a narrower, minimal-diff task, not a rebuild.

The workflow, for a PDF conversion

0. Check tooling once, up front

Prefer ephemeral installs over assuming anything is present:

for c in pdftotext pdfimages pdfinfo qpdf tesseract epubcheck ebook-convert java; do
  command -v "$c" >/dev/null && echo "$c OK" || echo "$c -"
done

• pdftotext/pdfimages/pdfinfo/qpdf — raw PDF inspection (usually present via poppler-utils).
• tesseract — only needed if the PDF turns out to be scanned (see the probe step); OCR first, then treat the OCR'd text layer as the input.
• PyMuPDF, ebooklib, beautifulsoup4, lxml, pillow — the actual conversion toolchain. Don't install these into the system Python; run every script through uv run --with pymupdf --with ebooklib ... python script.py, which fetches an ephemeral environment with zero footprint and no permission prompts.
• epubcheck (needs a JRE) and calibre (ebook-convert, ebook-meta) are independent validators, not build dependencies — get them installed via the system package manager if missing (e.g. pacman -S epubcheck calibre jre-openjdk-headless on Arch). This needs root, so ask the user to run it rather than trying it yourself.

1. Probe before writing a single line of conversion code

Run scripts/probe_pdf.py book.pdf. This tells you, in one pass: outline/bookmarks present or not, the font-size/family census (which reveals the body-text style and whether headings differ by size or by family), whether it's a real text layer or a scan, and where the images are. Then read a handful of actual pages at the span level — dump every line's text, bounding box, font, size, and bold/italic flags for a representative sample (front matter, a typical body page, a page with a figure, a page with a table if there is one) so you can see the real geometry rather than guessing it. This probe determines everything that follows; do not start writing extraction code before you've looked.

Specifically nail down, from real data, before writing anything else:

• Is there a PDF outline, or does the heading structure need to come from the printed table of contents? (references/extraction_patterns.md §1)
• What signals a heading — size, font family, boldness, some combination?
• What's the running head/footer band (a y-coordinate range near the top and bottom of the page to strip from body text)?
• Roughly how many figures, tables, and footnotes are there, and what do they look like at the span level? A single ruled table is worth handling differently than forty of them.

2. Build the structural spine

Get every heading matched to its location in the document, with correct levels, before touching paragraph reflow. See references/extraction_patterns.md §1: a real outline, a parsed printed TOC + folio map, or — for a short document with neither — the style census itself as the direct heading source, no matching step needed. Verify the match count where there is one to verify: every TOC/outline entry should land somewhere in the document — if some don't, that's worth fixing now, since a chapter with a broken heading becomes an un-navigable wall of text in the final book.

3. Classify page regions, then extract

For each page, before deciding what's body text and what isn't: cluster bounded regions (images + vector paths) and classify each one as a real figure, a ruled data table, or a prose callout box that must still flow as text (references/extraction_patterns.md §2). This ordering matters — classify first, decide what to strip second — because the callout-box case is easy to get backwards and silently drop real paragraphs.

Check for two-column layout per page (common in back-of-book indexes, sometimes elsewhere) and read in the right order (references/extraction_patterns.md §3).

Assemble lines into paragraph blocks using whatever combination of signals actually holds up for this book — leading-space quirks, y-gaps, bullet markers, font-size changes (§4) — verified by dumping a sample and looking at it, not assumed from what worked on a different book.

Handle de-hyphenation corpus-aware (§5) and footnote/reference markers by font-size, with the marker self-describing its true page (§6).

4. Build the EPUB

ebooklib is the practical choice for building the container (spine, manifest, nav, NCX) without hand-rolling XML. It has real, undocumented gotchas that fail silently — read references/ebooklib_gotchas.md before debugging a mysterious empty page or a crash inside ebooklib's own code.

Build a real nested navigation document (multiple heading levels, not just top-level chapters) — for a reference-heavy technical book, navigating straight to a numbered section/rule/requirement matters far more than it would in a novel. Clip-render figures at a generous DPI (≈200) so they hold up on a high-density e-reader screen. Give the CSS enough structure to distinguish headings, captions, asides, and body text — readable is the bar, not decorative.

5. QA — every time, no exceptions

Four checks, in this order, and don't report the job done until all four have actually run:

1. epubcheck the-book.epub — the objective bar is 0 errors. Read every error message; each one points at something concretely broken (undeclared manifest entry, malformed nested list, a missing fragment target). Fix and re-run until clean.

2. scripts/word_diff_qa.py book.pdf book.epub — a word-level fidelity check. Read the reminder it prints: the percentage is a lower bound, not the answer. A footnote correctly relocated to an end-of-chapter Notes section, a figure caption pulled to sit beside its image, a table's cells reordered from column-major to row-major — all of these are correct structural transformations that a linear diff sees as loss. Before quoting a number to the user, spot-check several of the largest MISSING lines: grep a distinctive phrase from one against the built EPUB's actual text. If it's in there, say the real number is higher and explain why, rather than reporting the raw score as if it were the truth.

3. Render and actually look — scripts/render_and_look.py book.epub --find "some heading text" "Figure 2.1" --out look.png --width 420 (and again with a narrower width, e.g. 260, to check phone-width reflow). Then use your Read tool on the saved PNG. This is the only step that catches a paragraph broken mid-sentence, a heading that isn't styled as a heading, a figure sitting on top of its caption, or a table that doesn't fit at narrow width. epubcheck and a word-diff both can pass clean on a file that looks wrong; only looking catches that.

4. ebook-convert book.epub /tmp/check.txt and ebook-meta book.epub — an independent parser (calibre) as a second opinion. If calibre chokes on something epubcheck didn't flag, that's worth knowing before the user's e-reader does the same thing.

6. Report honestly

State the epubcheck result plainly (0 errors, or what's still open and why). Give the word-diff percentage with the lower-bound caveat once you've actually spot-checked it, not as a bare number. Name anything you deliberately chose not to fix and why (a one-off table too narrow a case to justify bespoke handling, a stray warning that's cosmetic) — that's different from silently leaving something broken, and the user should be able to tell the difference from how you report it.

What doesn't transfer between books

Every concrete number in this skill so far — a body font size, a left margin, an indent threshold, which font family means "heading" — was measured from a specific book and will very likely be wrong for the next one. Re-derive them each time from the probe step. What does transfer is the method: probe first, classify regions before stripping text, verify de-hyphenation against the book's own corpus, detect columns from real geometry, and close the loop with epubcheck + word-diff + an actual look before calling it done.