# Digest PDF

> Digests one or many PDFs into a structured on-disk tree of (image, transcription) Units — textbooks (chapter/section tree), exam papers (per question, each as TWO images: with and without the worked solution), or plain pages. On invocation it analyzes every target PDF's structure, confirms the invoking repo's output **Layout Spec** once (bootstrapping `.digest/layout.yaml` when it's absent, divergent, or a filename doesn't match), then runs the zero-token digest-pdf tool as a fast unattended batch (a warm MinerU server + cross-PDF parallelism) and verifies every Unit has its image+text pair. Use when the user says "digest these PDFs", "split this PDF into images and text", "turn these exam papers / textbooks / notes into a wiki-ready tree", "batch-process these PDFs", "extract questions / chapters from these PDFs", "/digest-pdf", or otherwise points at PDF file(s)/dir(s) and wants structured (img, md) output on disk.

- Skill: `sunfmin/digest-pdf` (Agent Skill)
- Install (CLI): `npx skillmds@latest add sunfmin/digest-pdf`
- Raw SKILL.md: https://api.skillmd.com/api/skills/sunfmin/digest-pdf/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Docs & Writing
- Author: sunfmin (https://skillmd.com/u/sunfmin)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/sunfmin/digest-pdf

---


# digest-pdf — digest PDFs into a structured (image + text) tree

You orchestrate the **digest-pdf** tool. The tool does all recognition/cutting with **zero
LLM tokens** (MinerU is the sole transcriber, ADR-0010); your job is to understand the invoking
repo, confirm its **Layout Spec** (where Units land — CONTEXT: Layout Spec, ADR-0008) once, run a
fast batch, and verify. **Never** OCR or transcribe pages yourself by reading images — that
defeats the tool's design and is slow/costly.

## Inputs (parse from the user's message)

- **targets** — one or more PDF files and/or directories (the tool recurses dirs for `*.pdf`).
- **--out** — base override for the output root. Usually **omit it**: when the invoking repo
  has a Layout Spec, the tool lands each PDF per its `path` template resolved from the repo root.
  Pass `--out` only to write outside the repo (or on a no-spec run, where each PDF lands under
  `<out>/<pdf-stem>/`).
- **--strategy** — `auto` (default) | `page` | `outline` | `question`. A matching Layout Spec
  rule pins the strategy per PDF (overriding this); pass `--strategy` only on a no-spec run.
- **--layout** — path to a Layout Spec; defaults to auto-discovering `.digest/layout.yaml` by
  walking up from cwd.

## Invoke the tool

The tool is a published Python CLI; run it straight from its git repo with **uvx** — no clone,
no fixed install path, works the same on any machine:

```sh
DIGEST="uvx --from git+https://github.com/sunfmin/digest-pdf digest"
```

Every call below is `$DIGEST <args…>` (uvx resolves + caches the tool on first use). All commands
are run from the **invoking repo's** working directory — that (via cwd) is the only directory this
skill cares about; the tool manages its own venv and model cache internally, so you never reference
those paths.

## Workflow

### 0 — Understand the repo + confirm the Layout Spec (the gate)
The tool lands Units per a repo-owned **Layout Spec** (`<repo>/.digest/layout.yaml`, ADR-0008):
an ordered rule table, each rule mapping a **filename pattern** (regex + named captures like
year/region/subject) → a **segmentation strategy** + a **destination path template** (e.g.
`真题/{region}/{year}/{subject}/q{qno}`), resolved from the repo root. The spec is the one thing
you confirm — **once** — before an unattended batch. Steady-state runs (spec already matches)
are unattended; the gate only fires when the spec is absent, divergent, or incomplete.

1. Find the invoking repo (walk up from cwd for `.digest/`, else cwd); read its `CONTEXT.md`,
   if present, for naming conventions.
2. Probe + see matches in one cheap call (no MinerU/VLM):
   `$DIGEST --inspect <targets...>`
   Each row now carries `layout: {status, rule, strategy, dest}`; `status` ∈
   `matched | unmatched | no-spec`.
3. Gate on the result:
   - **Every row `matched`, strategy unambiguous** → *proceed unattended*: show the plan table
     (B) **once** and run. Do **not** ask for per-PDF confirmation.
   - **`no-spec`, any `unmatched`, an ambiguous strategy, or a plan that diverges from the stored
     spec** → **STOP and confirm.** Propose a Layout Spec: derive filename-pattern rules from the
     target stems + the repo's conventions, pick each rule's strategy, write `path` templates that
     fit the repo. Present it as a YAML block, wait for the user to approve or adjust, then
     **write** `<repo>/.digest/layout.yaml` and re-run `--inspect`.
   - An **`unmatched`** stem → offer to add a rule that covers it; never fall back silently to the
     tool's native `<stem>/` layout.

### A — Ensure readiness (one-time, only if needed)
Using the Phase 0 `--inspect` rows: if any row has `needs_mineru: true`, make sure MinerU is
installed. Ask the tool (no cache path to stat) and install once if missing — idempotent, ~2 GB of
models via ModelScope; print a one-line notice, then continue (the layout is already confirmed):
```sh
$DIGEST --mineru-status || $DIGEST --install-mineru
```

### B — The `--inspect` row shape + plan table (reference for Phase 0)
Each element of the `--inspect` JSON array =
`{path, pages, strategy, needs_mineru, out_subdir, estimate, layout}`.
- `strategy` is the resolved segmentation (a matching Layout Spec rule pins it); `layout` =
  `{status, rule, strategy, dest}`, `status` ∈ `matched | unmatched | no-spec`.
- `estimate`: for `question` = detected question count on text-layer PDFs, or
  `"unknown (scanned)"` when there's no text layer (normal — MinerU resolves it at run time);
  for `outline` = a note that the chapter/section tree is resolved *after* transcription;
  for `page` = the page count.

In Phase 0's unattended branch, render a markdown table for the user **once** (no per-PDF
confirmation once the spec is confirmed):
`| PDF | pages | strategy | dest | estimate | needs mineru |` — `dest` is the row's
`layout.dest` (the tool's native `<stem>/` when there is no spec).

### C — Warm the MinerU server (only if some row needs_mineru)
A warm server keeps models loaded across the whole batch (the per-PDF cold load is the slow part).
The tool owns its own venv/config, so just background `--serve-mineru`, wait for health, and export
the URL; the tool's runner then talks to it over HTTP automatically. If it fails to come up, warn
and continue without the URL (the tool falls back to per-PDF CLI — slower but works).
```sh
PORT="${MINERU_PORT:-8765}"
$DIGEST --serve-mineru --port "$PORT" >/dev/null 2>&1 &
SERVER_PID=$!
for i in $(seq 1 60); do curl -sf "http://127.0.0.1:$PORT/health" >/dev/null && break; sleep 2; done
export MINERU_API_URL="http://127.0.0.1:$PORT"
```
Skip this block entirely if no row needs MinerU.

### D — Run the batch (one process; parallel inside)
With a confirmed Layout Spec, **omit `--out` and `--strategy`** — the spec supplies the base
(repo root) and pins the strategy per PDF. Pass them only to override, or on a no-spec run.
```sh
$DIGEST <targets...> --concurrency "${DIGEST_CONCURRENCY:-}"
# no-spec / override:  add  --out "$OUT" --strategy <s>
```
The tool parallelizes across PDFs (planning + MinerU submits) and across pages (render/write)
within this single process, so one call saturates the machine — do **not** fan out multiple
`digest` processes (they'd race on the shared manifest). Re-running resumes from the manifest.

### E — Verify
Read `<out>/manifest.json`: per PDF, count `done` vs `failed` pages. Cross-check on disk that
every recorded Unit has **both** files:
```sh
# for each unit dir, every *.md should have a same-named *.png (and vice versa)
```
Use Glob/Read; flag any Unit missing its pair or any `failed` page.

### F — Tear down
```sh
[ -n "${SERVER_PID:-}" ] && kill "$SERVER_PID" 2>/dev/null   # only if C started a server
```

## Output layout (what the tool produces, per strategy)
The `<out>/<stem>/` prefix below is the **native (no-spec)** layout. With a matching Layout Spec
rule, that prefix is replaced by the rule's resolved `dest` directory (e.g. `真题/浙江/2016/理/`);
the leaf names (`page-NNNN`, `qNN`, the `第N章/<section>/` subtree) are unchanged in v1.
- **page** — `<out>/<stem>/page-NNNN.{png,md}` (flat).
- **outline** — `<out>/<stem>/第N章/<section>/page-NNNN.{png,md}` (tree resolved post-transcription).
- **question** — `<out>/<stem>/qNN.{png,md}` = full question (stem + options + solution), plus
  `<out>/<stem>/qNN-stem.{png,md}` = the question **without** the solution (cut just above
  `【答案】`/`【解析】`/`【分析】`/`【详解】`). A question whose PDF has no solution marker gets
  only `qNN` (no `-stem`) — that's expected, not an error.

Every `.md` carries a provenance header (`model@revision`, dpi, strategy, source page(s)).

## Report
End with a summary table: `| PDF | strategy | Units | ok | failed |` plus the output root path and
any caveats (scanned PDFs whose question count was unknown until run; questions without a stem
image; any failed pages with a suggested re-run).

## Boundaries / gotchas
- `auto` on a scanned PDF with no exploitable structure → `page`. The user can force `--strategy
  question` for scanned exam papers (MinerU handles them).
- The warm server (C) is what makes big batches fast; if you skipped it and the batch is slow,
  that's why — offer to re-run with it.
- The one file you may write is `<invoking-repo>/.digest/layout.yaml` (the Layout Spec bootstrap,
  Phase 0). Do not push, do not edit the tool's lockfile — this skill only *uses* the tool.