MinerU PDF to JSON
Use this skill when the input is a local directory of PDFs and the goal is to turn each undecoded PDF into a stable, per-PDF JSON artifact plus extracted image files.
Prefer the bundled script over writing one-off MinerU API code. The script is the stable contract for discovery, upload, polling, archive download, JSON promotion, image extraction, and final verification.
Required Environment
- Set
MINERU_API_TOKENbefore running the script, or store the token once in/root/.codex/skills/mineru-pdf-json/.mineru_api_token. - Run on one directory at a time; the scan is non-recursive.
- Expect outputs under a sibling directory named
pdf_decoded/.
Primary Command
python3 /root/.codex/skills/mineru-pdf-json/scripts/decode_pdfs.py --pdf-dir /path/to/pdfs
Useful options:
--batch-size 20by default; maximum200--model-version vlmor--model-version pipeline--poll-seconds 10by default--dry-runto stop after discovery anddecode_lists.mdgeneration
Output Contract
For a PDF directory like /data/folder1:
- Create
/data/folder1/decode_lists.md. - Ensure
/data/folder1/pdf_decoded/exists. - Treat
/data/folder1/pdf_decoded/<stem>/<stem>.jsonas the decoded marker. - Skip PDFs that already have that marker.
- For each undecoded PDF, save:
- JSON:
/data/folder1/pdf_decoded/<stem>/<stem>.json - Images:
/data/folder1/pdf_decoded/<stem>/images/<caption-or-fallback>.<ext>
- JSON:
- If final verification succeeds, delete
decode_lists.md. - If any file fails, keep
decode_lists.mdand surface the error instead of silently continuing.
The canonical JSON artifact is MinerU's *_content_list.json. Do not save model.json or middle.json as the final per-PDF result unless the user explicitly asks for a different output.
Workflow
- Run the decoder script.
- Let it build
decode_lists.mdfrom PDFs missing the decoded marker. - Let it request upload URLs from MinerU, upload the local PDFs, and poll
extract-results/batch. - Let it download each result archive, promote
*_content_list.jsonto<stem>.json, and extract image blocks throughimg_path. - Read
references/mineru_api.mdonly when you need the exact endpoint or output-file mapping.
Failure Handling
- Do not claim success if any pending PDF fails verification.
- Keep
decode_lists.mdon failure so the user can see which PDFs still need work. - Report concise, actionable failures:
- upload URL request rejected
- file upload failed
- MinerU returned
failed - result archive missing
*_content_list.json - image block references an archive path that does not exist
- saved JSON cannot be parsed on the final verification pass
Resources
scripts/decode_pdfs.py: main workflow implementationreferences/mineru_api.md: compact endpoint and output-format reference