PDF to Markdown

PDF to Markdown

Convert a PDF into audit-ready Markdown (and companion JSON/metadata) using OpenDataLoader PDF via the Node.js package @opendataloader/pdf. The source PDF remains authoritative; generated files are derivatives for search and AI analysis only.

When to use

• User provides a PDF file path and wants Markdown for AI analysis
• User mentions opendataloader-pdf, @opendataloader/pdf, PDF parsing, or document ingestion for RAG
• Legal, business, or operational PDFs that need traceable naming and logs
• Large PDFs (more than ~3 pages) or scanned/image PDFs needing OCR

When NOT to use

• Rename or classify only on small digital PDFs (typically 1–3 pages) — use documents/pdf-content-renamer with the pdf-context CLI (@singleton-sd/ai-plattform-tools-pdf-context) instead. It is faster, needs no Java, and does not create a /documents archive tree.
• Quick filename or date extraction without full Markdown, metadata versioning, or audit logs

Required inputs

Confirm these before running. Ask if missing.

Optional: --document-category (default legal), --confidentiality (default internal-confidential), --document-slug to override auto slug.

Prerequisites

• Node.js 20.19+
• Java 11+ (JVM) — required by OpenDataLoader PDF
• Install @singleton-sd/ai-plattform-tools-pdf-to-markdown (once per machine or project)

Install from GitLab npm registry (recommended)

The package is public — no GitLab token required. Point the @singleton-sd scope at the project registry (once per machine or project), in .npmrc:

@singleton-sd:registry=https://gitlab.com/api/v4/projects/singleton-sd%2Fai-plattform%2Ftools%2Fpdf-to-markdown/packages/npm/

Then install:

npm install @singleton-sd/ai-plattform-tools-pdf-to-markdown

One-off run (no install; .npmrc scope line still required):

npx @singleton-sd/ai-plattform-tools-pdf-to-markdown \
  --pdf "C:/path/to/document.pdf" \
  --workspace "C:/path/to/project" \
  --mode local \
  --operator "cursor-agent"

After install, the CLI binary is pdf-to-markdown (from node_modules/.bin).

Install from source (development)

git clone git@gitlab.com:singleton-sd/ai-plattform/tools/pdf-to-markdown.git
cd pdf-to-markdown
npm install

When working inside the ai-plattform workspace, the tool repo is at tools/pdf-to-markdown/.

For hybrid mode, install the Python hybrid server package and start it in a separate terminal:

pip install -U "opendataloader-pdf[hybrid]"
opendataloader-pdf-hybrid --port 5002

For scanned PDFs, add --force-ocr on the server. See reference.md for OCR and confidentiality rules.

Workflow

Task progress:
- [ ] Confirm pdf_path and workspace_root
- [ ] Choose mode (local vs hybrid)
- [ ] Install `@singleton-sd/ai-plattform-tools-pdf-to-markdown` if not present (see Prerequisites)
- [ ] Run convert_pdf.mjs
- [ ] Verify outputs exist (no overwrites)
- [ ] Report paths and review warnings to user

Step 1 — Resolve paths

• Resolve pdf_path to an absolute path; fail clearly if the file does not exist.
• Set workspace_root to the project folder that owns the /documents tree.

Step 2 — Run the conversion script

Always use the published CLI or bundled script (do not hand-roll commands unless the tool fails):

pdf-to-markdown \
  --pdf "C:/path/to/document.pdf" \
  --workspace "C:/path/to/project" \
  --mode local \
  --operator "cursor-agent"

If the package is not on PATH, use npx pdf-to-markdown or npx @singleton-sd/ai-plattform-tools-pdf-to-markdown.

When developing from source inside ai-plattform:

node tools/pdf-to-markdown/convert_pdf.mjs \
  --pdf "C:/path/to/document.pdf" \
  --workspace "C:/path/to/project" \
  --mode local \
  --operator "cursor-agent"

On Windows PowerShell, use quoted paths. The script:

1. Creates /documents subfolders (source, parsed/opendataloader-pdf, qa, logs, etc.)
2. Copies the PDF to /documents/source/{slug}__{hash}.pdf (immutable; never overwrites an existing copy)
3. Calls @opendataloader/pdf convert() and writes versioned outputs under /documents/parsed/opendataloader-pdf/
4. Writes metadata, extraction report, and processing log
5. Prints a JSON summary to stdout

Parse the JSON summary for markdown_path and related paths.

Step 3 — Verify

• Confirm markdown_path exists and is non-empty
• Confirm filenames include document_slug and source_hash
• Confirm review_status is not-reviewed
• Do not edit files under /parsed/opendataloader-pdf/ manually

Step 4 — Report to user

Use this completion template:

PDF converted to Markdown

Markdown: {markdown_path}
JSON: {json_path}
Source PDF (archived): {source_pdf}
Hash: {source_hash}
Version: {version}
Review status: not-reviewed

Legal reliance: The source PDF is authoritative. Do not rely on the Markdown for legal, financial, tax, employment, property, or compliance decisions without checking the source PDF.

Optional next steps (not run automatically):
- /normalized/ — cleaned Markdown for RAG
- /chunks/ — clause-aware JSONL chunks
- /reviewed/ — human-verified Markdown

Mode selection

Filename mode segment: odlpdf-local or odlpdf-hybrid (see reference.md).

Minimum output set

After a successful run:

/documents/source/{slug}__{hash}.pdf
/documents/parsed/opendataloader-pdf/{slug}__{hash}__odlpdf-{mode}__full__v{NNN}.md
/documents/parsed/opendataloader-pdf/{slug}__{hash}__odlpdf-{mode}__full__v{NNN}.json
/documents/qa/{slug}__{hash}__extraction-report__v{NNN}.json
/documents/logs/{slug}__{hash}__processing-log__v{NNN}.json
/documents/{slug}__{hash}__metadata__v{NNN}.json

Re-runs with the same source and mode increment the version (v002, v003, …). Files are never silently overwritten.

Troubleshooting

Additional resources

• Lightweight alternative for small PDFs: @singleton-sd/ai-plattform-tools-pdf-context (used by documents/pdf-content-renamer)
• Tool package: @singleton-sd/ai-plattform-tools-pdf-to-markdown
• Full legal/business naming, confidentiality, review, and chunking rules: reference.md
• Node.js quick start: opendataloader.org/docs/quick-start-nodejs
• Upstream tool docs: opendataloader-pdf on GitHub