# Paper Extract

> Hybrid PDF extract: prefer Marker `_raw/` for full text; take math from arXiv source, then Mathpix, then page image plus vision. Use when normalizing `_raw/` into `extracted/`, fixing equations or algorithms, or when the user mentions Marker, Docling, Mathpix, OCR, or arXiv source.

- Skill: `l4pislazuli/paper-extract` (Agent Skill)
- Install (CLI): `npx skillmds@latest add l4pislazuli/paper-extract`
- Raw SKILL.md: https://api.skillmd.com/api/skills/l4pislazuli/paper-extract/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Docs & Writing
- Author: L4pisLazuli (https://skillmd.com/u/l4pislazuli)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/l4pislazuli/paper-extract

---


# Paper extract (hybrid)

Do not treat direct PDF reading as the main path. Normalize machine output in `_raw/` into `extracted/`.

## Layout

```
sources/<source-id>/
├── original/          # PDF original (placed by human)
├── _raw/              # Machine extract (not the citation source of truth)
│   ├── marker.md      # Marker full-text draft (single full-text extractor)
│   ├── arxiv/         # arXiv LaTeX / e-print source (authoritative math; preferred)
│   ├── mathpix/       # high-accuracy math pages (fallback for math)
│   └── pages/         # page images pXX.png (optional, for re-extract)
└── extracted/         # human-readable source of truth (cite here)
```

## Path priority (full text)

1. Markdown exists under `_raw/` (Marker) → read it and generate/update `extracted/`
2. `_raw/` missing → ask the user to run Marker (see tooling). **Do not silently fill `extracted/` by reading the whole PDF**
3. Exceptions: short checks (e.g., Abstract) or filling pages missing from `_raw` may use the PDF / page images

## Math / equation reference priority

Equations and algorithms must come from an authoritative source, not from full-text OCR. Use this fixed order:

1. **arXiv LaTeX / e-print source** (`_raw/arxiv/`) — preferred (compiled TeX; no OCR guesswork)
2. **Mathpix** (or equivalent high-accuracy math OCR) on the PDF math pages (`_raw/mathpix/`)
3. **PDF page image + vision re-extract** of only the affected pages (`_raw/pages/`)

### Fallback on fetch failure (must be explicit)

- If the arXiv source cannot be obtained (paper not on arXiv, source withheld / PDF-only, or the fetch fails), fall to the **next path in the list above** (Mathpix → page image + vision).
- **Do not** substitute an undefined or arbitrary source (blog, mirror, a different paper, or model memory) for the missing equations.
- If no path resolves an equation, keep it and mark `[extraction uncertain]`, and state which source was used. When math correctness is load-bearing (survey / both), ask the user before proceeding.

## Tooling (Windows-friendly)

| Use case | Tool | Output |
|---|---|---|
| Full-text draft (all modes) | **Marker** (single full-text extractor) | `_raw/marker.md` |
| Math / algorithms (survey) | arXiv source → Mathpix → page image + vision (in that order) | `_raw/arxiv/` → `_raw/mathpix/` → `_raw/pages/` |
| Scanned PDF only | OCR (Llama vision OCR, or Tesseract) | `_raw/ocr.md` |
| Naive OCR on text-layer PDFs | **Not the main path** (math breaks easily) | — |

### Marker: known weaknesses and how to complement them

Marker is the single full-text extractor (Docling is no longer used):

- **Equations / inline math** can be imperfect → take math from the arXiv source (or Mathpix); do not trust Marker's math for `algorithm.md`
- **Complex / nested tables** can be misparsed → keep the caption only, verify against the original PDF page, and note "See original for table"
- **Multi-column layout, footnotes, references** can be reordered → sanity-check reading order against the PDF when a section looks scrambled
- **Figures** are captured as captions only (by design) → note "See original for figure"
- Mark any doubtful span `[extraction uncertain]` and re-extract only those pages via the math reference priority

### OCR (Llama) configuration

- Keep the OCR (Llama) configuration — API key, endpoint / model name, and cost-bearing options — **under the user's own management** (a local, gitignored env / config file that the user owns)
- Reason: it carries credentials and billable settings, so it must never be committed or auto-edited by the agent
- The agent may ask the user to run Llama OCR, but must not generate, hardcode, or commit these settings

By mode:

- **study**: full-text `_raw` from Marker is enough to ingest
- **survey / both**: use the math reference priority (arXiv first) for method / math / algorithm pages; re-extract page-by-page if still broken

## Normalize `_raw` → `extracted/`

- Do not change meaning. No translation, no “helpful” paraphrase that alters sense
- Split by chapter/section into `extracted/NN-<section-slug>.md` (~300–500 lines)
- Figures/tables: captions only; "See original for figure"
- Symbol corruption (e.g., `π`→`⇡`, `ρ`→`⇢`), broken equation line breaks, or likely omissions: mark `[extraction uncertain]` and, if possible, re-extract only those pages via the math reference priority
- Never cite `_raw/` (wikilinks must target `extracted/`)

### Block IDs (Obsidian)

- One ID names one block. Latin letters, digits, hyphens only. Unique within the file. Same-page blocks: `^p12-a`, `^p12-b`
- Simple paragraph: space + `^id` at the end of the last line

```markdown
Body text of the paragraph. ^p12-a
```

- Structured blocks (blockquote, list, callout, table): ID on its own line, with a blank line before and after

```markdown
> Quoted body from the paper.

^p12

Next paragraph.
```

- Do not invent a range ID to cover several blocks. If later citations need each paragraph, give each its own ID

## Roles

| Location | Role |
|---|---|
| `_raw/` | Machine output; ingest input |
| `extracted/` | Reading / citation source of truth |
| `survey/algorithm.md` | Cleaned math / pseudocode for implementation (cross-check the arXiv source or high-accuracy `_raw`) |

## Forbidden

- Skipping `_raw/` and filling `extracted/` by reading the whole PDF
- Treating broken extract as “correct math” without checking
- Using naive OCR (e.g., Tesseract) as the main path for text-layer papers
- On arXiv-source fetch failure, silently pulling equations from an undefined / arbitrary source instead of the defined fallback
- Committing or auto-editing OCR (Llama) credentials / config

## Done when

- `extracted/` chunks exist; meaning was not changed
- Each `extracted/` file has unique block IDs; placement matches the paragraph vs structured-block rules above
- Uncertain math is marked `[extraction uncertain]` rather than silently “fixed”
- Citations, if any were added in this pass, resolve to existing block IDs