Paper Audit (paper-audit)

Paper Audit (paper-audit)

Automated multi-mode paper auditing for LaTeX, Typst, and PDF documents.

Overview

The paper-audit skill provides comprehensive automated auditing for academic papers in multiple formats (.tex, .typ, .pdf). It orchestrates multiple specialized checkers and generates structured reports with severity-rated issues and quality scores.

Key Features

• Multi-format Support: LaTeX (.tex), Typst (.typ), and PDF (.pdf) inputs
• Three Audit Modes: self-check (comprehensive), review (peer-review focused), gate (submission readiness)
• PDF Visual Layout Check: Detects margin overflow, block overlaps, font inconsistency, low-resolution images, blank pages
• Reference Integrity Check: Undefined references, unreferenced labels, missing captions, forward references, numbering gaps
• ScholarEval Assessment: 8-dimension quality scoring (1–10 scale) with publication readiness label
• Online Bibliography Verification: CrossRef + Semantic Scholar API validation (no API key required)
• NeurIPS-aligned Scoring: Quality/Clarity/Significance/Originality dimensions (1–6 scale)

Audit Modes

Using the Skill in Claude Code

Simply mention relevant trigger words in your conversation:

Run a full audit on my paper before submission

Check paper quality with ScholarEval assessment

Audit my PDF for layout issues

Argument Conventions

• File path (required): .tex, .typ, or .pdf
• Mode (optional): self-check (default), review, gate
• Flags (optional): --online for bibliography verification, --scholar-eval for ScholarEval

Script CLI

# Basic audit (auto-detects mode)
python scripts/audit.py paper.tex --mode self-check

# With online bibliography verification
python scripts/audit.py paper.tex --mode review --online

# With ScholarEval 8-dimension assessment
python scripts/audit.py paper.tex --mode self-check --scholar-eval

# PDF audit (visual + extracted text checks)
python scripts/audit.py paper.pdf --mode gate

# Full audit with all options
python scripts/audit.py paper.tex --mode self-check --online --scholar-eval --email you@example.com

Check Modules

Reference Integrity Module

Checks figure and table references for completeness.

Checks Performed

Script CLI

# LaTeX files
python scripts/check_references.py main.tex

# Typst files
python scripts/check_references.py main.typ

# JSON output
python scripts/check_references.py main.tex --json

PDF Visual Layout Module

Analyzes camera-ready PDF for typesetting issues.

Checks Performed

Script CLI

python scripts/visual_check.py paper.pdf
python scripts/visual_check.py paper.pdf --margin 72 --min-dpi 150
python scripts/visual_check.py paper.pdf --json

ScholarEval Assessment Module

8-dimension academic quality scoring based on ScholarEval (arXiv:2510.16234).

Dimensions

Publication Readiness Labels

Script CLI

# From audit JSON output
python scripts/scholar_eval.py --audit-json audit_result.json

# Merge with LLM evaluation
python scripts/scholar_eval.py --audit-json result.json --llm-json llm_scores.json

# JSON output
python scripts/scholar_eval.py --audit-json result.json --json

LLM Assessment Prompt

After running the script-based audit, provide this prompt to Claude with the full paper text:

Please read the paper and provide 1-10 scores for these dimensions in JSON format:

{
  "novelty": {
    "score": <1-10>,
    "evidence": "<specific innovations and differences from prior work>"
  },
  "significance": {
    "score": <1-10>,
    "evidence": "<potential impact on the field>"
  },
  "reproducibility_llm": {
    "score": <1-10>,
    "evidence": "<completeness of experimental description, code/data availability>"
  },
  "ethics": {
    "score": <1-10>,
    "evidence": "<ethical considerations, conflicts of interest, data privacy>"
  }
}

Online Bibliography Verification

Verify citation metadata against CrossRef and Semantic Scholar databases.

Features

• DOI Verification: Validates DOI existence and cross-checks year/journal fields
• Title Search: Finds papers by title when no DOI is available
• DOI Suggestions: Recommends DOIs for entries missing them
• Rate Limiting: Built-in polite rate limiting (0.5s between requests)
• No API Key Required: Uses public CrossRef and Semantic Scholar APIs

Usage

Add --online flag to the audit command:

# With online verification
python scripts/audit.py paper.tex --mode self-check --online

# With polite email for faster CrossRef rate limits
python scripts/audit.py paper.tex --online --email you@example.com

Or run standalone:

# Verify bibliography file
python scripts/verify_bib.py references.bib --online --email you@example.com

Output Protocol

All issues use the standard protocol:

% REFERENCES (Line 45) [Severity: Critical] [Priority: P0]: Undefined reference: \ref{fig:arch}
% VISUAL (Page 3) [Severity: Major] [Priority: P1]: Content overflows left margin (x=28.3pt, margin=72pt)
% BIB (Line 12) [Severity: Major] [Priority: P1]: Online mismatch for 'smith2020': year: bib=2019 vs api=2020

Report Format

The audit generates a structured Markdown report with:

• Issue list by severity (Critical → Major → Minor)
• NeurIPS-aligned score (Quality/Clarity/Significance/Originality, 1-6 scale)
• ScholarEval 8-dimension table with readiness label (when --scholar-eval)
• Online verification summary (when --online)

Recommended Workflows

Pre-Submission Check

python scripts/audit.py paper.tex --mode gate --online --email you@example.com

Comprehensive Self-Review

python scripts/audit.py paper.tex --mode self-check --scholar-eval

PDF Camera-Ready Check

python scripts/audit.py paper.pdf --mode gate

Full Quality Assessment

python scripts/audit.py paper.tex --mode self-check --online --scholar-eval --email you@example.com

Reference Files

• resources/references/SCHOLAR_EVAL_GUIDE.md: Detailed ScholarEval scoring rubrics