# Check Paper Citations > Verify every citation in a paper is real, accurate, and correctly used. Downloads PDFs, reads abstracts, checks claims against source text, flags AI-hallucinated references, identifies uncited claims, and maintains structured annotations in the .bib file. Use when the user says "check citations", "verify references", "are these citations real", or when iter-refine-writing finds unannotated bib entries. Also use before submission as a final citation gate. - Skill: `eunomia-bpf/check-paper-citations` (Agent Skill, multi-file: 3 files) - Install (CLI): `npx skillmds@latest add eunomia-bpf/check-paper-citations` - Raw SKILL.md: https://api.skillmd.com/api/skills/eunomia-bpf/check-paper-citations/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Research & Search - Author: eunomia-bpf (https://skillmd.com/u/eunomia-bpf) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/eunomia-bpf/check-paper-citations --- # Citation Verification Systematically verify every citation in a paper. This skill reads cited papers, confirms they exist, checks that claims match sources, and maintains a structured annotation system in the .bib file. Do not perform any Git operation. Return citation and report changes to the caller. ## First Step Read the paper's `.tex` file and `.bib` file. Identify the `docs/reference/` directory (or create it) for storing downloaded PDFs. **Mechanical pre-check (mandatory).** Before any manual verification, run `scripts/verify_bib.py` (relative to this skill's directory) on the `.bib` file: ```bash python3 /path/to/check-paper-citations/scripts/verify_bib.py references.bib ``` This script checks every bib entry through two paths: 1. **API verification**: queries arXiv, Crossref, DBLP, and Semantic Scholar to compare title, author count, year, and venue against external metadata. DBLP is preferred over arXiv metadata for published computer-science conference and journal entries. 2. **URL verification** (fallback): if no API returns a match, the script checks an explicit `url` or the `https://doi.org/...` resolver and verifies that the resource is reachable. Reachability establishes existence only, not metadata correctness; the manual passes below still verify the citation's claims and fields. **Every entry must pass at least one path.** The script exits non-zero if: - Any `VERIFIED` entry has a metadata mismatch (ERROR) - Any entry has neither API match nor reachable URL (ERROR — add a `url` or `doi` field) - Any entry has a broken/unreachable URL (ERROR) The script distinguishes two severity levels. Errors make it exit non-zero; warnings are reported without blocking the run: - **ERROR** (must fix, blocks exit): metadata mismatch, missing/broken URL, missing required fields, hallucinated entries, `"and others"` with <3 named authors - **WARN** (should review): suspicious author names, workshop/conference venue mismatch, arXiv preprint that may have a published version, possible AI-generated title patterns If a title API is rate-limited or unavailable and the entry has no stable URL/DOI fallback, the script reports an **inconclusive ERROR** rather than claiming that the citation is hallucinated. Retry later or add a stable identifier; do not downgrade a real citation based only on transient API failure. Fix all errors before proceeding to manual passes. **Never mark an entry as `% VERIFIED: ` without this script passing for that entry.** **All citations must live in a standalone `.bib` file** (e.g., `references.bib`), never as inline `\bibitem{}` in the `.tex` file. If the paper uses `\bibitem{}`, migrate all entries to a `.bib` file first and switch to `\bibliography{references}`. Then run the three passes below in order. Do not skip passes. ## Pass 1: Existence and Authenticity For every entry in the `.bib` file: 1. **Check the bib annotation block.** Look for a comment block above each entry with these fields: ```bibtex % VERIFIED: % REAL: yes|no|unverified % PDF: % ABSTRACT: <1-2 sentence summary of what the paper actually says> % USED_FOR: ``` If this block is missing or incomplete, the entry needs full verification. 2. **Verify the paper exists and metadata is correct.** For each unverified entry: - Search for the paper by title on the web (arxiv, ACM DL, IEEE, USENIX, Google Scholar) - Confirm ALL metadata fields: title, authors (full list, correct spelling), year, venue/booktitle, pages, DOI - If any metadata is wrong, fix it in the .bib entry - If the paper does not exist at all, flag as **HALLUCINATED** (must-fix) - If found, download the PDF to `docs/reference/` (naming is not this skill's concern; keep whatever name is natural) - **Do NOT remove unused bib entries.** Mark them with `% STATUS: unused` but keep them. Only flag hallucinated entries for removal. The author may intend to use them later. 3. **Read the abstract and introduction** of each downloaded PDF. Write a 1-2 sentence summary of what the paper actually says (its main contribution, not what we want it to say). 4. **Update the .bib annotation block** with all fields filled in. ## Pass 2: Claim-Citation Alignment For every `\cite{}` in the `.tex` file: 1. **Extract the surrounding claim.** Read the sentence containing the citation and 1-2 surrounding sentences for context. 2. **Check: does the cited paper support this claim?** - Read the cited paper's abstract/intro (from PDF or annotation) - Does the paper actually say what we claim it says? - Common problems: - Mischaracterization: "Paper X shows Y" but Paper X actually shows Z - Over-attribution: claiming the paper says more than it does - Context mismatch: using a finding from one context to support a claim in a different context 3. **If a claim is inaccurate:** - Report as Must-fix with: the claim text, what the paper actually says, and a suggested rewrite - Edit the `.tex` to fix the claim (with Chinese translation if the paper uses bilingual format) - Update the `USED_FOR` field in the .bib annotation ## Pass 2.5: Academic Integrity Checks For every citation, check for these additional risks: 1. **Preprint vs published version.** If a bib entry cites an arXiv preprint, check whether a published (conference/journal) version now exists. If yes, update the bib entry to cite the published version with correct venue, pages, and DOI. 2. **Secondary citations.** If the paper attributes a claim to source A, but source A itself attributes it to source B, the paper should cite B directly (or cite both). Check key claims against their cited sources. 3. **String citations in claims.** `\cite{a,b,c}` chains are fine in related work surveys. Only flag when a specific factual claim in the paper's own argument rests on a chain without explaining which source supports it. 4. **Self-citation and double-blind.** Self-citation is encouraged — it shows research continuity. But under double-blind review: - Self-citations must use third person ("X et al." not "our prior work") - Flag if the .tex text says "our" or "we" when referring to a cited paper that shares authors - Do NOT flag self-citations as a problem — flag only identity leaks 5. **Retracted papers.** For each academic paper citation, check if it has been retracted (search for retraction notices). 6. **Venue accuracy.** Confirm that the venue/booktitle in the bib matches where the paper was actually published. Common errors: citing a workshop paper as a main conference paper, or vice versa. 7. **Ghost citations.** If a bib entry is cited (`\cite{X}` appears) but the surrounding text never explains WHY it is cited, flag as Should-fix. Every citation should earn its place. ## Pass 3: Missing Citations Scan the paper for citation gaps. This pass both checks existing text AND actively searches for missing citations. ### 3a. Factual claims without support Statements like "X% of Y", "research shows that Z", or "agents invoke dozens of calls per minute" without a citation. Every factual claim that is not the paper's own measurement or common knowledge needs a cite. ### 3b. System, technique, dataset, and benchmark mentions Every named system (Docker, gVisor, Firecracker, etc.) and technique (dual-LLM, lethal trifecta, least privilege) should cite the original source at first mention. Datasets, benchmark suites, and traces are citations too: every dataset, benchmark, workload suite, or trace source used in the evaluation must cite its origin (paper, official page, or artifact repository) at first mention. An evaluation that names a benchmark or public trace without a cite is a Must-fix. ### 3c. Citation density by section Check citation density per section. Flag sections where the ratio of claims to citations is low: - **Introduction** — should have a cite every 2-3 sentences on average - **Background/Motivation** — should be the MOST densely cited section. Nearly every factual sentence needs a cite. Flag if any paragraph has fewer than 1 cite per 2 sentences. - **Design** — lower density OK (it's the paper's own contribution), but borrowed concepts still need cites - **Related Work** — every paragraph should cite multiple papers ### 3d. Actively search and add missing citations For each gap found, do not just flag — **actively search** for the right paper to cite: - Use WebSearch / WebFetch to find papers - Download the PDF to `docs/reference/` - Add the bib entry with full annotation block (VERIFIED, REAL, PDF, ABSTRACT, USED_FOR) - Edit the .tex to add the `\cite{}` at the right location (with Chinese translation if the paper uses bilingual format) Common citations that background sections often need: - Classic principles: least privilege (Saltzer & Schroeder 1975), capability-based security (Dennis & Van Horn 1966) - Agent frameworks: Claude Code, OpenHands, SWE-Agent, Devin — cite the relevant ones - Measurement claims: if you say "dozens of calls per minute", cite AgentCgroup or your own measurement - Kernel mechanisms: cite original papers or documentation for Landlock, seccomp, namespaces, cgroups **Note:** For a comprehensive related-work coverage audit (are there important papers in the field that the paper doesn't discuss at all?), use `research-literature-novelty` separately. This pass focuses on citation gaps in the existing text. ## Bib Annotation Format Every bib entry must have this comment block immediately above it: ```bibtex % VERIFIED: 2026-07-07 % REAL: yes % PDF: docs/reference/agentcgroup.pdf % ABSTRACT: Characterizes OS-level resource dynamics of AI coding agents % across 144 tasks. Finds container-level resource controls are % mismatched with tool-call-level dynamics (granularity, responsiveness, % adaptability mismatches). % USED_FOR: Motivation for per-tool-call granularity (resource dimension) @misc{agentcgroup, title={...}, ... } ``` Fields: - `VERIFIED`: Date of last verification (YYYY-MM-DD) - `REAL`: `yes` (confirmed real), `no` (hallucinated, must remove), `unverified` - `PDF`: Path to downloaded PDF, or "not available" if not downloadable - `ABSTRACT`: Factual summary of what the paper says (2-3 lines max) - `USED_FOR`: How our paper uses this citation (1 line) ## No Standalone Verification Ledger The bib annotation blocks are the single source of truth for verification state. Do NOT create or maintain `docs/citation-verification.md` or any other standalone citation ledger, database, or report file — it duplicates the annotations and drifts. When a quick overview is needed, generate it on the fly from the bib file (e.g. `grep -c '% REAL: yes' references.bib`, or list entries missing annotation blocks) and report it inline. ## Output After all three passes, report: - Number of citations verified - Number of hallucinated citations (must-fix) - Number of inaccurate claims fixed - Number of missing citations added - Any entries that could not be verified (PDF not available, etc.) ## When Called from iter-refine-writing When triggered as a gate check (not a full run): - Only check whether all .bib entries have complete annotation blocks - If any annotation is missing or has `REAL: unverified`, trigger a full run of this skill - If all annotations are present and `REAL: yes`, just check for missing citations in the .tex (Pass 3 only)