Find Paper References

Source: https://github.com/aipoch/medical-research-skills

find-paper-references — Academic Paper PubMed Reference Finder

Tools

Scripts are located in the scripts/ subdirectory of this skill:

• batch_search.py — Main entry point: accepts all queries at once, parallel search, batch esummary fetch, typical runtime 10-20s
• remap_refs.py — Converts [PMID:XXXXXXXX] markers to [1][2] numbering and generates formal references (not called by this skill — handled by subsequent endnote/zotero skill)
• convert_to_docx.py — Converts .md to .docx (not called by this skill)

Skill Boundary

┌─────────────────────────────────────────────────────┐
│  find-paper-references                              │
│  ─────────────────────────────────────────────────  │
│  Step 0-3: Identify knowledge points → Build PubMed search JSON │
│  Step 4:   batch_search.py batch search                         │
│  Step 5:   Select articles → Insert [PMID:xxxxxxxx] markers     │
│  Step 6:   Write to .md file                                    │
│  Step 7:   Generate _candidates.md candidate reference list     │
│  Step 8:   Prompt user to choose EndNote / Zotero to continue   │
│  ──────────────── Workflow ends ──────────────────────────────── │
│                                                     │
│  Formatting → format-references-endnote (EndNote)              │
│            → format-references-zotero (Zotero)                 │
└─────────────────────────────────────────────────────┘

Workflow (Execute in Order)

Step 0: Ask for NCBI API Key

Before starting any search, check and ask:

Do you have an NCBI API key? If so, search speed increases from 3 req/s to 10 req/s.
No worries if not — just skip and we'll proceed.

• User provides key: Before all script calls in this session, execute:

  # Windows
  $env:NCBI_API_KEY = "user_provided_key"
  # Mac/Linux
  export NCBI_API_KEY="user_provided_key"
  

  Then continue the workflow normally. Key is valid only for this session, not written to any file.

• User doesn't have one / skips: Proceed to Step 1. The script automatically uses the conservative 3 req/s rate.

Step 1: Determine Target File + Article Type

Read the full file content, then determine article type — can be auto-detected or confirmed with the user:

Auto-detection rules:

• Contains ## Materials and Methods / ## Methods section → Research Article
• No such section, body consists mainly of review-style paragraphs → Review
• When uncertain, ask the user directly

Article Type Citation Rules

Based on the detected type, this workflow must follow the corresponding rules:

Research Article Rules

Citation zones:  Introduction and Discussion only
            Materials/Methods and Results → skip entirely, insert no citations
Per knowledge point: max 2 articles (top 2 by relevance, no padding)
Total target: at least 30 unique PMIDs in the text

Review Rules

Citation zones:  All paragraphs except Conclusion
            Conclusion paragraphs → skip
Per knowledge point: max 5 articles
Total target: approximately 1 citation per 100 words, +/- 20%
            Target range = [word_count*0.8/100, word_count*1.2/100] (rounded)
            Example: 5000 words → target 40-60 articles; 8000 words → target 64-96 articles

Step 2: Identify Sentences Requiring Citations

Read through the paper, strictly following the citation zone rules for the current article type, skip sections that don't need citations.

Within allowed citation zones, identify all knowledge points requiring literature support:

Needs citation (applicable to allowed sections):

• Epidemiological data ("Lung cancer is one of the most common malignancies worldwide")
• Known biological functions of proteins/genes ("GPX4 reduces lipid peroxides to non-toxic alcohols")
• Signaling pathway descriptions ("xCT mediates cellular cystine uptake")
• Published research conclusions ("TRIM3 is downregulated in breast cancer")
• Known mechanisms of treatments/drugs ("Erastin induces ferroptosis by inhibiting xCT")

Never needs citation:

• This study's own experimental results
• Instrument/reagent descriptions in Methods (entire section skipped for Research Articles)
• Results section (entirely skipped for Research Articles)
• Conclusion section (skipped for Reviews)
• General technical operation descriptions

After identification, estimate total knowledge points:

• Research Article: target ≥ 30 unique citations → typically need to identify 20-35 knowledge points
• Review: calculate total word count first, estimate target citations at 1 per 100 words, typically need knowledge points close to target number (1-2 articles per point)

Organize identified knowledge points into a list, each containing:

• Section name (to confirm it's in an allowed citation zone)
• The sentence from the original text
• Core concept of the knowledge point (for generating search terms)

Step 3: Organize All Queries into Temporary JSON

Organize all knowledge points into the following format, write to %TEMP%\ref_queries.json (or /tmp/ref_queries.json):

[
  {"id": 1, "description": "global lung cancer incidence", "query": "global cancer statistics 2020 GLOBOCAN lung cancer Sung", "method": "pubmed"},
  {"id": 2, "description": "ferroptosis definition", "query": "Ferroptosis iron-dependent nonapoptotic cell death Dixon 2012", "method": "pubmed"},
  {"id": 3, "description": "TRIM3 structure and function", "query": "TRIM3 ubiquitin ligase RING domain substrate degradation", "method": "litsense"},
  ...
]

Method options:

• "pubmed" — Keyword-based precise search (recommended for most knowledge points)
• "litsense" — Semantic search (suitable for descriptive sentences, specific protein functions, etc.)
• "auto" — Try LitSense first; if results < 2 articles, automatically fall back to PubMed

Query writing guidelines:

• Use English, 3-8 keywords
• Prefer MeSH terms (ferroptosis, non-small cell lung cancer, ubiquitin ligase)
• For known classic papers, add author or year to query ("Dixon 2012", "Stockwell 2017")

Step 4: Run batch_search.py Once to Get All Results

python "SKILL_DIR/scripts/batch_search.py" "%TEMP%\ref_queries.json" --max 5

Script searches all queries in parallel, batch-fetches esummary, typical runtime 10-20 seconds (vs. minutes for sequential searching).

Output JSON format:

[
  {"id": 1, "description": "...", "results": [{pmid, title, authors, journal, year, doi}, ...]},
  ...
]

If any result is empty, create a new JSON file with adjusted keywords and re-run batch_search.py.

Step 5: Select Most Relevant Articles and Insert PMID Markers

For each sentence requiring citation:

1. Sort candidates by relevance, select according to article type limits:
   • Research Article: max 2 per knowledge point, select only the most relevant
   • Review: max 5 per knowledge point, can cover different research angles
2. Insert [PMID:XXXXXXXX] at end of original sentence; for multiple: [PMID:111][PMID:222]
3. If same PMID already cited, reuse it — do not insert duplicates

After insertion, count current unique PMIDs:

• Research Article: if fewer than 30, return to Step 2 to identify more knowledge points
• Review: calculate word count, compute target range [words0.8/100, words1.2/100] (rounded), if not within range, return to Step 2 to add or reduce

Example (before/after):

Before: Ferroptosis is a form of regulated cell death driven by iron-dependent lipid peroxidation.
After:  Ferroptosis is a form of regulated cell death driven by iron-dependent lipid peroxidation[PMID:25789077].

Flexible formats supported by remap_refs.py (all handled correctly):

• Case insensitive: [PMID:123] [pmid:123] [Pmid:123]
• Spaces around colon: [PMID: 123] [ PMID : 123 ]
• Multiple in one bracket: [PMID:123, PMID:456] → [1][2] (spaces and comma variants accepted)

Step 6: Write PMID-Marked Full Text to New File

Do not overwrite the original file. Write the full text with [PMID:XXXXXXXX] markers to a new file:

• Original paper.md → New file paper_refs.md (generated in same directory)

Naming rule: remove .md suffix from original filename, add _refs.md. Examples:

• TRIM3_ferroptosis.md → TRIM3_ferroptosis_refs.md
• No extension case: manuscript → manuscript_refs.md

Keep the original file unchanged — no modifications.

Step 7: Write Candidate References to Separate File

Do not append the reference candidate section to the original .md file — it will interfere with subsequent formatting.

Instead, generate a separate candidate file originalfilename_candidates.md in the same directory:

## Reference Candidates

> Below are PubMed search results for each knowledge point. Those selected for the text are marked with checkmark.
> Next step: Use format-references-endnote or format-references-zotero skill for formatting.

### Knowledge Point 1: [brief description]
- ✓ **[PMID:25789077]** Dixon SJ et al. "Ferroptosis: an iron-dependent form of nonapoptotic cell death." *Cell* 2012;149(5):1060-72.
- [PMID:26593993] Stockwell BR et al. "Ferroptosis: a regulated cell death nexus linking metabolism, redox biology, and disease." *Cell* 2017;171(2):273-285.
- [PMID:31634899] ...

### Knowledge Point 2: [brief description]
...

Step 7 completion ends this workflow. Do not run remap_refs.py or convert_to_docx.py.

Step 8: Proactively Remind User to Choose Formatting Tool

After Step 7, must proactively ask the user to choose next step:

All citation markers have been inserted (N unique references total). Ready to format and export to Word?

Do you use EndNote or Zotero for reference management?

• User chooses EndNote → immediately load format-references-endnote skill to continue
• User chooses Zotero → immediately load format-references-zotero skill to continue
• User says "not now" / "let me check first" → inform them "say 'format references' anytime to continue"

Do not end silently — this reminder is mandatory.

Next Step: Reference Formatting (Separate Skill)

After the [PMID:xxxxxxxx]-marked .md file is generated, choose based on user's reference manager:

EndNote Users

Use format-references-endnote skill. It will:

1. Convert [PMID:xxxx] markers to EndNote CWYW-recognized {Author, Year, Title} placeholders
2. Generate .ris file for automatic import to local EndNote library
3. Output .docx file — user clicks "Update Citations" once in Word to complete formatting

Zotero Users

Use format-references-zotero skill. It will:

1. Convert [PMID:xxxx] markers to native Zotero field codes
2. Search/download any CSL citation style
3. Output .docx file — click "Refresh" once in Word to complete

NCBI API Key (Optional, Free)

NCBI provides free API keys. With a key, rate limit increases from 3 req/s to 10 req/s, concurrent workers increase from 4 to 8, making searches faster and less likely to trigger rate limiting.

How to get one:

1. Register NCBI account: https://www.ncbi.nlm.nih.gov/account/
2. Go to Account Settings → API Key Management → Generate

How to use:

# Temporary (current session only)
set NCBI_API_KEY=your_key    # Windows
export NCBI_API_KEY=your_key  # Mac/Linux

# Permanent (recommended: add to system env vars or .env file)

Works fine without a key — script automatically uses conservative 3 req/s rate and auto-retries on 429 errors.

Important Notes

• Search terms must be in English (PubMed indexes primarily English literature)
• For very recent research (2024 onwards), PubMed indexing may have delays
• On 429 rate limiting, script auto-retries with backoff (up to 4 times) — no manual intervention needed

Input Validation

This skill accepts requests that match the documented purpose of find-paper-references and include enough context to complete the workflow safely.

Do not continue the workflow when the request is out of scope, missing a critical input, or would require unsupported assumptions. Instead respond:

find-paper-references only handles its documented workflow. Please provide the missing required inputs or switch to a more suitable skill.