# Academic PDF Downloader

> Download PDFs from academic publishers (Springer, BMC, Nature, etc.) that have cookie-wall anti-bot protection. Use this skill whenever the user asks to download a paper, save a PDF from a journal, get an article from a DOI/PMID, or mentions academic publishers like Springer, Elsevier, BMC, Nature, Wiley, etc. Also triggers when curl/wget/requests fail on academic sites due to redirect challenges or proxy issues. Bundles a Node.js script that handles cookie-jar redirect chains and bypasses system proxy automatically. The user may say things like "download this paper", "get the PDF", "save this article to my desktop", or just paste a DOI.

- Skill: `aliner-hello/academic-pdf-downloader` (Agent Skill)
- Install (CLI): `npx skillmds@latest add aliner-hello/academic-pdf-downloader`
- Raw SKILL.md: https://api.skillmd.com/api/skills/aliner-hello/academic-pdf-downloader/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Docs & Writing
- Author: Aliner-hello (https://skillmd.com/u/aliner-hello)
- Updated: 2026-09-22
- Page: https://skillmd.com/skills/aliner-hello/academic-pdf-downloader

---


# Academic PDF Downloader

Download PDFs from academic publishers that employ anti-bot / cookie-wall
protections (Springer, BMC, Nature, and others).

## Why this exists

Academic publishers, especially Springer/BMC, use an IDP (identity provider)
redirect chain to verify browsers support cookies:

```
/article/doi → 303 redirect → idp.springer.com/authorize?response_type=cookie
→ redirect back with idp_session cookie → article page
```

Standard tools fail for two reasons:

1. **curl / Python requests** — go through system proxy (127.0.0.1:7897 or
   similar), which may be blocked or unreliable for academic sites.
2. **Node.js `fetch()`** — bypasses the proxy but does NOT persist cookies
   across redirects, so the IDP cookie check fails with
   `error=cookies_not_supported`.

The bundled script solves both: Node.js native `http/https` module (no proxy)
+ manual cookie jar across redirects.

## How to use

### 1. Run the bundled script directly

```bash
node "<skill-dir>/scripts/download.js" <doi|url|pmid> [output-path]
```

Examples:
```bash
# By DOI
node ".../download.js" "10.1186/s13075-026-03828-4" "/path/to/paper.pdf"

# By URL
node ".../download.js" "https://link.springer.com/article/10.1186/s13075-026-03828-4"

# By PMID (auto-resolves via PubMed)
node ".../download.js" "42143348"

# Custom output dir via env
PDF_OUTPUT_DIR="$HOME/Desktop" node ".../download.js" "10.1186/..."
```

### 2. If the script fails for a new publisher

First check `<skill-dir>/scripts/download.js` — the publisher resolution
logic is in the `PUBLISHERS` array. Each publisher has:

- `test(url)` — returns true if this handler applies
- `getPdfUrl(articleUrl, cookies, ua)` — returns the PDF download URL

For a new publisher, add a handler. The existing `springer` handler and
`generic` fallback cover most cases. If the generic fallback returns a
non-PDF response, inspect the article page HTML to find the real download
URL pattern.

### 3. What the script does

```
1. Resolve input (DOI → doi.org, PMID → pubmed, URL → as-is)
2. Visit article page with cookie jar (handles IDP redirects)
3. Find PDF download URL from page HTML
4. Download PDF with auth cookies + Referer
5. Save to disk, report size & page count
```

## Common failure modes

| Symptom | Cause | Fix |
|---------|-------|-----|
| 404 on PDF URL | Wrong URL pattern for this publisher | Add publisher handler in `PUBLISHERS` |
| `error=cookies_not_supported` in URL | Script not following redirects with cookies | Ensure using bundled script, not fetch/curl |
| Empty file / HTML saved | PDF behind additional auth step | Check `res.headers.content-type`, add cookies from extra redirect |
| Timeout | Large PDF, slow connection | Increase timeout in script (default 120s for PDF) |

## Adding a new publisher

Open `<skill-dir>/scripts/download.js` and add an entry to the `PUBLISHERS` array:

```js
{
  name: 'elsevier',
  test: (url) => /sciencedirect\.com|elsevier\.com/.test(url.hostname),
  async getPdfUrl(articleUrl, cookies, ua) {
    const res = await fetchWithCookies(articleUrl, {
      headers: { 'User-Agent': ua, Accept: 'text/html', Cookie: cookies },
    });
    const html = res.body.toString('utf-8');
    // Find PDF link in page — adjust selector per publisher
    const match = html.match(/href="([^"]+\.pdf[^"]*)"/i);
    return match ? new URL(match[1], articleUrl).href : null;
  },
},
```