Search the literature the way a librarian would: several databases, explicit search strings, deduplication, and a record of what was searched so the process can be reported. Every result must carry a real identifier (DOI, arXiv id, or OpenAlex id) that resolves. A paper you cannot resolve does not go on the list.
Sources and how to query them
All are free and need no API key. Use curl -s with a User-Agent header and a mailto where the API asks for one (OpenAlex and CrossRef give a faster "polite pool" with it). Read reference/apis.md for the exact endpoints, parameters, and the fields to keep.
| Source |
Best for |
Endpoint |
| OpenAlex |
broad coverage, citation counts, concepts, open-access links |
api.openalex.org/works?search=... |
| Semantic Scholar |
CS and AI, citation graph, TL;DR, influential citations |
api.semanticscholar.org/graph/v1/paper/search?query=... |
| arXiv |
preprints, latest ML/systems work |
export.arxiv.org/api/query?search_query=... |
| CrossRef |
resolving a DOI to full metadata, verifying a reference |
api.crossref.org/works/<doi> |
Respect rate limits: at most one request per second per source without a key; back off on HTTP 429.
Procedure
- Build the search strings. From the research question, extract two to four concept groups (the phenomenon, the method, the setting, the population). For each group list synonyms and abbreviations. Combine as
(a OR a') AND (b OR b'). Record every string; the thesis method chapter will report them.
- Query each source with each string, most specific first. Keep the top 50 per query per source at most. Filter by year only when the student asks (
--since), and say so in the log.
- Normalise and deduplicate. Key on lowercase DOI; fall back to arXiv id, then to normalised title (lowercase, ASCII, no punctuation) plus first author surname. Merge fields across sources: prefer CrossRef for bibliographic fields, OpenAlex for citation count and OA link, Semantic Scholar for abstract and TL;DR.
- Snowball once. For the three to five most relevant hits, fetch their references and their citing works from Semantic Scholar (
/paper/<id>/references, /paper/<id>/citations) and add anything that appears twice or more.
- Rank. Score = relevance (title and abstract term overlap with the concept groups) with a bonus for citation count normalised by age, and a small bonus for recency. Show the formula used. Never rank purely by citations: the newest relevant work has few.
- Screen titles and abstracts against the inclusion criteria the student states (or propose criteria and ask). Mark each as include, exclude with reason, or unsure.
- Verify every included identifier resolves:
curl -sI https://doi.org/<doi> returns 3xx, or the arXiv abs page returns 200. Drop anything that does not.
Output
Write two files (default under research/), and summarise in the reply:
search-log.md: date, sources, every search string, result counts per source, inclusion criteria, and the snowballing seeds. This is a method-chapter artifact.
reading-list.md: one entry per included paper, ranked:
## 1. Title (Year)
Authors. Venue. DOI: 10.xxxx/yyyy | arXiv: 2401.01234 | cited by 142 | OA: url
Why it is here: one sentence tying it to the research question.
Abstract: (verbatim, from the source)
Offer the next steps: bibtex-manager to turn the list into a .bib, paper-reader for structured notes on each, citation-graph to expand from the key papers.
Rules
- Never fabricate a paper, author, venue, or identifier. If an API returns nothing, say so and widen the search string.
- Never paraphrase an abstract as if it were the paper's finding; the reading happens in
paper-reader.
- Report negative results: "no paper matched string 3 in any source" is useful for the gap argument.
1---2name: literature-search3description: "Find the literature for a research question by querying OpenAlex, Semantic Scholar, arXiv, and CrossRef from the shell with curl (no keys, no MCP), deduplicating by DOI, ranking by relevance, citations, and recency, and producing a reading list with verified identifiers ready for paper-reader and bibtex-manager. Use when starting a literature review, checking whether a gap is real, or looking for the papers that cite or are cited by a known key paper."4argument-hint: "(optional) the research question or search terms; add --since YEAR or --limit N"5metadata:6 source: https://skillmd.com/skills/khaledsaeed18/literature-search7---89Search the literature the way a librarian would: several databases, explicit search strings, deduplication, and a record of what was searched so the process can be reported. Every result must carry a real identifier (DOI, arXiv id, or OpenAlex id) that resolves. A paper you cannot resolve does not go on the list.1011## Sources and how to query them1213All are free and need no API key. Use `curl -s` with a `User-Agent` header and a `mailto` where the API asks for one (OpenAlex and CrossRef give a faster "polite pool" with it). Read [reference/apis.md](reference/apis.md) for the exact endpoints, parameters, and the fields to keep.1415| Source | Best for | Endpoint |16| --- | --- | --- |17| OpenAlex | broad coverage, citation counts, concepts, open-access links | `api.openalex.org/works?search=...` |18| Semantic Scholar | CS and AI, citation graph, TL;DR, influential citations | `api.semanticscholar.org/graph/v1/paper/search?query=...` |19| arXiv | preprints, latest ML/systems work | `export.arxiv.org/api/query?search_query=...` |20| CrossRef | resolving a DOI to full metadata, verifying a reference | `api.crossref.org/works/<doi>` |2122Respect rate limits: at most one request per second per source without a key; back off on HTTP 429.2324## Procedure25261. **Build the search strings.** From the research question, extract two to four concept groups (the phenomenon, the method, the setting, the population). For each group list synonyms and abbreviations. Combine as `(a OR a') AND (b OR b')`. Record every string; the thesis method chapter will report them.272. **Query each source** with each string, most specific first. Keep the top 50 per query per source at most. Filter by year only when the student asks (`--since`), and say so in the log.283. **Normalise and deduplicate.** Key on lowercase DOI; fall back to arXiv id, then to normalised title (lowercase, ASCII, no punctuation) plus first author surname. Merge fields across sources: prefer CrossRef for bibliographic fields, OpenAlex for citation count and OA link, Semantic Scholar for abstract and TL;DR.294. **Snowball once.** For the three to five most relevant hits, fetch their references and their citing works from Semantic Scholar (`/paper/<id>/references`, `/paper/<id>/citations`) and add anything that appears twice or more.305. **Rank.** Score = relevance (title and abstract term overlap with the concept groups) with a bonus for citation count normalised by age, and a small bonus for recency. Show the formula used. Never rank purely by citations: the newest relevant work has few.316. **Screen titles and abstracts** against the inclusion criteria the student states (or propose criteria and ask). Mark each as include, exclude with reason, or unsure.327. **Verify** every included identifier resolves: `curl -sI https://doi.org/<doi>` returns 3xx, or the arXiv abs page returns 200. Drop anything that does not.3334## Output3536Write two files (default under `research/`), and summarise in the reply:3738- `search-log.md`: date, sources, every search string, result counts per source, inclusion criteria, and the snowballing seeds. This is a method-chapter artifact.39- `reading-list.md`: one entry per included paper, ranked:4041 ```42 ## 1. Title (Year)43 Authors. Venue. DOI: 10.xxxx/yyyy | arXiv: 2401.01234 | cited by 142 | OA: url44 Why it is here: one sentence tying it to the research question.45 Abstract: (verbatim, from the source)46 ```4748Offer the next steps: `bibtex-manager` to turn the list into a `.bib`, `paper-reader` for structured notes on each, `citation-graph` to expand from the key papers.4950## Rules5152- Never fabricate a paper, author, venue, or identifier. If an API returns nothing, say so and widen the search string.53- Never paraphrase an abstract as if it were the paper's finding; the reading happens in `paper-reader`.54- Report negative results: "no paper matched string 3 in any source" is useful for the gap argument.