ReviewSearch
Search a corpus of 205,988 peer reviews (with author rebuttals) from ICLR / ICML / NeurIPS / COLM, served by the ReviewSearch hybrid-search API.
When to use
Reach for this when the user wants real peer-review evidence: what reviewers criticize about a kind of paper, examples of a specific concern, or how authors rebutted a critique. Not for searching paper content — this searches reviews.
How to search
Run the bundled script (Python 3, no dependencies). Paths here are relative to this skill's own directory, the one this SKILL.md was loaded from, so they resolve in any host that discovers skills:
python "<this skill dir>/scripts/search_reviews.py" "<query>" [options]
Options: --top-k N (default 10), --accepted-only, --year-min YYYY, --year-max YYYY (corpus spans 2023–2026).
Example:
python "<this skill dir>/scripts/search_reviews.py" \
"reviews asking for a statistical significance test" --top-k 5 --accepted-only
Writing good queries
- Queries are English and work best as concern-style natural language, e.g. "missing ablation on learning rate", "reviewers doubting reproducibility", "papers criticized for weak baselines".
- The primary search target is each review's concern text (weaknesses + questions).
Reading results
The script prints JSON: { count, results: [...] }. Each result has:
venue,year,title,decision(Accept/Reject variants)summary— the reviewer's summary of the paperconcern— the reviewer's weaknesses + questions (what to focus on)rebuttal— the author's response (every result has one)score— relevance
When you present results to the user, cite the venue/year and note the source (see attribution below).
Notes
- The API is a free demo and may cold-start (take a minute) after being idle — if a call times out, wait ~1 minute and retry once.
- Point at a different backend with the
RS_API_URLenv var if needed.
Attribution
Data: peer reviews from OpenReview (ICLR / ICML / NeurIPS / COLM), licensed CC-BY-4.0.