# Reviewer > Use this academic citation-review skill when the user wants to analyze how later papers cite or use a target paper. It supports OpenAlex citation expansion from a DOI or OpenAlex work ID, regex or internal-agent/cloud-LLM classification of citing papers as yes/no/dont_know, optional open-PDF download and full-text extraction, review-prompt construction, and cautious literature-review drafting. Trigger for literature-review, citation-review, paper-usage review, related-work synthesis, OpenAlex citation analysis, and "which papers use this paper" tasks. Do not use for generic software code review. - Skill: `nmelanitis/reviewer` (Agent Skill, multi-file: 5 files) - Install (CLI): `npx skillmds@latest add nmelanitis/reviewer` - Raw SKILL.md: https://api.skillmd.com/api/skills/nmelanitis/reviewer/raw - Safety review: pending (external: skill-scanner PASS, skillspector PASS) - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: DevOps & Infra - Author: nmelanitis (https://skillmd.com/u/nmelanitis) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/nmelanitis/reviewer --- # Reviewer Reviewer is a bundled academic citation-review workflow. Given one target paper, it can fetch citing papers from OpenAlex, classify whether each citing paper appears to use the target work, build a review-writing prompt, and optionally write the review using either the agent running this skill or an external cloud LLM. Reviewer execution code upstream: https://github.com/nmelanitis/Reviewer/tree/main ## Before Running Start each new run by asking where the workflow should stop: ```text Where should Reviewer stop: fetch papers, classify papers, or write the review? ``` Default: write the review. Then ask: ```text Do you want to set Reviewer options interactively, or use defaults? ``` Always ask for the target paper DOI unless the user already provided it. If the user chooses defaults, use these defaults: - Full texts: yes, using open-access PDFs when available. - Classification method: LLM. - LLM execution: internal agent, not external cloud/API, unless the user explicitly chooses a provider. - Review length: left unspecified. - Final output: write the review, not only generate the prompt. - Papers included in the review: `yes` papers only. - References and bibliography: include paper references and a bibliography at the end, as in a scientific paper. - Stop point: write the review. If the user chooses interactive setup, ask for at least: - Target paper identifier: DOI, DOI URL, or OpenAlex work ID. - Where to stop: fetch papers, classify papers, or write the review. Default: write the review. - Whether to use full texts. Default: yes. - How to classify papers: regex or LLM. Default: LLM. - Whether LLM steps should use the internal agent or an external cloud/API provider. Default: internal agent. - Review length in pages, or leave unspecified. Default: unspecified. - Whether to include in-text paper references and an end bibliography. Default: yes. - Whether to write the review or only generate the review prompt. Default: write the review. - Which papers to include: `yes`, `yes and dont_know`, or `all`. Default: `yes`. Also clarify or infer these inputs as needed: - Target aliases: acronyms, method names, spelling variants, and common short names that citing papers may use. - Output folder name. Default: `Papername_out`, where `Papername` is the target paper title truncated to 20 characters and made filesystem-safe, for example `Masked_Autoencoders_out`. - Classification route: metadata regex, generated regex, internal-agent LLM, or external cloud/API LLM. - Evidence route: abstract-only or open-PDF full text. - Cloud provider/model, only when the user wants external cloud/API LLM steps. - Cost/privacy constraints before sending abstracts, full text, or prompts to a cloud provider. Use a small `--max-results` or `--limit` smoke test before large cloud/API runs or very large internal-agent classification passes. ## Resource Map - `src/`: executable Reviewer pipeline scripts. - `docs/README.md`: full command reference and examples. - `docs/ExecutionDiagram.md`: choose between regex, direct LLM, abstract-only, and full-text routes. - `docs/Classify-papers-as-relevant-README.md`: details for regex-context generation, generated regexes, and direct LLM classification. - `docs/Control-Review-size.README.md`: prompt and output size controls. - `docs/paper_usage_regex_prompt.md`: instruction prompt for generated-regex workflows. - `examples/`: development examples for the MAE paper; read only when an example run pattern is useful. ## Setup Install the local dependencies in the environment used to run the scripts: ```bash pip install pandas pypdf PyPDF2 PyMuPDF ``` For cloud LLM support, install only the providers needed: ```bash pip install openai anthropic google-genai ``` Provider keys can be exported in the shell or stored in a local `.env` copied from `.env.example`. Never commit real API keys. The internal-agent LLM path does not require provider SDKs or API keys. ## Preferred Workflow Prefer running individual scripts from the skill root with explicit input and output paths. This keeps generated artifacts out of the bundled `src/` code and works after the code was reorganized into `src/`. The preferred LLM execution path is the internal-agent path: the agent running this skill uses preserved prompt and evidence files to generate regexes, classify papers, and write the review without calling external LLM APIs. Use the external cloud/API path only when the user explicitly chooses it. Create one output directory for the run. Name it `Papername_out`, where `Papername` is the target paper title truncated to 20 characters and made filesystem-safe. In examples, `OUT_DIR` means that folder. All generated artifacts must stay inside this folder: CSV files, metadata JSON, prompt Markdown files, downloaded PDFs, extracted full text, generated regexes, raw LLM artifacts, and the final review. ```bash OUT_DIR="Papername_out" mkdir -p "$OUT_DIR" ``` Write `$OUT_DIR/run_config.json` before running the pipeline, and update it as resolved metadata or options become known. Include all parameters: target DOI or OpenAlex ID, resolved target title, output directory, stop point, aliases, full-text setting, classification method, LLM execution path, provider/model if external cloud/API is used, review length, references/bibliography setting, paper groups to include, prompt-size limits, PDF limits, smoke-test limits, and the commands or internal-agent steps used. Use this shape as the minimum config: ```json { "target": "", "resolved_title": "", "output_dir": "Papername_out", "stop_point": "write_review", "aliases": [], "use_full_text": true, "classification_method": "llm", "llm_execution": "internal_agent", "external_provider": null, "external_model": null, "review_length_pages": null, "include_references_bibliography": true, "include_papers": "yes", "limits": {}, "commands_or_steps": [] } ``` Fetch citing papers: ```bash python3 src/fetch_openalex_citations.py TARGET \ --output "$OUT_DIR/CitePaper.csv" \ --max-results 5 ``` If the paper title is only learned after fetching, rename the output directory to the resolved-title `Papername_out` form, then update `run_config.json`. If the selected stop point is `fetch papers`, stop after `CitePaper.csv`, `CitePaper.meta.json`, and `run_config.json` have been preserved in `OUT_DIR`. If using full text, download open PDFs and extract text: ```bash python3 src/download_open_pdfs.py \ --input "$OUT_DIR/CitePaper.csv" \ --out-dir "$OUT_DIR/open-pdfs" python3 src/extract_pdf_text.py \ --manifest "$OUT_DIR/open-pdfs/manifest.csv" \ --out-dir "$OUT_DIR/open-pdfs-text" ``` For the default internal-agent LLM classification path, read `$OUT_DIR/CitePaper.csv`, `$OUT_DIR/CitePaper.meta.json`, and extracted text from `$OUT_DIR/open-pdfs-text` when available. Classify each paper as `yes`, `no`, or `dont_know`, then write `$OUT_DIR/classified_papers.csv` with the columns expected by `src/build_review_prompt.py`. Preserve the classification evidence and reasoning in CSV audit fields where possible. For the regex path, classify with metadata-derived regexes and user aliases: ```bash python3 src/keywords.py \ --input-csv "$OUT_DIR/CitePaper.csv" \ --metadata "$OUT_DIR/CitePaper.meta.json" \ --output "$OUT_DIR/classified_papers.csv" \ --alias ALIAS ``` If the selected stop point is `classify papers`, stop after `classified_papers.csv`, supporting prompt/evidence files, and `run_config.json` have been preserved in `OUT_DIR`. Build an abstract-only review prompt: ```bash python3 src/build_review_prompt.py \ --classified "$OUT_DIR/classified_papers.csv" \ --metadata "$OUT_DIR/CitePaper.meta.json" \ --output "$OUT_DIR/review_prompt.md" ``` For full-text review prompts, include the extracted text directory: ```bash python3 src/build_review_prompt.py \ --classified "$OUT_DIR/classified_papers.csv" \ --metadata "$OUT_DIR/CitePaper.meta.json" \ --full-text-dir "$OUT_DIR/open-pdfs-text" \ --full-text-chars 8000 \ --output "$OUT_DIR/review_prompt_fulltext.md" ``` If the user requested prompt generation only, stop after preserving the prompt Markdown file. Otherwise, write `$OUT_DIR/review_draft.md` using the internal agent by default. Use the prompt file as the review-writing input and respect the requested review length when one was provided. When `include_references_bibliography` is true, write the review like a scientific paper: cite the target paper and discussed citing papers in the text, then add a `References` or `Bibliography` section at the end. Build bibliography entries from `CitePaper.csv` and `CitePaper.meta.json`; include title, year, venue/source when available, DOI, and OpenAlex URL. If author metadata is unavailable, use a title-first reference entry rather than inventing authors. Write a cloud/API LLM review draft only when the user has chosen a provider and model. If `include_references_bibliography` is true, preserve a prompt file in `OUT_DIR` that explicitly asks for in-text references and an end bibliography before calling the external provider: ```bash python3 src/llm_write_review.py \ --provider PROVIDER \ --model MODEL \ --mode selective \ --prompt "$OUT_DIR/review_prompt.md" \ --output "$OUT_DIR/review_draft.md" ``` ## Generated-Regex Route Use this when aliases and title-derived metadata are not precise enough, but the user wants an auditable regex classifier instead of per-paper LLM calls. ```bash python3 src/prepare_regex_context.py \ --input-csv "$OUT_DIR/CitePaper.csv" \ --metadata "$OUT_DIR/CitePaper.meta.json" \ --output "$OUT_DIR/regex_context.md" \ --alias ALIAS ``` For internal-agent regex generation, do not call `src/llm_generate_regex.py`. Instead, read `docs/paper_usage_regex_prompt.md` and `$OUT_DIR/regex_context.md`, then write `$OUT_DIR/generated_keywords_regexp.py` directly. For external cloud/API regex generation, use: ```bash python3 src/llm_generate_regex.py \ --provider PROVIDER \ --model MODEL \ --prompt docs/paper_usage_regex_prompt.md \ --context "$OUT_DIR/regex_context.md" \ --output "$OUT_DIR/generated_keywords_regexp.py" ``` Then classify with the generated regex file: ```bash python3 src/keywords.py \ --input-csv "$OUT_DIR/CitePaper.csv" \ --metadata "$OUT_DIR/CitePaper.meta.json" \ --mode generated \ --regexp "$OUT_DIR/generated_keywords_regexp.py" \ --output "$OUT_DIR/classified_papers.csv" ``` Treat generated regex files as trusted local Python. Inspect them before use when they came from any LLM, internal or external. Preserve the instruction prompt, `regex_context.md`, generated regex file, and classified CSV for auditability. ## Direct LLM Classification Use this when paper-use evidence is subtle. Prefer internal-agent LLM classification when the user has not explicitly chosen an external cloud/API provider. The classifier returns `yes`, `no`, or `dont_know` plus audit fields. For internal-agent classification, read the citation CSV, target metadata, and abstract/full-text evidence directly, then write the same `classified_papers.csv` shape used by the scripts. Preserve per-paper reasons, evidence quotes, and confidence where practical. For external cloud/API classification, use: ```bash python3 src/llm_classify_papers.py \ --input-csv "$OUT_DIR/CitePaper.csv" \ --metadata "$OUT_DIR/CitePaper.meta.json" \ --provider PROVIDER \ --model MODEL \ --evidence-mode abstract \ --output "$OUT_DIR/classified_papers.csv" \ --raw-dir "$OUT_DIR/llm-classification-raw" ``` For full-text classification, download and extract open PDFs before classifying: ```bash python3 src/download_open_pdfs.py \ --input "$OUT_DIR/CitePaper.csv" \ --out-dir "$OUT_DIR/open-pdfs" python3 src/extract_pdf_text.py \ --manifest "$OUT_DIR/open-pdfs/manifest.csv" \ --out-dir "$OUT_DIR/open-pdfs-text" python3 src/llm_classify_papers.py \ --input-csv "$OUT_DIR/CitePaper.csv" \ --metadata "$OUT_DIR/CitePaper.meta.json" \ --provider PROVIDER \ --model MODEL \ --evidence-mode fulltext \ --full-text-dir "$OUT_DIR/open-pdfs-text" \ --full-text-chars 30000 \ --output "$OUT_DIR/classified_papers.csv" ``` ## Full-Text Review Prompt Full text is optional for review writing. Downloaded PDFs are converted to text, then selected snippets are inserted into the Markdown prompt. ```bash python3 src/build_review_prompt.py \ --classified "$OUT_DIR/classified_papers.csv" \ --metadata "$OUT_DIR/CitePaper.meta.json" \ --full-text-dir "$OUT_DIR/open-pdfs-text" \ --full-text-chars 8000 \ --output "$OUT_DIR/review_prompt_fulltext.md" ``` Use `docs/Control-Review-size.README.md` when raising `--abstract-chars`, `--full-text-chars`, `--max-input-chars`, or `--max-output-tokens`. ## Orchestrator Use `src/run_pipeline.py` for quick one-command runs. It resolves bundled scripts from `src/` and the generated-regex prompt from `docs/`, while output artifacts are written relative to the current working directory unless explicit paths are passed. Always pass explicit paths into `OUT_DIR` when using the orchestrator or individual scripts. ```bash python3 src/run_pipeline.py TARGET \ --alias ALIAS \ --citation-csv "$OUT_DIR/CitePaper.csv" \ --classified-output "$OUT_DIR/classified_papers.csv" \ --review-output "$OUT_DIR/review_prompt.md" \ --pdf-dir "$OUT_DIR/open-pdfs" \ --fulltext-dir "$OUT_DIR/open-pdfs-text" \ --fulltext-review-output "$OUT_DIR/review_prompt_fulltext.md" \ --llm-regex-output "$OUT_DIR/generated_keywords_regexp.py" \ --llm-review-output "$OUT_DIR/review_draft.md" \ --max-results 5 ``` For complex or partially manual workflows, use the explicit script commands above. ## Review Standards - Prefer `dont_know` over a weak `yes`; false positives damage the review more than uncertain papers that can be manually checked. - Put every generated file for a run inside one `Papername_out` folder. Do not scatter CSV files, prompt files, PDFs, extracted text, generated regexes, or final reviews outside that folder. - Keep `$OUT_DIR/run_config.json` with all selected/defaulted parameters and update it when resolved metadata, stop point, or output paths change. - If `include_references_bibliography` is true, include in-text references and an end bibliography for the target paper and all papers discussed in the review. Do not fabricate missing bibliographic fields. - Preserve auditability and accountability: keep all generated prompt Markdown files, including `regex_context.md`, `review_prompt.md`, `review_prompt_fulltext.md`, chunk prompts, synthesis prompts, and any raw LLM prompts/responses produced during the run. - Distinguish abstracts-only evidence from full-text evidence in the final answer or draft. - Report counts for fetched papers, classified `yes`, `no`, and `dont_know`, downloaded PDFs, extracted texts, and papers included in the prompt. - Keep `classified_papers.csv`, generated regexes, and raw LLM classification responses when they were produced. - Do not claim the pipeline read paywalled or unavailable full texts. - Before large external cloud/API calls, estimate prompt size and confirm the user is comfortable with the cost and data exposure. ## Final Response When completing a task with this skill, summarize: - The route used: metadata regex, generated regex, direct LLM, abstract-only, or full-text, and whether LLM work was internal-agent or external cloud/API. - Stop point reached: fetch papers, classify papers, or write the review. - The `Papername_out` output folder path and `run_config.json` path. - Key output files written inside the output folder. - All preserved prompt files generated during the run. - Classification counts and any notable uncertainty. - Provider/model used for external cloud/API steps, if any. - Important limitations, especially missing abstracts, missing full text, or cases that need manual review. - A message suggesting the user check `run_config.json` to re-run with adjusted parameters, for example: "To re-run or tune this review, start from `$OUT_DIR/run_config.json` and change the parameters you want."