Academic Manuscript Reviewer
This skill provides expert guidance for reviewing academic manuscripts with methodological rigor, focusing on causal identification, research design, and constructive critique.
When to Use This Skill
Use this skill when:
- Asked to review an academic paper or manuscript
- Evaluating a research design or empirical strategy
- Providing feedback on causal identification approaches
- Critiquing experimental designs (field, survey, or lab experiments)
- Assessing quasi-experimental methods (DiD, RDD, IV, etc.)
Review Process
Follow these steps to conduct a thorough review:
1. Initial Reading and Understanding
- Read the paper carefully to understand the research question, argument, methods, and findings
- Identify the core empirical strategy and identification approach
- Consider the theoretical contribution and stakes
2. Consult Reference Materials
Before writing the review, consult these reference documents as needed:
- Reviewer Profile: Detailed guidance on methodological approach, common critiques, and reviewing philosophy
- Review Structure: Recommended structure for organizing your review
- Checklists: Manuscript checklist and reviewer self-checklist to ensure thorough evaluation
- Review Examples: Example reviews demonstrating the style and approach
3. Evaluate Core Elements
Identification Strategy (Primary Focus):
- Assess the credibility of causal identification
- Evaluate whether identifying assumptions are justified
- Check for plausible alternative explanations
Research Design:
- For DiD: check parallel trends, treatment timing endogeneity, estimator choice
- For RDD: check covariate balance, bandwidth selection, interpretation
- For IV: check exclusion restriction, instrument strength
- For experiments: check external validity, statistical power, realistic scenarios
Statistical Inference:
- Check subgroup analysis (are interactions formally tested?)
- Assess multiple comparison issues
- Evaluate clustering of standard errors
- Identify "bad controls" or post-treatment bias
Theory-Empirics Link:
- Can the design distinguish the proposed mechanism from alternatives?
- Is there a clear theoretical payoff (especially for top journals)?
4. Structure Your Review
Follow this format:
- Metadata: Journal, dates, recommendation
- Summary: Concise paragraph (~100 words) of paper's core elements
- Overall Assessment: High-level verdict with strengths before reservations
- Major Issues: 2-4 fundamental flaws (numbered list)
- Minor Issues: Less critical suggestions (numbered list)
- Recommendation: Clear decision justified by major issues
5. Maintain Appropriate Tone
- Be constructive but firm on methodological issues
- Praise genuine strengths and author efforts
- Provide actionable suggestions when possible
- Avoid ad hominem criticism
- Distinguish between fatal flaws and suggestions for improvement
Key Principles
Primacy of Identification: The quality of causal identification strategy is paramount. Clever questions or novel data cannot compensate for flawed identification.
Methodological Fluency: Cite relevant methodological literature (Goodman-Bacon on DiD, Cattaneo et al. on RDD, Hainmueller et al. on conjoints, etc.).
Skepticism of Survey Experiments: Be particularly skeptical of survey experiments on sensitive topics (corruption, clientelism) due to social desirability bias and external validity concerns.
Subgroup Analysis: Consistently check whether apparent differences across subgroups are formally tested with interaction terms.
Statistical Power: Assess whether studies (especially field experiments) are adequately powered to detect plausible effect sizes.
Self-Check Before Submitting
Use the reviewer self-checklist (in references/checklists.md) to ensure:
- Requests for controls are justified by confounding arguments
- Literature suggestions explain why citations are essential
- Comments about framing relate to scientific validity
- No unnecessary or ad hominem criticism
- Positive reviews adequately explain contributions
1---2name: academic-reviewer3description: Expert guidance for reviewing academic manuscripts submitted to journals, particularly in political science, economics, and quantitative social sciences. Use when asked to review, critique, or provide feedback on academic papers, research designs, or empirical strategies. Emphasizes methodological rigor, causal identification strategies, and constructive feedback on research design.4metadata:5 source: https://skillmd.com/skills/diegosouzapw/academic-reviewer6---78# Academic Manuscript Reviewer910This skill provides expert guidance for reviewing academic manuscripts with methodological rigor, focusing on causal identification, research design, and constructive critique.1112## When to Use This Skill1314Use this skill when:15- Asked to review an academic paper or manuscript16- Evaluating a research design or empirical strategy17- Providing feedback on causal identification approaches18- Critiquing experimental designs (field, survey, or lab experiments)19- Assessing quasi-experimental methods (DiD, RDD, IV, etc.)2021## Review Process2223Follow these steps to conduct a thorough review:2425### 1. Initial Reading and Understanding2627- Read the paper carefully to understand the research question, argument, methods, and findings28- Identify the core empirical strategy and identification approach29- Consider the theoretical contribution and stakes3031### 2. Consult Reference Materials3233Before writing the review, consult these reference documents as needed:3435- **[Reviewer Profile](references/reviewer-profile.md)**: Detailed guidance on methodological approach, common critiques, and reviewing philosophy36- **[Review Structure](references/review-structure.md)**: Recommended structure for organizing your review37- **[Checklists](references/checklists.md)**: Manuscript checklist and reviewer self-checklist to ensure thorough evaluation38- **[Review Examples](references/review-examples.md)**: Example reviews demonstrating the style and approach3940### 3. Evaluate Core Elements4142**Identification Strategy (Primary Focus)**:43- Assess the credibility of causal identification44- Evaluate whether identifying assumptions are justified45- Check for plausible alternative explanations4647**Research Design**:48- For DiD: check parallel trends, treatment timing endogeneity, estimator choice49- For RDD: check covariate balance, bandwidth selection, interpretation50- For IV: check exclusion restriction, instrument strength51- For experiments: check external validity, statistical power, realistic scenarios5253**Statistical Inference**:54- Check subgroup analysis (are interactions formally tested?)55- Assess multiple comparison issues56- Evaluate clustering of standard errors57- Identify "bad controls" or post-treatment bias5859**Theory-Empirics Link**:60- Can the design distinguish the proposed mechanism from alternatives?61- Is there a clear theoretical payoff (especially for top journals)?6263### 4. Structure Your Review6465Follow this format:66671. **Metadata**: Journal, dates, recommendation682. **Summary**: Concise paragraph (~100 words) of paper's core elements693. **Overall Assessment**: High-level verdict with strengths before reservations704. **Major Issues**: 2-4 fundamental flaws (numbered list)715. **Minor Issues**: Less critical suggestions (numbered list)726. **Recommendation**: Clear decision justified by major issues7374### 5. Maintain Appropriate Tone7576- Be constructive but firm on methodological issues77- Praise genuine strengths and author efforts78- Provide actionable suggestions when possible79- Avoid ad hominem criticism80- Distinguish between fatal flaws and suggestions for improvement8182## Key Principles8384**Primacy of Identification**: The quality of causal identification strategy is paramount. Clever questions or novel data cannot compensate for flawed identification.8586**Methodological Fluency**: Cite relevant methodological literature (Goodman-Bacon on DiD, Cattaneo et al. on RDD, Hainmueller et al. on conjoints, etc.).8788**Skepticism of Survey Experiments**: Be particularly skeptical of survey experiments on sensitive topics (corruption, clientelism) due to social desirability bias and external validity concerns.8990**Subgroup Analysis**: Consistently check whether apparent differences across subgroups are formally tested with interaction terms.9192**Statistical Power**: Assess whether studies (especially field experiments) are adequately powered to detect plausible effect sizes.9394## Self-Check Before Submitting9596Use the reviewer self-checklist (in references/checklists.md) to ensure:97- Requests for controls are justified by confounding arguments98- Literature suggestions explain why citations are essential99- Comments about framing relate to scientific validity100- No unnecessary or ad hominem criticism101- Positive reviews adequately explain contributions