Researchstack Paper Reproduction

Paper Reproduction

Read ../researchstack/references/workflow.md, ../researchstack/references/venues.md, and ../researchstack/references/review-rubric.md.

Use the bundled templates:

• assets/reproduction-brief-template.md
• assets/ambiguity-log-template.md
• assets/reconstruction-matrix-template.md
• assets/reproduction-verdict-template.md

Treat reproduction as a structured audit, not a vague attempt to "see if it runs."

Core outputs:

1. Reproduction brief.
2. Claim extraction.
3. Method reconstruction.
4. Experiment reconstruction matrix.
5. Ambiguity log.
6. Reproduction verdict.

Work in this order:

1. Extract the paper's top claims and intended contribution.
2. Identify the exact experiments, datasets, workloads, metrics, and baselines that support those claims.
3. Reconstruct the minimal environment and dependency assumptions needed to rerun the evidence.
4. List missing or ambiguous details before pretending the paper is reproducible.
5. Decide which experiments are mandatory for a faithful reproduction and which are nice-to-have.
6. Distinguish four outcomes clearly:
   • reproduced
   • partially reproduced
   • blocked by ambiguity or missing assets
   • contradicted by rerun evidence

Be explicit about failure sources:

• paper underspecification,
• code-release mismatch,
• environment mismatch,
• hidden tuning or preprocessing,
• unstable results,
• invalid assumptions by the reproducer.

Do not equate "result mismatch" with "paper is wrong" unless the evidence is strong enough to support that conclusion.

When a repository is available, connect this skill to:

• researchstack-experiment-design for the reproduced experiment matrix,
• researchstack-experiment-ops for run tracking,
• researchstack-artifact-audit for provenance and credibility,
• researchstack-peer-review for reviewer-style reproducibility critique.