Ajps Replication And Verification

Replication & Verification (ajps-replication-and-verification)

This is AJPS's signature differentiator. Unlike journals that check reproducibility in-house (or not at all), AJPS sends your deposited materials to an independent third party that actually re-runs the code and confirms it reproduces the numerical results reported in the main text — before publication. If it does not reproduce, the article does not publish until it does. Build the package as you analyze (ajps-data-analysis); it cannot be improvised at acceptance.

When to trigger

• Building the replication / verification package
• A manuscript is accepted and you have received upload instructions (after final draft, during the technical check, before copyediting)
• Data cannot be fully shared (privacy, ethics, provider restriction) and you need a path
• A qualitative or multi-method paper whose evidence needs a documented verification route

How AJPS verification actually works (official baseline checked 2026-06-20)

1. Deposit to the AJPS Dataverse. Materials go into a Dataset within the AJPS Dataverse on the Harvard Dataverse Network — not a personal site or generic cloud link. Other copies may exist elsewhere as long as everything needed is in the AJPS Dataverse Dataset.
2. Independent third-party verification. A verifier re-runs the code and confirms it reproduces the numerical results in the main text. The published article carries a statement: "The Cornell Center for Social Sciences verified that the data and replication code submitted to the AJPS Dataverse replicates the numerical results reported in the main text of this article." MPSA also identifies Cornell Center for Social Sciences as the contracted replication-and-verification provider.
3. Qualitative path. Qualitative / multi-method materials follow the AJPS verification guidelines and qualitative checklist: document collection procedures, instruments, analytic operations, source fragments or access restrictions, and any exemption route needed for human-subjects, copyright, or legal constraints.
4. Timing. It happens after the final draft is submitted, during the technical check, before copyediting — late enough that fixing an unscripted analysis under deadline is painful.

Build-as-you-go checklist (so the re-run matches)

• readme.txt lists and describes every file (group as "Data files", "Stata .do files", "Files to Reproduce Table 1", ...)
• One master script runs the modular scripts in order and sets the working directory once
• set seed for every stochastic step (Monte Carlo, bootstrap, randomization inference, jitter)
• Explicit software-version statements (e.g., "R 4.3.2", "Stata/MP 18.0"); packages pinned
• Every main-text number, table, and figure is regenerated by the code, names matched
• Restricted data: explain why, give exact access instructions, provide synthetic data where feasible
• Sharing terms and any access restrictions documented in the Dataverse materials
• Preregistration / pre-analysis plan linked (anonymized) where applicable

When data cannot be shared

• Explain why (ethics, privacy, legal/provider restriction).
• Provide README instructions on exactly how others can obtain the data (process, application, contact).
• Where possible, provide synthetic data resembling the real data so the code can run end to end.

Anti-patterns

• Treating the deposit as a post-publication afterthought (it gates publication)
• Depositing code that does not actually regenerate the printed main-text numbers
• A personal URL instead of the AJPS Dataverse
• Unseeded, unpinned, hard-coded-path code that "works on my machine" but not on the verifier's
• Claiming data are restricted with no access path or synthetic substitute

Output format

【Repository】AJPS Dataverse (Harvard) — Dataset staged? [Y/N]
【Reproduces main-text numbers?】master script re-run locally, matches? [Y/N]
【Files】readme.txt + master script + seeds + pinned versions? [Y/N]
【Verifier】Cornell Center for Social Sciences / AJPS qualitative checklist path
【Restricted data?】why + access path + synthetic data?
【Sharing terms】Dataverse/exemption notes complete?
【Next】ajps-review-process

Supplementary resources

• ../../resources/external_tools.md — replication-package tooling and qualitative-transparency options
• ../../resources/official-source-map.md — AJPS Replication & Verification Policy, verifier transition, and AJPS Dataverse

Source: brycewang-stanford/Awesome-Journal-Skills → American-Journal-of-Political-Science-Skills/skills/ajps-replication-and-verification/SKILL.md