Io Research Design

Research Design (io-research-design)

IO accepts quantitative, formal, and qualitative IR work but is demanding about each. The design must credibly connect the IR theory (io-theory-building) to evidence at or across the international level, and rule out the strongest rival international explanation. This skill is mode-aware: pick the section that matches your work.

When to trigger

• Specifying identification, case selection, or experimental design for an IR question
• A reviewer questioned causal claims, the level of analysis, generalization, or a confound
• Preparing a pre-analysis plan for a foreign-policy experiment
• Justifying why the design adjudicates the rival IR account from io-literature-positioning

Quantitative / causal inference (often dyadic, TSCS, or network)

• Get the level of analysis right. State the unit (state-year, dyad-year, IGO, directed dyad, sub-state, transnational) and why it matches the theory. Many IR mistakes are unit/level mistakes.
• Identification first. State the estimand and the assumptions that license a causal reading (ignorability, parallel trends, exclusion, continuity); defend them, don't assert them.
• IR-specific inference traps: non-independence in dyadic data (cluster on both members / multiway, or use AME/network models); spatial and temporal dependence in TSCS; selection into alliances/treaties/conflict; reverse causation (does the institution cause the behavior or vice versa).
• Designs: natural experiments and DID/event study (modern staggered estimators, not naive TWFE), IV with credible exclusion at the international level, RDD where thresholds exist, matching/weighting with balance + sensitivity, gravity/PPML for trade flows.
• Sensitivity: how strong must an unobserved confounder be to overturn the result?

Qualitative / case-based (international cases)

• Case selection justified by design logic (typical, deviant, most/least-likely, paired comparison across dyads or institutions) — say what the case is a case of in IR terms.
• Process tracing with explicit tests (hoop, smoking-gun, straw-in-the-wind); state what evidence would have disconfirmed the international mechanism.
• Source transparency: archives, diplomatic records, elite interviews — plan documentation and citation now (QDR; see io-transparency-and-data-policy).

Experiments (survey / conjoint / lab on foreign-policy attitudes)

• Preregister design and primary analyses; report power/MDE; pre-specify subgroups.
• Be explicit about external validity: from public-opinion or elite-survey results to actual state behavior is a real inferential leap — caveat it.

Formal-empirical linkage (a core IO design)

• Make the empirical test follow from the model's comparative statics, not a loose analogy.
• Distinguish predictions unique to your model from those shared with rival IR theories.
• Plan the proof appendix the IO editorial staff will verify before final acceptance.

The adjudication test (IO-specific)

For the single strongest rival international explanation, write one sentence: "If the rival were true rather than my argument, the international data/cases would look like ___; instead they look like ___." If you cannot, the design does not yet identify the IR contribution.

Worked design vignette (illustrative): a treaty-ratification design

Claim: ratifying an international monitoring treaty raises later compliance. The naive cross-section confounds the effect with selection into membership — states that mean to comply ratify. An IO-credible design exploits variation in ratification timing: among eventual ratifiers, treat the staggered timing as identification with a modern staggered DID/event-study estimator (not naive TWFE) and report pre-trends. The adjudication sentence: if selection drove compliance, the gain would appear before ratification; instead it appears only after and tracks monitoring intensity. A sensitivity check then asks how strong an unobserved confounder must be to overturn it (say, twice the observed regime-type effect — illustrative). Timing-based identification, a rival-ruling counterfactual, and a sensitivity bound convert an association into an IR causal claim at IO.

Identification-threat table (IR-specific confounds and the design answer)

Execution bridge (StatsPAI / Stata MCP)

Estimate and audit the design, don't only describe it. Full map: execution-with-mcp. International Organization is IR — country/dyad panels with difficult identification; foreground the source of variation and robustness to alternative explanations.

• detect_design → recommend → fit with as_handle=true → audit_result.
• Observational causal claims: staggered DiD (callaway_santanna / sun_abraham + bacon_decomposition + honest_did_from_result); IV (effective_f_test + anderson_rubin_ci); RDD (rdrobust + mccrary_test).
• Experiments: randomization-based inference, romano_wolf for many-outcome family-wise control, and mediate for mediation (not naive controlling-away).
• Sensitivity: oster_delta / sensemakr for observational claims.

Report the effect size in interpretable units; route the full battery to the appendix/supplement. A run end-to-end (synthetic data, real returns) is in the JF execution walkthrough.

Anti-patterns

• Ignoring dyadic non-independence; clustering at the wrong level; naive TWFE on staggered treatment
• A domestic-level design used to support an international-level claim (level mismatch)
• "Causal" language on a design that only supports association across states
• Convenience case selection dressed up as theory-driven
• Over-generalizing survey-experiment attitudes to real state behavior with no caveat
• Treating treaty membership as exogenous when ratification is plainly a choice

Output format

【Mode】quant-causal / qualitative / experiment / formal-empirical
【Level of analysis】unit + why it matches the IR theory
【Estimand or claim】what is being identified/shown
【Key assumption(s)】and how each is defended (incl. dyadic/TSCS dependence)
【Rival ruled out】the adjudication sentence
【Robustness/sensitivity / proof appendix】planned
【Next】io-data-analysis

Supplementary resources

• ../../resources/external_tools.md — dyadic/network/gravity packages and CAQDAS for qualitative IR
• ../../resources/official-source-map.md — formal-proof and quantitative-result verification policy

Source: brycewang-stanford/Awesome-Journal-Skills → International-Organization-Skills/skills/io-research-design/SKILL.md