Data Analysis (jpart-data-analysis)
JPART reviewers are methodologically sophisticated public-management scholars, and the journal requires
authors to release the data and software code underlying the paper as a condition of publication (see
jpart-transparency-and-data). Analyze as if a referee will re-run the code — because the materials are
public. This skill covers execution and reporting; design lives in jpart-research-design.
When to trigger
- Running main and supporting analyses; building the results section
- A reviewer asked for robustness, heterogeneity, or alternative specifications
- Reconciling preregistered vs. exploratory analyses
- Making the analysis reproducible before the mandatory data/code deposit
Analysis norms JPART expects
- Report uncertainty and magnitude. Confidence/credible intervals and the substantive size of the
effect (e.g., a fraction of an SD of PSM), not stars alone.
- Robustness that probes, not decorates. Show specifications that could break the result
(alternative measures of red tape/PSM, samples, estimators, fixed effects), and say what you learned.
- Confront the PA-specific threats. Common-method/common-source bias, social desirability, and
self-selection into public service are the objections raised first — address them, don't ignore them.
- Heterogeneity with discipline. Pre-specify subgroups where possible; correct for multiple
comparisons; do not mine for a significant interaction and theorize it post hoc.
- Right inference. Cluster at the assignment/agency level; randomization inference for experiments;
small-cluster corrections (wild-cluster bootstrap) when agencies are few.
- Preregistration discipline. Separate confirmatory from exploratory analyses; reconcile any
deviation from the plan and justify it.
Measurement (a perennial JPART referee focus)
- Validate constructs (PSM, red tape, goal ambiguity); report reliability; show the result is not an
artifact of a single scale or coding choice. Concept defined in
jpart-theory-building must match the
measure used here.
Reproducibility while you work (not at the end)
- One master script regenerates every table and figure from raw/constructed data.
- Set and report seeds for bootstrap, randomization inference, simulation, any stochastic step.
- Pin software/package versions (
renv.lock, requirements.txt, recorded ssc/net installs).
- Keep table/figure numbers matched to script outputs — the materials are public and will be checked.
What JPART reviewers probe, by design
| Design |
The check a JPART referee runs first |
The fix that earns benefit of the doubt |
| Survey of public employees |
Are X and Y from the same self-report (common-method)? |
separate sources / objective Y / marker variable + Harman caution |
| Survey/field experiment |
Is it pre-registered, powered, on the right population? |
preregistered estimand, MDE reported, public-employee sample |
| Observational causal |
Is "effect" really selection into public service? |
state estimand + assumption; sensitivity to an unobserved confounder |
| Multilevel |
Is the agency-level nesting modeled? |
random effects / clustered SEs, ICC reported |
| Mixed methods |
Do quant and qual actually corroborate? |
show agreement and own divergence |
Worked micro-example (illustrative numbers)
A hypothetical JPART field experiment tests whether a goal-clarity intervention raises frontline
performance among real caseworkers. The pre-registered ITT is +0.18 SD (95% CI 0.06 to 0.30),
randomization-inference p = 0.006. An exploratory split by tenure shows +0.41 SD for new hires,
but it was not pre-registered and the interaction p = 0.03 before correction; after a Bonferroni
adjustment across five exploratory subgroups it crosses 0.20. The disciplined write-up reports the
confirmatory +0.18 SD effect with its interval and substantive meaning, flags the +0.41 figure as
exploratory and not multiplicity-robust, and frames it as a hypothesis for future work. (All
numbers illustrative.)
Referee-pushback patterns and the JPART repair
- "This is common-method bias, not an effect." → Use a separate/objective outcome or a marker
variable; report the sensitivity, don't wave it away with a single Harman test.
- "The robustness table only reruns near-identical specs." → Replace decorative checks with specs that
could break the result (alternative PSM/red-tape measures, samples), and say what held.
- "This is selection into public service." → State the estimand and assumption; report how strong an
unobserved confounder must be to overturn it.
- "I cannot tell confirmatory from exploratory." → Segregate them explicitly; the deposited code is
public, so the split must survive a re-run.
Calibration anchors (hedged)
- The bar is a public-management theory payoff carried by credible numbers — an estimate with no
mechanism rarely clears JPART review.
- JPART increasingly rewards experimental and causal designs, but a rigorous multilevel or mixed
study is judged on its own terms.
- The data-and-code release is mandatory (where ethically possible) — write the analysis so the
public package reproduces every printed number. Confirm exact wording on the live policy page.
Execution bridge (StatsPAI / Stata MCP)
Run the battery, don't just enumerate it. Full map:
execution-with-mcp. JPART is public management — observational and experimental designs on public organizations; identification + clustered/multilevel inference.
- Many outcomes / specifications:
romano_wolf (step-down FWER) or benjamini_hochberg.
- OVB sensitivity:
oster_delta / sensemakr.
- Inference:
wild_cluster_bootstrap (few clusters), twoway_cluster / conley.
- Re-fit off one handle:
audit_result(result_id) lists missing checks + the exact
suggest_function for each.
- Exhibits:
etable / did_summary_to_latex from the handle — no retyped numbers.
Decisive checks in the body, exhaustive battery in the appendix.
JF execution walkthrough.
Output format
【Main estimate】magnitude + interval + substantive meaning
【PA threat handled】common-method / selection — how?
【Robustness】specs that could break it → what held
【Heterogeneity】pre-specified? MHT-adjusted?
【Confirmatory vs exploratory】clearly separated?
【Reproducible】master script + seeds + pinned versions? [Y/N]
【Next】jpart-tables-figures
Supplementary resources
1---2name: jpart-data-analysis3description: Use when executing and reporting the analysis for a Journal of Public Administration Research and Theory (JPART) manuscript so it survives expert, double-blind review and the journal's mandatory data-and-code release. Covers honest uncertainty, robustness, and the PA-specific traps (common-method bias, selection). Guides analysis norms; it does not fabricate results.4metadata:5 source: https://skillmd.com/skills/brycewang-stanford/jpart-data-analysis6---78# Data Analysis (jpart-data-analysis)910JPART reviewers are methodologically sophisticated public-management scholars, and the journal **requires11authors to release the data and software code** underlying the paper as a condition of publication (see12`jpart-transparency-and-data`). Analyze as if a referee will re-run the code — because the materials are13public. This skill covers execution and reporting; design lives in `jpart-research-design`.1415## When to trigger1617- Running main and supporting analyses; building the results section18- A reviewer asked for robustness, heterogeneity, or alternative specifications19- Reconciling preregistered vs. exploratory analyses20- Making the analysis reproducible before the mandatory data/code deposit2122## Analysis norms JPART expects23241. **Report uncertainty and magnitude.** Confidence/credible intervals and the *substantive* size of the25 effect (e.g., a fraction of an SD of PSM), not stars alone.262. **Robustness that probes, not decorates.** Show specifications that could *break* the result27 (alternative measures of red tape/PSM, samples, estimators, fixed effects), and say what you learned.283. **Confront the PA-specific threats.** Common-method/common-source bias, social desirability, and29 self-selection into public service are the objections raised first — address them, don't ignore them.304. **Heterogeneity with discipline.** Pre-specify subgroups where possible; correct for multiple31 comparisons; do not mine for a significant interaction and theorize it post hoc.325. **Right inference.** Cluster at the assignment/agency level; randomization inference for experiments;33 small-cluster corrections (wild-cluster bootstrap) when agencies are few.346. **Preregistration discipline.** Separate **confirmatory** from **exploratory** analyses; reconcile any35 deviation from the plan and justify it.3637## Measurement (a perennial JPART referee focus)38- Validate constructs (PSM, red tape, goal ambiguity); report reliability; show the result is not an39 artifact of a single scale or coding choice. Concept defined in `jpart-theory-building` must match the40 measure used here.4142## Reproducibility while you work (not at the end)43- One **master script** regenerates every table and figure from raw/constructed data.44- **Set and report seeds** for bootstrap, randomization inference, simulation, any stochastic step.45- Pin software/package versions (`renv.lock`, `requirements.txt`, recorded `ssc`/`net` installs).46- Keep table/figure numbers matched to script outputs — the materials are public and will be checked.4748## What JPART reviewers probe, by design4950| Design | The check a JPART referee runs first | The fix that earns benefit of the doubt |51|--------|--------------------------------------|------------------------------------------|52| Survey of public employees | Are X and Y from the same self-report (common-method)? | separate sources / objective Y / marker variable + Harman caution |53| Survey/field experiment | Is it pre-registered, powered, on the right population? | preregistered estimand, MDE reported, public-employee sample |54| Observational causal | Is "effect" really selection into public service? | state estimand + assumption; sensitivity to an unobserved confounder |55| Multilevel | Is the agency-level nesting modeled? | random effects / clustered SEs, ICC reported |56| Mixed methods | Do quant and qual actually corroborate? | show agreement and own divergence |5758## Worked micro-example (illustrative numbers)5960A hypothetical JPART field experiment tests whether a goal-clarity intervention raises frontline61performance among **real caseworkers**. The pre-registered ITT is **+0.18 SD (95% CI 0.06 to 0.30)**,62randomization-inference *p* = 0.006. An exploratory split by tenure shows **+0.41 SD** for new hires,63but it was *not* pre-registered and the interaction *p* = 0.03 before correction; after a Bonferroni64adjustment across five exploratory subgroups it crosses 0.20. The disciplined write-up reports the65confirmatory +0.18 SD effect with its interval and substantive meaning, flags the +0.41 figure as66**exploratory and not multiplicity-robust**, and frames it as a hypothesis for future work. (All67numbers illustrative.)6869## Referee-pushback patterns and the JPART repair7071- *"This is common-method bias, not an effect."* → Use a separate/objective outcome or a marker72 variable; report the sensitivity, don't wave it away with a single Harman test.73- *"The robustness table only reruns near-identical specs."* → Replace decorative checks with specs that74 could break the result (alternative PSM/red-tape measures, samples), and say what held.75- *"This is selection into public service."* → State the estimand and assumption; report how strong an76 unobserved confounder must be to overturn it.77- *"I cannot tell confirmatory from exploratory."* → Segregate them explicitly; the deposited code is78 public, so the split must survive a re-run.7980## Calibration anchors (hedged)8182- The bar is a **public-management theory** payoff carried by credible numbers — an estimate with no83 mechanism rarely clears JPART review.84- JPART increasingly rewards **experimental and causal** designs, but a rigorous multilevel or mixed85 study is judged on its own terms.86- The **data-and-code release is mandatory** (where ethically possible) — write the analysis so the87 public package reproduces every printed number. Confirm exact wording on the live policy page.8889## Execution bridge (StatsPAI / Stata MCP)9091Run the battery, don't just enumerate it. Full map:92[`execution-with-mcp`](../../../shared-resources/empirical-methods/execution-with-mcp.md). JPART is public management — observational and experimental designs on public organizations; identification + clustered/multilevel inference.9394- **Many outcomes / specifications:** `romano_wolf` (step-down FWER) or `benjamini_hochberg`.95- **OVB sensitivity:** `oster_delta` / `sensemakr`.96- **Inference:** `wild_cluster_bootstrap` (few clusters), `twoway_cluster` / `conley`.97- **Re-fit off one handle:** `audit_result(result_id)` lists missing checks + the exact98 `suggest_function` for each.99- **Exhibits:** `etable` / `did_summary_to_latex` from the handle — no retyped numbers.100101Decisive checks in the body, exhaustive battery in the appendix.102[JF execution walkthrough](../../../Journal-of-Finance-Skills/resources/worked-examples/02-execution-walkthrough.md).103## Output format104105```106【Main estimate】magnitude + interval + substantive meaning107【PA threat handled】common-method / selection — how?108【Robustness】specs that could break it → what held109【Heterogeneity】pre-specified? MHT-adjusted?110【Confirmatory vs exploratory】clearly separated?111【Reproducible】master script + seeds + pinned versions? [Y/N]112【Next】jpart-tables-figures113```114115## Supplementary resources116117- [`../../resources/code/`](../../resources/code/) — Stata + Python estimation/inference skeleton118- [`../../resources/external_tools.md`](../../resources/external_tools.md) — estimation, inference, and experiment packages119- [`../../resources/official-source-map.md`](../../resources/official-source-map.md) — data-and-code release policy