# Joap Data Analysis > Use when analyzing and reporting results for a Journal of Applied Psychology (JAP) manuscript using SEM, multilevel (HLM) models, mediation/moderation, or meta-analysis. JAP requires effect sizes with confidence intervals, model-based indirect effects with bootstrap CIs, fit indices, full disclosure, and a clean confirmatory/exploratory split. Guides analysis norms; it does not fabricate results. - Skill: `thedixitjain/joap-data-analysis` (Agent Skill) - Install (CLI): `npx skillmds add thedixitjain/joap-data-analysis` - Raw SKILL.md: https://api.skillmd.com/api/skills/thedixitjain/joap-data-analysis/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: AI & ML - Author: thedixitjain (https://skillmd.com/u/thedixitjain) - Updated: 2026-09-08 - Page: https://skillmd.com/skills/thedixitjain/joap-data-analysis --- # Data Analysis (joap-data-analysis) JAP analyses must be **model-appropriate, fully reported, and reproducible**. The house toolkit is **SEM/CFA**, **multilevel (HLM) models**, **mediation and moderation** with proper inference, and **meta-analysis**. The journal expects **effect sizes with confidence intervals**, **fit indices**, **bootstrap CIs for indirect effects**, full disclosure of how data were handled, and a clean **confirmatory vs. exploratory** separation, with data and code shareable under TOP. ## When to trigger - Specifying and reporting the main and supporting analyses - A reviewer asked for fit indices, indirect-effect CIs, robustness, or disclosure - Reconciling preregistered analyses with exploratory follow-ups - Preparing analysis scripts and a data/codebook for deposit ## Reporting norms JAP expects 1. **Measurement before structure.** Report the measurement model (CFA fit: χ²/df, CFI, TLI, RMSEA, SRMR) and reliability/AVE before interpreting the structural model; report measurement invariance when comparing groups or waves. 2. **Effect sizes + uncertainty.** Give standardized and/or unstandardized estimates **with confidence intervals** for key paths — not just p-values and stars. 3. **Mediation done right.** Report the **indirect effect with a bootstrap (or Monte Carlo) CI** (not only the Baron-Kenny steps or a Sobel z); for multilevel mediation use the appropriate (e.g., 1-1-1, 2-1-1, 2-2-1) decomposition and within/between separation. 4. **Multilevel correctly.** Report ICC(1)/ICC(2), center predictors appropriately (group-mean vs grand-mean), model random effects, and justify the estimator; do not run OLS on nested data. 5. **Full disclosure.** Report how sample size was determined, **all** exclusions (careless responding, attrition) and reasons, all conditions, and all measures. Label confirmatory vs. exploratory. 6. **Meta-analysis (if applicable).** Report the model (random vs fixed), heterogeneity (Q, I²), artifact corrections, publication-bias checks, and a transparent coding protocol. 7. **Reproducibility.** Provide scripts and a codebook; results should regenerate from shared data in a fresh session (see `joap-open-science-and-transparency`). ## Worked micro-example (illustrative numbers) The servant-leadership package: a multilevel mediation in the field and a causal test in the lab. ``` Measurement model (field, N = 612 in 74 teams) CFA: χ²/df = 2.1, CFI = .96, TLI = .95, RMSEA = .045, SRMR = .04 Reliabilities ω: leadership .91, safety .88, performance .87 Aggregation: ICC(1) = .16, ICC(2) = .77, r_wg(j) = .85 → team-level OK Confirmatory (preregistered) — multilevel mediation (2-2-2) a (leadership→safety) = .42 [.27, .57]; b (safety→performance) = .31 [.14, .48] Indirect = .13, 95% Monte Carlo CI [.05, .23] → mediation supported Direct (leadership→performance | safety) = .09 [-.06, .24], ns Confirmatory (preregistered) — lab experiment (causal leg) Servant vs control on safety: d = 0.46, 95% CI [0.21, 0.71] H3 boundary: interaction with interdependence, ΔR² = .03, CI excludes 0 Exploratory (labeled): voice as a serial L1 mediator surfaced post hoc; reported as exploratory, flagged for confirmation in future work. ``` Why this passes JAP scrutiny: the measurement model is reported before structure; the indirect effect carries a bootstrap/Monte Carlo CI; nesting is modeled and aggregation justified; the experimental leg supplies causal warrant; and the post hoc serial path is honestly demoted to exploratory. ## Analysis-stage reviewer pushback and the venue fix | Reviewer pushback | What it signals | JAP fix | |-------------------|-----------------|---------| | "Mediation by Sobel/steps only" | outdated inference | report indirect effect + bootstrap/Monte Carlo CI | | "OLS on nested data" | dependence ignored | multilevel model; report ICC, centering, random effects | | "No fit indices / measurement model" | construct validity unchecked | report CFA fit + reliability before structure | | "Which exclusions were preregistered?" | forking-paths concern | disclosure table: rule, count, preregistered vs post hoc, result with/without | | "Is this confirmatory?" | HARKing concern | point to the preregistration; relabel post hoc analyses exploratory | | "Reviewer 2 couldn't rerun your code" | reproducibility gate | ship a fresh-session run log (see open-science skill) | ## Calibration anchors - Report measurement before structure: a beautiful path model on a misfitting measurement model convinces no JAP reviewer. - Prefer estimation language with CIs to dichotomous "significant/not"; bare p-value sentences read as pre-reform. - For mediation and cross-level effects, the *interval on the carrying effect* is the result — design and report so that interval is defensible. ## Execution bridge (StatsPAI / Stata MCP) Run the battery, don't just enumerate it. Full map: [`execution-with-mcp`](../../../shared-resources/empirical-methods/execution-with-mcp.md). JAP is organizational psychology — multilevel survey/field data and experiments; cluster at the right level and apply mediation/moderation discipline. - **Many outcomes / specifications:** `romano_wolf` (step-down FWER) or `benjamini_hochberg` — report the adjusted threshold. - **OVB sensitivity:** `oster_delta` / `sensemakr`. - **Inference:** `wild_cluster_bootstrap` (few clusters), `twoway_cluster` / `conley`; multilevel data → cluster at the right level. - **Re-fit off one handle:** `audit_result(result_id)` lists the missing checks and the exact `suggest_function` for each. - **Exhibits:** `etable` / `did_summary_to_latex` from the handle — no retyped numbers. Keep the decisive checks in the body and the exhaustive battery in the supplement. See the executed chain in the [JF execution walkthrough](../../../Journal-of-Finance-Skills/resources/worked-examples/02-execution-walkthrough.md). ## Anti-patterns - p-values and stars with no effect size, CI, or fit indices - Mediation claimed via Baron-Kenny steps or Sobel z without a bootstrap/Monte Carlo CI - Ignoring nesting (OLS on multilevel data) or aggregating without ICC/r_wg justification - Selectively reporting conditions, measures, or exclusions (undisclosed flexibility) - HARKing exploratory findings into confirmatory hypotheses - Code that does not reproduce the reported numbers ## Output format ``` 【Measurement model】CFA fit + reliability + invariance reported? [Y/N] 【Main result】effect size(s) + CI(s) + meaning 【Mediation/multilevel】indirect-effect bootstrap/Monte Carlo CI; nesting modeled? [Y/N/NA] 【Disclosure】N-determination + all exclusions + all conditions + all measures? [Y/N] 【Confirmatory vs exploratory】clearly separated? [Y/N] 【Reproducible】scripts + codebook + fresh-session check? [Y/N] 【Next】joap-tables-figures ``` ## Supplementary resources - [`../../resources/external_tools.md`](../../resources/external_tools.md) — Mplus, `lavaan`, `lme4`/`nlme`, `psych`, `metafor`/`metaSEM`, bootstrap/Monte Carlo tools - [`../../resources/official-source-map.md`](../../resources/official-source-map.md) — statistical and disclosure requirements --- **Source:** [`brycewang-stanford/Awesome-Journal-Skills`](https://github.com/brycewang-stanford/Awesome-Journal-Skills) → `Journal-of-Applied-Psychology-Skills/skills/joap-data-analysis/SKILL.md`