Data Analysis (devpsych-data-analysis)
Developmental Psychology holds analyses to a developmental and a credibility standard at once: the model
must actually capture change (not just a cross-sectional snapshot), and reporting must meet JARS —
effect sizes with confidence intervals, full disclosure, and a clean confirmatory vs. exploratory
split. The most common fatal error is interpreting trajectories without first establishing that the
construct is measured the same way across ages.
When to trigger
- Fitting growth-curve / multilevel / SEM models, or mediation/moderation of developmental effects
- A reviewer asked for measurement invariance, effect sizes, intervals, or attrition handling
- Reconciling preregistered developmental hypotheses with exploratory trajectory findings
- Preparing analysis scripts and a data dictionary for deposit
Reporting norms Developmental Psychology expects
- Model change correctly. Use the method the claim requires: latent growth / multilevel models
for trajectories, SEM for latent constructs, cross-lagged / RI-CLPM for reciprocal effects,
mediation/moderation for mechanism and moderated change. State time coding and centering.
- Establish measurement invariance first. Test configural → metric → scalar across ages/waves
before interpreting mean change; report partial invariance honestly if full scalar fails.
- Effect sizes + uncertainty. Report a standardized or unstandardized effect size and confidence
intervals for major results — slope estimates, interactions, indirect effects — not just stars.
- Handle missing data and attrition principledly. Use FIML or multiple imputation; report the
attrition analysis (completers vs. dropouts) and the missingness assumption.
- JARS disclosure. Report how sample size was determined, all exclusions and reasons, all conditions
and measures; keep confirmatory and exploratory analyses clearly separated.
- Reproducibility. Provide analysis scripts and a data dictionary; the numbers should regenerate in
a fresh session (see
devpsych-open-science-and-transparency).
Worked micro-example (illustrative numbers)
A preregistered three-wave latent-growth study (ages 4, 6, 8; N = 300, 18% attrition) of effortful
control, testing maternal scaffolding as a driver of the growth slope.
Invariance (reported first):
configural fit good; metric and scalar invariance hold across waves
(ΔCFI < .01) → mean change is interpretable.
Confirmatory (preregistered):
Latent slope > 0: b = 0.42/year, 95% CI [0.31, 0.53] (within-person growth).
Scaffolding × time: b = 0.18, 95% CI [0.07, 0.29] (steeper growth with
higher wave-1 scaffolding).
Missing data: FIML; MAR; completers and dropouts did not differ on baseline
covariates (attrition analysis in supplement).
Exploratory (labeled):
RI-CLPM suggests child→parent effects in later waves; reported as
exploratory and flagged for confirmation in a future sample.
Why this passes scrutiny: invariance is reported before the growth claim; every developmental parameter
carries an effect size and a CI; missingness is modeled, not deleted; the reciprocal-effects finding is
honestly demoted to exploratory.
Analysis-stage reviewer pushback and the venue fix
| Reviewer pushback |
What it signals here |
Developmental Psychology fix |
| "Is the construct the same at each age?" |
invariance not tested |
report configural→metric→scalar before interpreting change |
| "You deleted dropouts" |
attrition bias |
refit with FIML/MI; add the completers-vs-dropouts analysis |
| "ANOVA on age groups for a change claim" |
wrong model for the claim |
fit a growth/multilevel model on within-person data |
| "Stars, no effect size" |
pre-reform reporting |
report slope/interaction effect sizes with CIs |
| "Is this confirmatory?" |
HARKing concern |
point to preregistration; relabel post hoc trajectories exploratory |
Calibration anchors
- A clean latent-growth slope with a tight CI, on an invariant measure, beats an age-group ANOVA with
stars — the venue's currency is credible change, not a snapshot contrast.
- Prefer estimation language ("effortful control grew 0.42/year, 95% CI [...]") to "significant effect of
age." Bare p-value sentences read as thin here.
- When attrition is non-trivial, state the missingness assumption and show the trajectory is robust to a
reasonable alternative (e.g., pattern-mixture sensitivity), rather than implying complete data.
Anti-patterns
- Interpreting mean change without establishing measurement invariance
- Listwise deletion or ignoring differential attrition
- Using age-group ANOVA to support a within-person change claim
- p-values and stars with no effect sizes or confidence intervals
- HARKing exploratory trajectory shapes into confirmatory hypotheses
Output format
【Model】growth / multilevel / SEM / cross-lagged / mediation-moderation — matches the change claim?
【Invariance】configural→metric→scalar tested before interpreting change? [Y/N]
【Main result】effect size + confidence interval + meaning
【Missing data】FIML/MI + attrition analysis reported? [Y/N]
【Confirmatory vs exploratory】clearly separated (JARS)? [Y/N]
【Reproducible】scripts + data dictionary + fresh-session check? [Y/N]
【Next】devpsych-tables-figures
Supplementary resources
Source: brycewang-stanford/Awesome-Journal-Skills → Developmental-Psychology-Skills/skills/devpsych-data-analysis/SKILL.md
1---2name: devpsych-data-analysis3description: Use when analyzing and reporting results for a Developmental Psychology (APA) manuscript. The journal expects analyses that model developmental change correctly — growth-curve/multilevel/SEM, mediation/moderation, measurement invariance — with effect sizes and confidence intervals, JARS-compliant disclosure, and a clear confirmatory/exploratory split. Guides analysis norms; it does not fabricate results.4---567# Data Analysis (devpsych-data-analysis)89Developmental Psychology holds analyses to a developmental and a credibility standard at once: the model10must actually capture **change** (not just a cross-sectional snapshot), and reporting must meet **JARS** —11**effect sizes with confidence intervals**, full disclosure, and a clean **confirmatory vs. exploratory**12split. The most common fatal error is interpreting trajectories without first establishing that the13construct is measured the same way across ages.1415## When to trigger1617- Fitting growth-curve / multilevel / SEM models, or mediation/moderation of developmental effects18- A reviewer asked for measurement invariance, effect sizes, intervals, or attrition handling19- Reconciling preregistered developmental hypotheses with exploratory trajectory findings20- Preparing analysis scripts and a data dictionary for deposit2122## Reporting norms Developmental Psychology expects23241. **Model change correctly.** Use the method the claim requires: **latent growth / multilevel models**25 for trajectories, **SEM** for latent constructs, **cross-lagged / RI-CLPM** for reciprocal effects,26 **mediation/moderation** for mechanism and moderated change. State time coding and centering.272. **Establish measurement invariance first.** Test **configural → metric → scalar** across ages/waves28 *before* interpreting mean change; report partial invariance honestly if full scalar fails.293. **Effect sizes + uncertainty.** Report a standardized or unstandardized effect size **and confidence30 intervals** for major results — slope estimates, interactions, indirect effects — not just stars.314. **Handle missing data and attrition principledly.** Use **FIML or multiple imputation**; report the32 attrition analysis (completers vs. dropouts) and the missingness assumption.335. **JARS disclosure.** Report how sample size was determined, all exclusions and reasons, all conditions34 and measures; keep confirmatory and exploratory analyses clearly separated.356. **Reproducibility.** Provide analysis scripts and a data dictionary; the numbers should regenerate in36 a fresh session (see `devpsych-open-science-and-transparency`).3738## Worked micro-example (illustrative numbers)3940A preregistered three-wave latent-growth study (ages 4, 6, 8; N = 300, 18% attrition) of effortful41control, testing maternal scaffolding as a driver of the growth slope.4243```44Invariance (reported first):45 configural fit good; metric and scalar invariance hold across waves46 (ΔCFI < .01) → mean change is interpretable.47Confirmatory (preregistered):48 Latent slope > 0: b = 0.42/year, 95% CI [0.31, 0.53] (within-person growth).49 Scaffolding × time: b = 0.18, 95% CI [0.07, 0.29] (steeper growth with50 higher wave-1 scaffolding).51 Missing data: FIML; MAR; completers and dropouts did not differ on baseline52 covariates (attrition analysis in supplement).53Exploratory (labeled):54 RI-CLPM suggests child→parent effects in later waves; reported as55 exploratory and flagged for confirmation in a future sample.56```5758Why this passes scrutiny: invariance is reported *before* the growth claim; every developmental parameter59carries an effect size and a CI; missingness is modeled, not deleted; the reciprocal-effects finding is60honestly demoted to exploratory.6162## Analysis-stage reviewer pushback and the venue fix6364| Reviewer pushback | What it signals here | Developmental Psychology fix |65|-------------------|----------------------|------------------------------|66| "Is the construct the same at each age?" | invariance not tested | report configural→metric→scalar before interpreting change |67| "You deleted dropouts" | attrition bias | refit with FIML/MI; add the completers-vs-dropouts analysis |68| "ANOVA on age groups for a change claim" | wrong model for the claim | fit a growth/multilevel model on within-person data |69| "Stars, no effect size" | pre-reform reporting | report slope/interaction effect sizes with CIs |70| "Is this confirmatory?" | HARKing concern | point to preregistration; relabel post hoc trajectories exploratory |7172## Calibration anchors7374- A clean latent-growth slope with a tight CI, on an invariant measure, beats an age-group ANOVA with75 stars — the venue's currency is credible *change*, not a snapshot contrast.76- Prefer estimation language ("effortful control grew 0.42/year, 95% CI [...]") to "significant effect of77 age." Bare p-value sentences read as thin here.78- When attrition is non-trivial, state the missingness assumption and show the trajectory is robust to a79 reasonable alternative (e.g., pattern-mixture sensitivity), rather than implying complete data.8081## Anti-patterns8283- Interpreting mean change without establishing measurement invariance84- Listwise deletion or ignoring differential attrition85- Using age-group ANOVA to support a within-person change claim86- p-values and stars with no effect sizes or confidence intervals87- HARKing exploratory trajectory shapes into confirmatory hypotheses8889## Output format9091```92【Model】growth / multilevel / SEM / cross-lagged / mediation-moderation — matches the change claim?93【Invariance】configural→metric→scalar tested before interpreting change? [Y/N]94【Main result】effect size + confidence interval + meaning95【Missing data】FIML/MI + attrition analysis reported? [Y/N]96【Confirmatory vs exploratory】clearly separated (JARS)? [Y/N]97【Reproducible】scripts + data dictionary + fresh-session check? [Y/N]98【Next】devpsych-tables-figures99```100101## Supplementary resources102103- [`../../resources/external_tools.md`](../../resources/external_tools.md) — `lavaan`, Mplus, `lme4`/`nlme`, `semTools` invariance, `mice`, effect-size tooling104- [`../../resources/official-source-map.md`](../../resources/official-source-map.md) — JARS statistical and disclosure requirements105106---107108**Source:** [`brycewang-stanford/Awesome-Journal-Skills`](https://github.com/brycewang-stanford/Awesome-Journal-Skills) → `Developmental-Psychology-Skills/skills/devpsych-data-analysis/SKILL.md`
thedixitjain