Data Analysis (jmf-data-analysis)
JMF reviewers are methodologically sophisticated and family data are tricky: complex-survey weights,
panel attrition, missingness, and the non-independence of partners and family members all bite. This
skill covers execution and reporting norms; design decisions live in jmf-research-design.
When to trigger
- Running main and supporting analyses; building the results section
- A reviewer asked for robustness, heterogeneity, selection checks, or alternative specifications
- Reconciling preregistered vs. exploratory analyses
- Making the analysis reproducible before deposit
Analysis norms JMF expects
- Report uncertainty and magnitude honestly. Confidence intervals and effect sizes (not stars
alone); state the substantive size in family terms (months to divorce, percentage-point change in a
transition, points on a relationship-quality scale).
- Address selection explicitly. In observational family research, show what survives within-person/
couple fixed effects, sibling comparisons, propensity adjustment + sensitivity, or honest scoping.
- Respect the data structure. Apply complex-survey weights/clusters/strata for PSID, NSFG,
Add Health, etc.; cluster or model non-independence for dyads and families; use survival models
for time-to-event outcomes.
- Handle missingness principled-ly. Multiple imputation or FIML rather than listwise deletion;
report the missing-data mechanism assumption and the share imputed; probe panel attrition.
- Robustness that probes, not decorates. Alternative measures, samples, estimators, and
specifications that could break the result — and what you learn.
- Heterogeneity with discipline. Pre-specify subgroups (by gender, family structure, race/
ethnicity, cohort) where possible; correct for multiple comparisons; do not mine interactions.
Family-data specifics
- Report the analytic sample and how it was derived from the full dataset (eligibility, attrition,
missingness) — reviewers will check the funnel.
- For dyadic models, report actor and partner effects and whether dyads are distinguishable.
- For growth/event-history models, report the time metric, censoring, and competing risks.
Reproducibility while you work (not at the end)
- One master script regenerates every table and figure from the (raw or constructed) data.
- Set and report seeds for imputation, bootstrap, simulation, and any stochastic step.
- Pin software/package versions (
renv.lock, requirements.txt, recorded ssc/net installs).
- Keep table/figure numbers in the manuscript matched to script outputs.
What JMF referees check in a results section
| Check |
Passes review |
Triggers a revision request |
| Survey design |
PSID/NSFG/Add Health weights, strata, clusters applied |
Unweighted estimates from a complex sample |
| Non-independence |
Dyads/families clustered or modeled (APIM, multilevel) |
Partners treated as independent rows |
| Selection |
Within-couple FE, sibling, PSM+sensitivity, or honest scoping |
"Effect of" language on raw associations |
| Magnitude |
Effect in family units (months to divorce) + CI |
Stars only, no substantive size |
For the flagship journal of the National Council on Family Relations, the fastest desk-reject-adjacent
outcome is an analysis that ignores the survey design or the dependence built into couple and family data.
Worked micro-example (illustrative)
A divorce-timing study uses discrete-time event-history on national-panel marriages. Numbers illustrative.
- Naive (flagged): a logit pooling person-years with no clustering reports OR = 1.40 for job loss on
divorce, stars only.
- JMF-grade: a discrete-time hazard with survey weights and SEs clustered on the couple; job loss raises
the divorce hazard ~28% (HR 1.28, 95% CI 1.09–1.50), shifting modeled median time to divorce earlier by
~1.6 years. Selection probe: within-couple fixed effects leave a smaller HR ≈ 1.18, so it is not purely
sorting. Competing risks are modeled and the sample funnel reported.
Referee-pushback patterns and the venue-specific fix
- "Dyadic dependence ignored." Re-estimate with members nested in couples; for distinguishable dyads
report actor and partner effects separately and test equality.
- "Selection into family transitions." Add a within-unit (couple/sibling) comparison or a sensitivity
bound, and soften causal language to match.
- "Survey design ignored." Apply the dataset's weight/strata/PSU variables and report design-based SEs.
Calibration (hedged): for family panels (PSID, Fragile Families, Add Health, NSFG), design-based inference
and principled missing-data handling are defaults; confirm dataset-specific guidance with provider
documentation.
Execution bridge (StatsPAI / Stata MCP)
Run the battery, don't just enumerate it. Full map:
execution-with-mcp. JMF is quantitative family demography/sociology; emphasize identification, selection, and decomposition methods.
- Many outcomes / specifications:
romano_wolf (step-down FWER) or
benjamini_hochberg — report the adjusted threshold.
- OVB sensitivity:
oster_delta / sensemakr.
- Inference:
wild_cluster_bootstrap (few clusters), twoway_cluster / conley;
multilevel data → cluster at the right level.
- Re-fit off one handle:
audit_result(result_id) lists the missing checks and the
exact suggest_function for each.
- Exhibits:
etable / did_summary_to_latex from the handle — no retyped numbers.
Keep the decisive checks in the body and the exhaustive battery in the supplement. See
the executed chain in the JF execution walkthrough.
Anti-patterns
- Stars-only tables with no effect sizes, intervals, or substantive interpretation
- Ignoring survey weights/design or treating dyad members as independent
- Listwise deletion that silently changes the sample; ignoring attrition
- p-hacking / fishing for a significant interaction; HARKing exploratory results into hypotheses
- A results section whose numbers the code cannot reproduce
Output format
【Main estimate】magnitude + interval + substantive (family) meaning
【Selection check】(per research-design) result
【Data structure】weights/clusters/dyad-family non-independence handled? [Y/N]
【Missing data】MI/FIML + attrition probed? [Y/N]
【Robustness】specs that could break it → what held
【Reproducible】master script + seeds + pinned versions? [Y/N]
【Next】jmf-tables-figures
Supplementary resources
1---2name: jmf-data-analysis3description: Use when executing and reporting the analysis for a Journal of Marriage and Family (JMF) manuscript so it survives expert double-blind review — honest uncertainty, robustness, attention to selection and non-independence, and correct handling of complex-survey, longitudinal, and dyadic data. Guides analysis norms; it does not fabricate results.4metadata:5 source: https://skillmd.com/skills/brycewang-stanford/jmf-data-analysis6---78# Data Analysis (jmf-data-analysis)910JMF reviewers are methodologically sophisticated and family data are tricky: complex-survey weights,11panel attrition, missingness, and the non-independence of partners and family members all bite. This12skill covers execution and reporting norms; design decisions live in `jmf-research-design`.1314## When to trigger1516- Running main and supporting analyses; building the results section17- A reviewer asked for robustness, heterogeneity, selection checks, or alternative specifications18- Reconciling preregistered vs. exploratory analyses19- Making the analysis reproducible before deposit2021## Analysis norms JMF expects22231. **Report uncertainty and magnitude honestly.** Confidence intervals and effect sizes (not stars24 alone); state the substantive size in family terms (months to divorce, percentage-point change in a25 transition, points on a relationship-quality scale).262. **Address selection explicitly.** In observational family research, show what survives within-person/27 couple fixed effects, sibling comparisons, propensity adjustment + sensitivity, or honest scoping.283. **Respect the data structure.** Apply **complex-survey** weights/clusters/strata for PSID, NSFG,29 Add Health, etc.; cluster or model **non-independence** for dyads and families; use survival models30 for time-to-event outcomes.314. **Handle missingness principled-ly.** Multiple imputation or FIML rather than listwise deletion;32 report the missing-data mechanism assumption and the share imputed; probe panel **attrition**.335. **Robustness that probes, not decorates.** Alternative measures, samples, estimators, and34 specifications that could *break* the result — and what you learn.356. **Heterogeneity with discipline.** Pre-specify subgroups (by gender, family structure, race/36 ethnicity, cohort) where possible; correct for multiple comparisons; do not mine interactions.3738## Family-data specifics39- Report the analytic sample and how it was derived from the full dataset (eligibility, attrition,40 missingness) — reviewers will check the funnel.41- For dyadic models, report actor and partner effects and whether dyads are distinguishable.42- For growth/event-history models, report the time metric, censoring, and competing risks.4344## Reproducibility while you work (not at the end)45- One **master script** regenerates every table and figure from the (raw or constructed) data.46- **Set and report seeds** for imputation, bootstrap, simulation, and any stochastic step.47- Pin software/package versions (`renv.lock`, `requirements.txt`, recorded `ssc`/`net` installs).48- Keep table/figure numbers in the manuscript matched to script outputs.4950## What JMF referees check in a results section5152| Check | Passes review | Triggers a revision request |53|-------|---------------|------------------------------|54| Survey design | PSID/NSFG/Add Health weights, strata, clusters applied | Unweighted estimates from a complex sample |55| Non-independence | Dyads/families clustered or modeled (APIM, multilevel) | Partners treated as independent rows |56| Selection | Within-couple FE, sibling, PSM+sensitivity, or honest scoping | "Effect of" language on raw associations |57| Magnitude | Effect in family units (months to divorce) + CI | Stars only, no substantive size |5859For the flagship journal of the National Council on Family Relations, the fastest desk-reject-adjacent60outcome is an analysis that ignores the survey design or the dependence built into couple and family data.6162## Worked micro-example (illustrative)6364A divorce-timing study uses discrete-time event-history on national-panel marriages. Numbers illustrative.6566- *Naive (flagged):* a logit pooling person-years with no clustering reports OR = 1.40 for job loss on67 divorce, stars only.68- *JMF-grade:* a discrete-time hazard with survey weights and SEs clustered on the couple; job loss raises69 the divorce hazard ~28% (HR 1.28, 95% CI 1.09–1.50), shifting modeled median time to divorce earlier by70 ~1.6 years. Selection probe: within-couple fixed effects leave a smaller HR ≈ 1.18, so it is not purely71 sorting. Competing risks are modeled and the sample funnel reported.7273## Referee-pushback patterns and the venue-specific fix7475- *"Dyadic dependence ignored."* Re-estimate with members nested in couples; for distinguishable dyads76 report actor and partner effects separately and test equality.77- *"Selection into family transitions."* Add a within-unit (couple/sibling) comparison or a sensitivity78 bound, and soften causal language to match.79- *"Survey design ignored."* Apply the dataset's weight/strata/PSU variables and report design-based SEs.8081*Calibration (hedged):* for family panels (PSID, Fragile Families, Add Health, NSFG), design-based inference82and principled missing-data handling are defaults; confirm dataset-specific guidance with provider83documentation.8485## Execution bridge (StatsPAI / Stata MCP)8687Run the battery, don't just enumerate it. Full map:88[`execution-with-mcp`](../../../shared-resources/empirical-methods/execution-with-mcp.md). JMF is quantitative family demography/sociology; emphasize identification, selection, and decomposition methods.8990- **Many outcomes / specifications:** `romano_wolf` (step-down FWER) or91 `benjamini_hochberg` — report the adjusted threshold.92- **OVB sensitivity:** `oster_delta` / `sensemakr`.93- **Inference:** `wild_cluster_bootstrap` (few clusters), `twoway_cluster` / `conley`;94 multilevel data → cluster at the right level.95- **Re-fit off one handle:** `audit_result(result_id)` lists the missing checks and the96 exact `suggest_function` for each.97- **Exhibits:** `etable` / `did_summary_to_latex` from the handle — no retyped numbers.9899Keep the decisive checks in the body and the exhaustive battery in the supplement. See100the executed chain in the [JF execution walkthrough](../../../Journal-of-Finance-Skills/resources/worked-examples/02-execution-walkthrough.md).101## Anti-patterns102103- Stars-only tables with no effect sizes, intervals, or substantive interpretation104- Ignoring survey weights/design or treating dyad members as independent105- Listwise deletion that silently changes the sample; ignoring attrition106- p-hacking / fishing for a significant interaction; HARKing exploratory results into hypotheses107- A results section whose numbers the code cannot reproduce108109## Output format110111```112【Main estimate】magnitude + interval + substantive (family) meaning113【Selection check】(per research-design) result114【Data structure】weights/clusters/dyad-family non-independence handled? [Y/N]115【Missing data】MI/FIML + attrition probed? [Y/N]116【Robustness】specs that could break it → what held117【Reproducible】master script + seeds + pinned versions? [Y/N]118【Next】jmf-tables-figures119```120121## Supplementary resources122123- [`../../resources/external_tools.md`](../../resources/external_tools.md) — estimation, survey, dyadic, survival, and imputation packages124- [`../../resources/official-source-map.md`](../../resources/official-source-map.md) — replication-detail and data expectations