thedixitjain Clinical Statistics & Reporting (nejm-statistics)
When to trigger
- Results report P values without confidence intervals.
- The primary analysis population (ITT vs per-protocol) is unclear.
- Many secondary endpoints are tested with no multiplicity plan.
- Subgroup findings are highlighted without pre-specification or interaction tests.
- Only relative risk is reported, with no absolute risk or NNT.
Confidence intervals over bare P values
NEJM's statistical reporting guidance favors effect estimates with 95% confidence intervals over bare significance tests.
- Every primary and key secondary outcome: effect estimate + 95% CI (HR/RR/OR or mean difference).
- Report exact P values (e.g., P=0.03), not "P<0.05", except for very small values.
- A CI that includes the null is not "no difference" — report the estimate and its width; interpret precision.
- For observational data, use cautious, non-causal language (associated with, not causes).
The pre-specified primary analysis
- The primary outcome is single and pre-specified; its analysis follows the SAP.
- Intention-to-treat is the primary analysis population for superiority trials; per-protocol is a sensitivity analysis. (For non-inferiority, both ITT and per-protocol matter — report both.)
- State the analysis model (e.g., Cox, logistic, mixed model) and pre-specified covariates/stratification.
Multiplicity (the secondary-endpoint trap)
- Pre-specify a testing hierarchy or a multiplicity adjustment (e.g., gatekeeping, Hochberg, Bonferroni) for multiple primary/secondary endpoints.
- Endpoints not in the controlled testing scheme are exploratory — say so, and report estimates with CIs but no inferential P claims.
- Do not declare a secondary endpoint "significant" if it sits outside the multiplicity plan.
Subgroups (warn against over-interpretation)
- Subgroup analyses must be pre-specified; label post-hoc subgroups as exploratory.
- Test the interaction (treatment × subgroup), not just within-subgroup P values — a within-subgroup "significant" effect without a significant interaction is weak evidence.
- Present subgroups in a forest plot with interaction P values (see
nejm-figures-tables). - Do not let a subgroup result become the headline of an otherwise null trial.
Absolute risk and NNT
- Report absolute risk difference alongside relative measures — a large relative reduction on a rare outcome can be a tiny absolute one.
- Where clinically meaningful, give the number needed to treat (NNT) (or number needed to harm).
- For safety, report absolute event rates by arm, not relative comparisons alone.
Missing data, survival, and other essentials
- State the amount and pattern of missing data and the handling (e.g., multiple imputation); avoid naive last-observation-carried-forward as the primary approach.
- Time-to-event: Kaplan–Meier with numbers-at-risk; Cox model with the proportional-hazards assumption checked; report median follow-up.
- Pre-specify interim analyses / stopping boundaries; report them if a DSMB acted.
- For non-inferiority: state the pre-specified margin and its justification.
What the statistical reviewer probes
NEJM routinely assigns trials a dedicated statistical reviewer (see nejm-rebuttal). Pre-empt the line items:
- SAP concordance — every analysis traces to the pre-specified SAP; deviations listed and dated.
- Event counts behind every ratio — per-arm numerators and denominators for every reported estimate.
- CI width, not just position — an interval spanning trivial and decisive effects is inconclusive.
- Assumptions checked — proportional hazards verified; how stratification variables entered the model.
- Cross-location consistency — the same estimate, to the same decimal, in abstract, text, tables, and figures.
Worked micro-example — a subgroup claim (before → after)
- Before: "The benefit was significant in patients older than 65 years (P=0.04) but not younger (P=0.31), so older patients should receive the drug."
- After: "The hazard ratio was 0.68 (95% CI, 0.47 to 0.98) among patients older than 65 years and 0.89 (95% CI, 0.64 to 1.24) among younger patients (P=0.29 for interaction)."
The estimates, CIs, and interaction test replace significance chatter; the nonsignificant interaction disciplines the conclusion. (Numbers invented.)
Output format
【Per-outcome reporting】 effect + 95% CI present for primary + key secondary? gaps: [...]
【Statistical-reviewer preflight】 SAP concordance / event counts / assumptions / cross-location consistency? yes/no
【Primary analysis population】 ITT primary? per-protocol as sensitivity? yes/no
【Multiplicity】 hierarchy/adjustment pre-specified? exploratory endpoints labeled? yes/no
【Subgroups】 pre-specified? interaction tests reported? over-interpretation flags: [...]
【Absolute risk + NNT】 reported alongside relative measures? yes/no
【Missing data / survival / non-inferiority margin】 handled & stated? yes/no
【Causal language (observational)】 appropriately cautious? yes/no
【Next】 nejm-figures-tables
Anti-patterns
- Do not report relative risk without absolute risk for clinical outcomes.
- Do not declare secondary or subgroup endpoints significant outside the multiplicity plan.
- Do not infer "no effect" from a wide CI crossing the null on an underpowered comparison.
- Do not use causal verbs ("reduced", "caused") for observational associations.
- Do not make per-protocol the primary analysis in a superiority trial.
Source: brycewang-stanford/Awesome-Journal-Skills → NEJM-Skills/skills/nejm-statistics/SKILL.md