gabrielmoreira Version Compatibility
Reference examples tested with: R mmrm 0.3+ (Roche/openpharma), R rbmi 1.5+ (Roche/Bayer via insightsengineering), R mice 3.16+, R mitools 2.4+, Python sklearn 1.4+, statsmodels 0.14+.
Before using code patterns, verify installed versions match. If versions differ:
- R:
packageVersion('<pkg>')then?function_name - Python:
pip show <package>thenhelp(module.function)
If code throws ImportError, AttributeError, or TypeError, introspect the installed package and adapt the example to match the actual API rather than retrying.
Missing Data Sensitivity for Confirmatory Trials
"Handle missing data in a confirmatory clinical trial" -> Pre-specify the missing-data assumption per ICH E9(R1); execute the primary analysis under the chosen assumption (typically MAR via MMRM or MI); run clinically-articulable MNAR sensitivity analyses (reference-based MI per Carpenter-Roger 2013); report the tipping delta that would overturn the conclusion (Permutt 2016).
The Foundation -- NRC 2010 and ICH E9(R1)
The U.S. National Research Council Panel ("The Prevention and Treatment of Missing Data in Clinical Trials," 2010; chaired by Roderick Little; Little, D'Agostino, Cohen et al 2012 NEJM 367:1355): 18 recommendations grouped as prevention (Recs 1-7), analysis (Recs 8-14), sensitivity (Recs 15-18).
Key recommendations:
- Rec 10/11: explicitly REJECT LOCF and BOCF as default; they are biased even under MCAR
- Rec 13: endorses WGEE (weighted GEE) for marginal estimands
- Rec 15: "examining sensitivity to assumptions about the missing-data mechanism should be a mandatory component of reporting"
ICH E9(R1) (2019) forces the ordering: define the estimand (5 attributes including ICE strategy) BEFORE choosing the analysis. The missing-data strategy maps to the ICE handling strategy:
- Treatment policy ICE strategy + missing post-ICE data -> reference-based MI (J2R typical)
- Hypothetical ICE strategy -> MMRM under MAR; g-computation
- Composite ICE strategy -> ICE becomes part of endpoint; no missing-data problem for that subject
- While-on-treatment ICE strategy -> pre-ICE values only; censored at ICE
- Principal stratum -> latent stratum, requires Bayesian or sensitivity over unverifiable assumptions
Missing-Data Mechanisms
| Mechanism | Definition | Testable? | Valid method |
|---|---|---|---|
| MCAR | Independent of all data | Partially (Little's 1988 test) | Complete-case unbiased but loses power |
| MAR | Depends on observed data only | NOT testable | MMRM under MAR; MI under MAR |
| MNAR | Depends on unobserved values | NOT testable | Sensitivity analysis (J2R, CR, CIR, tipping point, pattern-mixture, selection model) |
Critical philosophical point: MAR vs MNAR cannot be distinguished from observed data alone. This is fundamental. Pre-specify the assumed mechanism in the SAP based on clinical reasoning, not the data.
Algorithmic Taxonomy
| Method | Estimand strategy | Identification | Variance | Strength | Fails when |
|---|---|---|---|---|---|
| Complete-case analysis | MAR or MCAR | MAR | Standard | Simple; valid under MCAR | Loses power; biased under MAR with informative covariates |
| LOCF | Implicit MNAR | Assumes flat post-ICE trajectory | Standard | Historically used | Biased even under MCAR (Mallinckrodt 2008); NRC 2010 rejects |
| MMRM with UN+KR | Hypothetical via MAR | MAR | Kenward-Roger SE | FDA-favoured continuous longitudinal | High differential dropout makes MAR implausible |
| Multiple imputation (MAR) | Hypothetical via MAR | MAR | Rubin's rules | Flexible; handles arbitrary patterns | sample_posterior must be enabled; only works with default estimator in sklearn |
| Reference-based MI (J2R, CR, CIR, LMCF) | Treatment policy / MNAR sensitivity | Clinical narrative (e.g., "after withdrawal patient resembles placebo") | Cro 2019 information-anchored vs Wolbers 2022 frequentist (active debate) | FDA-acceptable; clinician-interpretable | Variance choice contested; Rubin over-conservative, jackknife may inflate Type-I |
| Tipping-point delta-adjustment | MNAR sensitivity | Pre-specified delta function | Standard | Direct regulatory question: "how bad would missing data have to be?" | Delta interpretation depends on scale |
| Pattern-mixture with CCMV | Pattern-mixture MNAR | "Missing pattern resembles completer pattern" | Multiple imputation | Identifies missing cells via restriction | CCMV may be implausible if completers are atypical |
| Pattern-mixture with NCMV | Pattern-mixture MNAR | "Missing pattern resembles neighbouring pattern" | MI | Less extreme assumption than CCMV | Choice of "neighbouring" is ambiguous |
| Pattern-mixture with ACMV | Pattern-mixture MNAR (equivalent to MAR) | "Missing pattern resembles available cases" | MI | Equivalent to MAR (Molenberghs 1998) | Reduces to standard MAR analysis |
| Selection model (Diggle-Kenward 1994) | MNAR | Joint normal outcome + logistic dropout | Likelihood | Theoretically elegant | Conclusions driven by untestable parametric assumptions; FDA discouraged |
| Retrieved-dropout MI | Treatment policy | Sampling from observed post-discontinuation data | MI variance | Empirically grounded (no model assumption for missing) | Requires actual post-ICE data collection |
Postdoc reading list:
- NRC 2010 Prevention and Treatment of Missing Data in Clinical Trials (National Academies)
- Little RJA, D'Agostino R, Cohen ML et al 2012 NEJM 367:1355 (NRC summary)
- Mallinckrodt CH 2008/2014 Drug Information Journal / TIRS (MMRM case)
- Carpenter JR, Roger JH, Kenward MG 2013 J Biopharm Stat 23:1352 (reference-based MI)
- Cro S, Carpenter JR, Kenward MG 2019 JRSS-A 182:623 (information-anchored variance)
- Bartlett JW 2021 Stat Biopharm Res 15(1):178 (frequentist variance counter)
- Wolbers M, Noci A, Delmar P et al 2022 Pharm Stat 21(6):1246-1257 (CMI+jackknife)
- Permutt T 2016 Stat Med 35:2876 (analyst as adversary; tipping point)
- Diggle PJ, Kenward MG 1994 JRSS-C 43:49 (selection model + canonical critique)
- Molenberghs G, Michiels B, Kenward MG, Diggle PJ 1998 Stat Neerl 52:153 (pattern-mixture identifying restrictions; ACMV = MAR under monotone)
- Olarte Parra C, Bartlett JW, Daniel RM 2022 Stat Biopharm Res (MMRM-MAR IS a hypothetical estimator)
Decision Tree by Scenario
| Scenario | Recommended approach | Why |
|---|---|---|
| Continuous endpoint, monotone missingness, MAR plausible | MMRM with UN + Kenward-Roger via R mmrm |
FDA-favoured Mallinckrodt 2008/2014 default |
| Continuous endpoint, ICE = treatment discontinuation, treatment policy estimand | Hybrid: J2R for discontinuation ICEs, MMRM-MAR for other (Aprocitentan 2024 precedent) | De facto FDA standard 2024-2025 |
| Continuous endpoint, treatment policy + post-ICE data collected | Retrieved-dropout MI (Wegovy STEP precedent) | Empirically grounded; FDA 2025 obesity guidance endorses |
| Continuous endpoint, MNAR sensitivity required | J2R via rbmi with both Rubin's variance AND frequentist (CMI+jackknife) |
Cro vs Bartlett debate; report both for safety |
| Continuous endpoint, tipping-point analysis | Permutt 2016 delta-adjustment in active arm only | Direct regulatory question; pre-specify delta range |
| Binary endpoint with missing primary | MI with logistic imputation; modified Poisson for marginal RR | Per FDA 2023 covariate adjustment |
| Time-to-event with informative censoring | IPCW (Robins) or sensitivity under composite | Censoring-as-event composite; cite Lewis 2023 |
| Very high missingness (>40%) | Report as hypothesis-generating; multiple sensitivity analyses | NRC 2010 caveat |
| Aducanumab-style differential dropout pattern | MAR primary is questionable; treatment-policy with reference-based MI primary | Lessons from FDA 6-1 AdCom decision (2021) |
| Pediatric trial with hard-to-retain population | Retrieved-dropout MI + Bayesian extrapolation from adult data | FDA 2025 obesity pediatric extension guidance |
MMRM Under MAR -- The FDA-Preferred Continuous Analysis
Goal: Estimate the treatment-by-visit contrast at the primary timepoint for a continuous longitudinal endpoint under MAR with valid Type-I control in small/moderate trials.
Approach: Fit MMRM with unstructured covariance, REML, Kenward-Roger DF correction via the Roche/openpharma mmrm package; pre-specify the convergence fallback hierarchy in the SAP.
Mallinckrodt 2008/2014: for continuous longitudinal endpoints under monotone (or near-monotone) MAR, an MMRM with treatment + visit + treatment-by-visit + baseline + baseline-by-visit, UN covariance, REML, contrast at primary timepoint -- is consistent and FDA-preferred.
library(mmrm)
fit <- mmrm(
formula = change_from_baseline ~ baseline + arm * visit + us(visit | subject),
data = trial_data,
method = "Kenward-Roger-Linear", # matches SAS PROC MIXED bit-for-bit
reml = TRUE
)
summary(fit)
Kenward-Roger flavour question:
method = "Kenward-Roger"-- full second-order KR (Kenward-Roger 1997 Biometrics 53:983)method = "Kenward-Roger-Linear"-- drops second-order Cholesky-derivative term; matches SAS PROC MIXED bit-for-bit- Most regulatory submissions use Kenward-Roger-Linear for SAS-R reproducibility
Convergence fallback hierarchy (pre-specify in SAP):
- UN with KR (preferred)
- UN with Satterthwaite (if KR fails)
- Heterogeneous Toeplitz (k+1 parameters)
- AR(1) with heterogeneous variances
- CS with heterogeneous variances (last resort)
The Olarte Parra unification (2022): MMRM under MAR IS a causal hypothetical estimand by g-formula equivalence under specific identifying assumptions. The issue is articulation, not statistical machinery.
Reference-Based Multiple Imputation -- The rbmi Framework
Carpenter-Roger-Kenward 2013 operationalises MNAR sensitivity as clinical narrative, not numeric delta:
- J2R: "after withdrawal the patient instantly resembles the placebo arm"
- CR: "the entire post-baseline trajectory copied from placebo, with subject's baseline deviation preserved"
- CIR: "the patient retains on-treatment increment but trends with placebo arm thereafter, anchored at last observed value"
- LMCF: "patient stays at last on-treatment mean"
library(rbmi)
# Define imputation model
vars <- set_vars(
outcome = 'CHG', # change from baseline
visit = 'AVISIT',
subjid = 'USUBJID',
group = 'ARM',
covariates = c('BASE', 'STRATA1'),
method = method_bayes(n_samples = 100)
)
# Draws -> Impute -> Analyse -> Pool pipeline
draws_obj <- draws(data = trial_data, vars = vars)
imputed_j2r <- impute(draws_obj,
references = c('Active' = 'Placebo', 'Placebo' = 'Placebo'))
analyses <- analyse(imputed_j2r,
fun = ancova,
vars = list(outcome = 'CHG', visit = 'AVISIT',
group = 'ARM', covariates = c('BASE')))
result <- pool(analyses) # Rubin's rules pooling
summary(result)
Inference engine choice (rbmi)
- Bayesian MI + Rubin's rules (historical default) -- information-anchored variance
- Approximate Bayesian via REML + bootstrap -- frequentist variance
- Conditional mean imputation + jackknife (Wolbers 2022) -- deterministic; FDA-friendly
- BMLMI (Lipkovich-Ratitch) -- bootstrapped MI with within/between decomposition
The Variance Debate -- Cro vs Bartlett
The single most active argument in current biostatistics.
Cro/Carpenter/Kenward 2019 JRSS-A 182:623: Rubin's-rules variance applied to J2R/CR/CIR is approximately information-anchored — the relative loss of information from missingness in sensitivity analysis matches the relative loss in MAR primary analysis. True repeated-sampling variance is "information positive" because reference-based MI borrows from reference arm, reducing marginal variance of active arm BELOW what an MAR analysis with same missingness would give.
Philosophical position (Cro et al): a sensitivity analysis should not import information the primary analysis didn't have; if borrowing from placebo makes active CI narrower, it's no longer anchored.
The clearer framing for postdocs: the dispute is NOT about "which variance is conservative" — it is about "which variance answers the right question." Cro: if the sensitivity analysis is meant to assess robustness to MAR, the variance should be the one that matches the informational scope of the primary MAR analysis (Rubin's, which under-states borrowing). Bartlett: if J2R is taken as the true data-generating mechanism, then the variance of inference under that mechanism is the frequentist (jackknife) variance, which delivers nominal Type-I. Both can be correct under different framings of "what is the sensitivity analysis for?"
Bartlett 2021 Stat Biopharm Res 15(1):178 + Wolbers 2022 Pharm Stat counter: if J2R is the actual sampling model under which inference is made, the correct frequentist variance is the one delivering nominal Type-I and coverage — the jackknife/bootstrap variance, NOT Rubin's. Simulations: Bayesian MI + Rubin's gives Type-I 0.9-2.5% (over-conservative); CMI+jackknife gives 4.84-4.96% (nominal) under J2R.
Regulatory practice 2024-2026:
- EMA tolerates either
- FDA reviewers increasingly flag Rubin's variance under reference-based MI as needing a frequentist sensitivity analysis in addition
- Safe approach: report both — Bayesian MI + Rubin's as primary (per Cro), CMI+jackknife frequentist as supportive (per Wolbers)
Permutt Tipping-Point Analysis
Permutt 2016 Stat Med 35:2876 (Permutt was head of FDA Division of Biometrics IV at the time): the regulator's question is not "what is a reasonable MNAR adjustment?" but "how bad would the missing data have to be in the active arm to overturn the significant primary result?"
library(rbmi)
# Delta-template: per-visit, per-arm, per-pattern delta
delta_grid <- seq(0, 20, by = 2) # delta range to scan
results <- list()
for (delta in delta_grid) {
delta_template <- delta_template(imputed, delta = delta,
dlag = c(1, 1, 1, 1)) # apply to all post-ICE visits
analyses <- analyse(imputed, delta = delta_template, fun = ancova, vars = vars)
pooled <- pool(analyses)
results[[as.character(delta)]] <- pooled
}
# Find minimum delta that flips p > 0.05 -> tipping delta
Delta-adjustment patterns:
- One-arm shift (FDA preferred): add delta to imputed values in active arm only
- Symmetric shift: both arms worsened by delta (probes systematic optimism)
- Reverse shift: placebo improved by delta (more aggressive)
Report tipping delta in residual SD units (FDA preference for cross-trial comparison), not raw outcome units.
Pattern-Mixture and Selection Models
Pattern-mixture identifying restrictions (Little 1993, Molenberghs et al 1998)
Pattern-mixture factorises joint distribution as observed-data distribution stratified by dropout pattern × pattern probability. Introduces unidentifiable parameters for unobserved cells, resolved by restrictions:
- CCMV (Complete-Case Missing-Value): equates missing conditional to completer pattern's conditional
- NCMV (Neighbouring-Case Missing-Value): uses conditional of patients with one additional measurement
- ACMV (Available-Case Missing-Value): weights across all patterns where relevant components observed
Molenberghs et al 1998 proved ACMV is exactly equivalent to MAR under monotone missingness — so pattern-mixture under ACMV is a reparameterisation of MAR analysis.
J2R/CR/CIR/LMCF are pattern-mixture models with reference-arm-based identifying restrictions.
Selection model (Diggle-Kenward 1994)
Joints a multivariate normal response model with logistic dropout model that depends on the unobserved current value.
Canonical critique (Diggle-Kenward 1994 discussion; Molenberghs/Kenward/Verbeke subsequent work): selection-model MNAR conclusions are driven not by data but by parametric assumptions that are empirically untestable — the difference between MAR and MNAR fit is identified entirely from joint normality assumption. A non-normal response will spuriously appear MNAR.
FDA position: selection models are used as sensitivity, never primary. FDA Division of Biometrics has repeatedly pushed back on selection models on the grounds that "the sponsor cannot tell me in clinical English what assumption I am being asked to accept." Reference-based MI is preferred because it is clinically articulable.
Multiple Imputation in Python -- The sklearn Caveats
from sklearn.experimental import enable_iterative_imputer
from sklearn.impute import IterativeImputer
import statsmodels.formula.api as smf
import numpy as np
import pandas as pd
n_imputations = 20 # rule: m >= 100 * FMI
# CRITICAL: sample_posterior=True only works with BayesianRidge (default estimator)
imputer = IterativeImputer(max_iter=10, random_state=0, sample_posterior=True)
results = []
for i in range(n_imputations):
imputer.set_params(random_state=i)
imputed = pd.DataFrame(imputer.fit_transform(df[numeric_cols]), columns=numeric_cols)
for col in ['ARM', 'sex']:
imputed[col] = df[col].values
model = smf.logit('outcome ~ C(ARM, Treatment(reference="Placebo")) + age', data=imputed).fit(disp=0)
results.append({'coef': model.params.iloc[1], 'se': model.bse.iloc[1]})
# Rubin's rules
pooled_coef = np.mean([r['coef'] for r in results])
within_var = np.mean([r['se']**2 for r in results])
between_var = np.var([r['coef'] for r in results], ddof=1)
total_var = within_var + (1 + 1/n_imputations) * between_var
pooled_se = np.sqrt(total_var)
Critical caveats:
sample_posterior=TrueSILENTLY ignored if estimator changed from BayesianRidge (e.g., to RandomForest) -> MI degenerates to single imputation- IterativeImputer is experimental; API may change without standard deprecation
- For mixed types (binary + continuous), consider
miceforestor Rmice/rbmi - Imputation model must include all analysis-model predictors (congeniality per Meng 1994)
- Include outcome as predictor in imputation model but exclude from imputed variables
- Never impute treatment assignment (fully determined by randomisation)
For confirmatory regulatory work, prefer R rbmi or mice over Python sklearn — the SAS / R precedent is stronger, the variance theory is better-developed.
Decisive Regulatory Cases
Aducanumab (Biogen BLA 761178, 2021)
EMERGE and ENGAGE both stopped early for futility; EMERGE high-dose positive, ENGAGE negative. MMRM-MAR primary. FDA Office of Biostatistics (Tristan Massie review) argued futility-stop-induced missingness was NOT MAR (differential ARIA-driven unblinding). 6-1 AdCom against approval; subsequent approval over-ruled OB. Textbook case showing MAR-based primary in trial with high differential missingness is regulator-divisive.
Lesson: when differential dropout patterns differ by arm in clinically meaningful ways, MAR is questionable; primary should be treatment-policy with reference-based MI.
Aprocitentan (Idorsia PRECISION, FDA 2024)
Mathur 2025 Pharm Stat (PMC12753554) documents the negotiation. FDA pushed back on MMRM-MAR primary; accepted compromise: stratified imputation — J2R for treatment-discontinuation ICEs, MAR-MMRM for other missingness.
Lesson: this hybrid is now the de facto FDA standard for treatment-policy estimands. Pre-specify in SAP.
Wegovy/Ozempic STEP trials (Wilding 2021 NEJM)
Retrieved-dropout MI as primary for treatment-policy estimand. Missing body weight at week 68 imputed by sampling from observed week-68 measurements among "retrieved dropouts" (patients who discontinued semaglutide but remained in follow-up). J2R-MI as supportive.
Lesson: RD-MI is now standard for chronic weight management. FDA 2025 obesity guidance explicitly endorses MI as primary.
Per-Method Failure Modes
MMRM-MAR with high differential missingness
- Trigger: Dropout rate or reason differs by arm (toxic active vs tolerated placebo).
- Mechanism: MAR requires that conditional on observed covariates, missingness is unrelated to outcome — implausible when patients drop out because the treatment isn't working.
- Symptom: Discontinuation reasons differ qualitatively; primary p sensitive to model specification.
- Fix: Treatment-policy estimand with retrieved-dropout MI as primary; J2R as sensitivity; tipping-point delta in active arm.
Rubin's variance under reference-based MI
- Trigger: J2R/CR/CIR with only Rubin's-rules variance reported.
- Mechanism: Rubin's variance is information-anchored but over-conservative (Cro 2019).
- Symptom: Power loss vs frequentist variance.
- Fix: Report frequentist variance via CMI+jackknife as supportive (Wolbers 2022); cite both Cro and Bartlett.
sample_posterior=False in sklearn IterativeImputer
- Trigger: Default behaviour ignored OR estimator changed from BayesianRidge.
- Mechanism: Imputations are point predictions, near-identical across draws; between-imputation variance ~ 0.
- Symptom: Artificially narrow Rubin's-pooled CIs.
- Fix: Always
sample_posterior=True; verify estimator is BayesianRidge; consider Rrbmi.
Diggle-Kenward selection model as primary
- Trigger: Selection model used as primary MNAR analysis.
- Mechanism: MNAR vs MAR distinction driven by joint normality assumption, not data.
- Symptom: Reviewer asks "what clinical assumption am I being asked to accept?" and sponsor cannot answer in clinical English.
- Fix: Switch to pattern-mixture (reference-based MI) for primary; selection model as supportive sensitivity only.
LOCF as primary or "conservative"
- Trigger: SAP specifies LOCF as primary analysis or as "conservative" sensitivity.
- Mechanism: LOCF is biased even under MCAR (Mallinckrodt 2008); discards uncertainty.
- Symptom: Reviewer cites NRC 2010 Rec 11 against LOCF.
- Fix: MMRM under MAR as primary; reference-based MI for MNAR sensitivity; LOCF only as historical comparison if at all.
Imputation model uncongenial with analysis model
- Trigger: Analysis model includes treatment-by-covariate interaction but imputation model does not.
- Mechanism: Uncongeniality biases estimates and invalidates Rubin's variance pooling.
- Symptom: Pooled SE smaller than independent bootstrap suggests.
- Fix: Imputation model must be at least as flexible as analysis model (Meng 1994).
Tipping delta in raw units only
- Trigger: Tipping-point analysis reports delta in raw outcome scale.
- Mechanism: Hard to compare across trials; clinical plausibility unclear.
- Symptom: Reviewer asks "how many SDs is that?"
- Fix: Report in residual SD units (FDA preference); both raw and standardised.
Reconciliation: When Methods Disagree
| Pattern | Likely cause | Action |
|---|---|---|
| MMRM-MAR CI overlaps null; J2R MI CI excludes null | Reference-based MI borrows information from reference arm, reducing active-arm marginal variance below what MAR analysis with same missingness would give (Cro 2019) | Both valid under their assumptions; choose by clinical plausibility (MAR vs MNAR after differential dropout); report both; cite estimand strategy |
| Bayesian MI + Rubin's variance Type-I ~1%; CMI+jackknife Type-I ~5% under J2R | Rubin's information-anchored, over-conservative (Cro 2019); jackknife frequentist nominal (Wolbers 2022); active methodological debate | Report both; flag as Cro vs Bartlett debate; FDA increasingly accepts jackknife as supportive |
| LOCF "conservative" sensitivity gives smaller effect than MMRM-MAR | LOCF assumes flat post-ICE trajectory; biased even under MCAR (Mallinckrodt 2008) | Replace LOCF with reference-based MI; cite NRC 2010 Rec 11; reframe sensitivity as "MNAR robustness" not "conservative" |
| Tipping delta is small relative to MCID | MAR plausibility weak; primary result fragile to mild MNAR | Report tipping delta in residual SD AND relative to MCID; reconsider primary estimand toward treatment-policy with reference-based MI |
| Pattern-mixture with CCMV vs J2R conclusions differ | CCMV assumes missing pattern mirrors completers; J2R assumes mirror reference arm; different MNAR mechanisms | Choose based on which clinical scenario is more plausible (Carpenter-Roger 2013); report both as sensitivity range |
| Selection model (Diggle-Kenward) rejects MAR; pattern-mixture (reference-based MI) accepts MAR | Selection model MNAR conclusion driven by joint normality assumption (not data); pattern-mixture clinically articulable | Report pattern-mixture as primary sensitivity; selection model as supportive only; FDA prefers clinical articulation |
| MMRM with UN converges in arm A but not arm B | Convergence fragility in unstructured covariance with high dropout in one arm | Apply pre-specified SAP fallback (UN+KR -> UN+Satterthwaite -> heterogeneous Toeplitz -> AR(1)); document in CSR |
Quantitative Thresholds
| Threshold | Source | Rationale |
|---|---|---|
| m >= 100 * FMI imputations | von Hippel 2020 Sociol Methods Res 49:699 | Stable pooled SE; with 40% missingness and FMI~0.3, m=30 needed |
| Missing > 40% on key variable | NRC 2010 | MI under MAR unreliable; treat as hypothesis-generating |
| Kenward-Roger DF correction for MMRM with UN | Kenward-Roger 1997 | Without it MMRM-REML under-covers; Type-I inflates 1-2 pp |
| Tipping delta in residual SD units | FDA Division of Biometrics preference | Cross-trial comparison |
| Rubin's vs frequentist for reference-based MI | Cro 2019 vs Wolbers 2022 | Report both for regulatory safety |
| Pre-specify ICE strategy in SAP | ICH E9(R1) | Estimand-before-method; cite Kahan 2023 |
| Examine DS domain for differential dropout | NRC 2010 implicit | If dropout differs by arm, MAR is suspect |
Common Errors
| Error / symptom | Cause | Solution |
|---|---|---|
| LOCF reported as "conservative" | Persistent misconception | LOCF is biased under MCAR (Mallinckrodt 2008); cite NRC 2010 |
| MAR primary in trial with differential dropout | MAR plausibility not verified | Examine DS; if differential, switch to treatment-policy + reference-based MI |
| Selection model used as primary | Untestable parametric assumption drives MNAR conclusion | Pattern-mixture (reference-based MI) as primary; selection model as supportive |
| sample_posterior silently ignored | Estimator changed from BayesianRidge in sklearn | Verify estimator; consider R rbmi or mice |
| Imputation model missing interaction term | Uncongeniality with analysis model | Include analysis-model predictors AND interactions |
| Tipping delta in raw outcome scale only | Hard to compare | Report in residual SD units alongside raw |
| Rubin's variance for J2R without frequentist sensitivity | Cro 2019 over-conservative | CMI+jackknife frequentist as supportive (Wolbers 2022) |
| MMRM with CS forced after UN convergence failure | Pre-specification of fallback missing | Pre-specify hierarchy in SAP; document deviation if invoked |
| MI without outcome as predictor in imputation model | Underestimates association | Include outcome as predictor; exclude from imputed variables |
| MMRM in Python (statsmodels.mixedlm) treated as FDA-equivalent | Lacks Kenward-Roger | Use R mmrm for confirmatory; statsmodels.mixedlm only for exploratory |
Anticipated Reviewer Pushback
| Pushback | Response |
|---|---|
| "Why MAR not MNAR primary?" | Examined DS domain; dropout rates and reasons symmetric across arms; cite Mallinckrodt 2008 MMRM-MAR convention |
| "Why not LOCF as sensitivity?" | LOCF biased even under MCAR; cite NRC 2010 Rec 11; J2R and tipping-point are clinically articulable alternatives |
| "Rubin's variance for J2R?" | Cite Cro 2019 information-anchored argument as rationale; report frequentist (CMI+jackknife) as supportive per Wolbers 2022 |
| "Tipping delta plausibility?" | Tipping delta = X.X in residual SD units; X.X exceeds MCID of Y.Y; deemed clinically implausible |
| "Selection model sensitivity?" | Provided as supportive; cite Diggle-Kenward 1994 critique that conclusions are driven by joint normality assumption; pattern-mixture is the primary sensitivity |
| "Imputation model congeniality?" | Imputation model includes all analysis-model predictors and interactions per Meng 1994 |
| "ICE strategy?" | Pre-specified per ICH E9(R1): treatment policy primary, hypothetical sensitivity, with explicit ICE mechanism in protocol |
| "Why m=20 imputations?" | Computed m >= 100 * FMI; observed FMI=0.20 -> m=20 sufficient (von Hippel 2020) |
| "Retrieved-dropout MI with sparse post-ICE data?" | If post-ICE retrieval is <50% complete, retrieved-dropout MI degrades to reference-based; switch to J2R as primary with reference-based MI as named sensitivity per Aprocitentan precedent |
| "Why is the reference arm chosen for J2R clinically plausible?" | Per ICH E9(R1), reference-based MI assumes post-discontinuation trajectory mirrors the placebo arm; this matches the clinical scenario of "patient stops treatment due to AE and returns to standard-of-care baseline." Documented in protocol with medical-monitor sign-off. |
References
- Bartlett JW. 2021. Reference-based multiple imputation -- what is the right variance and how to estimate it. Stat Biopharm Res 15(1):178-186.
- Carpenter JR, Roger JH, Kenward MG. 2013. Analysis of longitudinal trials with protocol deviation: a framework for relevant, accessible assumptions, and inference via multiple imputation. J Biopharm Stat 23:1352-1371.
- Cro S, Carpenter JR, Kenward MG. 2019. Information-anchored sensitivity analysis: theory and application. JRSS-A 182:623-645.
- Diggle PJ, Kenward MG. 1994. Informative drop-out in longitudinal data analysis. JRSS-C 43:49-93.
- EMA. 2010. Guideline on Missing Data in Confirmatory Clinical Trials. EMA/CPMP/EWP/1776/99 Rev.1.
- ICH. 2019. E9(R1) Addendum on Estimands and Sensitivity Analysis.
- Kenward MG, Roger JH. 1997. Small sample inference for fixed effects from REML. Biometrics 53:983-997.
- Little RJA. 1988. A test of missing completely at random for multivariate data with missing values. JASA 83:1198-1202.
- Little RJA. 1993. Pattern-mixture models for multivariate incomplete data. JASA 88:125-134.
- Little RJA, D'Agostino R, Cohen ML et al. 2012. The prevention and treatment of missing data in clinical trials. NEJM 367:1355-1360.
- Mallinckrodt CH, Lane PW, Schnell D, Peng Y, Mancuso JP. 2008. Recommendations for the primary analysis of continuous endpoints in longitudinal clinical trials. Drug Information Journal 42:303-319.
- Mathur AR et al 2025. Regulatory experiences with multiple imputation. Pharm Stat.
- Meng XL. 1994. Multiple-imputation inferences with uncongenial sources of input. Stat Sci 9:538-558.
- Molenberghs G, Michiels B, Kenward MG, Diggle PJ. 1998. Monotone missing data and pattern-mixture models. Stat Neerl 52:153-161.
- NRC. 2010. The Prevention and Treatment of Missing Data in Clinical Trials. National Academies Press.
- Olarte Parra C, Bartlett JW, Daniel RM. 2022. Hypothetical estimands: a unification of causal inference and missing data methods. Stat Biopharm Res.
- Permutt T. 2016. Sensitivity analysis for missing data in regulatory submissions. Stat Med 35:2876-2879.
- von Hippel PT. 2020. How many imputations are needed (a two-stage calculation using a quadratic rule). Sociol Methods Res 49:699-718.
- Wolbers M, Noci A, Delmar P, Gower-Page C, Yiu S, Bartlett JW. 2022. Standard and reference-based conditional mean imputation. Pharm Stat 21:1262-1281.
Related Skills
- clinical-biostatistics/trial-reporting - Estimand framework + CONSORT 2025 missing-data item 21c
- clinical-biostatistics/logistic-regression - Adjusted logistic with MI for binary endpoints
- clinical-biostatistics/effect-measures - Effect-measure CIs under MI pooling
- clinical-biostatistics/cdisc-data-handling - DS domain reasoning for missingness mechanism
- clinical-biostatistics/survival-analysis - Informative censoring as missing-data analogue
- clinical-biostatistics/adaptive-designs - Interim missing-data assumptions