Bio Clinical Biostatistics Survival Analysis

Version Compatibility

Reference examples tested with: lifelines 0.27+, scikit-survival 0.21+, statsmodels 0.14+, pandas 2.1+, numpy 1.26+. R packages cited (still the SOTA for survival): survival 3.8+, survRM2, cmprsk, riskRegression, mstate, flexsurv, icenReg, rpsftm.

Before using code patterns, verify installed versions match. If versions differ:

• Python: pip show <package> then help(module.function) to check signatures
• R: packageVersion('<pkg>') then ?function_name

If code throws ImportError, AttributeError, or TypeError, introspect the installed package and adapt the example to match the actual API rather than retrying.

Time-to-Event Analysis for Clinical Trials

"Analyze time-to-event endpoint" -> Estimate a hazard, survival probability, cumulative incidence, or restricted mean time using a method calibrated to (a) whether proportional hazards holds, (b) whether competing events exist, (c) whether censoring is informative, and (d) which estimand the trial targets under ICH E9(R1).

The Single Most Important Modern Insight -- PH Almost Never Holds

In modern oncology with checkpoint inhibitors, targeted therapies, crossover, and depleted high-risk subjects over follow-up, proportional hazards (PH) violations are the rule, not the exception. The Cross-Pharma NPH Working Group (Lin et al 2020 Stat Biopharm Res; Magirr-Burman 2021 Stat Biopharm Res 15(2):293) documented systematic PH violations across phase III oncology trials, particularly delayed-effect patterns from checkpoint inhibitors.

The Cox HR is a time-averaged log-hazard ratio under PH violation (Xu-O'Quigley 2000), which may or may not be the estimand of interest. RMST (Royston-Parmar 2013) provides a clinically interpretable, hazard-free alternative.

Algorithmic Taxonomy

Postdoc reading list:

• Royston-Parmar 2013 BMC Med Res Methodol 13:152 (RMST as primary)
• Uno et al 2014 JCO 32:2380 (RMST in oncology)
• Andersen-Keiding 2012 Stat Med 31:1074 (Fine-Gray semantic critique)
• Putter-Schumacher-van Houwelingen 2020 Biom J 62:790 (Fine-Gray revisited; reduction factor)
• Putter-Fiocco-Geskus 2007 Stat Med 26:2389 (competing risks tutorial)
• Magirr-Burman 2021 Stat Biopharm Res 15(2):293 (MaxCombo critique)
• Grambsch-Therneau 1994 Biometrika 81:515 (scaled Schoenfeld residuals)
• Buyse-Molenberghs 1998 Biometrics 54:1014 (PFS-OS surrogacy framework)
• Lewis et al 2023 (ICH E9(R1) censoring rules and estimand)
• Sun 2006 The Statistical Analysis of Interval-Censored Failure Time Data (Springer)

Decision Tree by Scenario

Cox PH Diagnostics -- The Therneau-Grambsch Test (and Its Pitfalls)

Goal: Detect violations of the proportional hazards assumption that would invalidate the Cox HR as a meaningful summary statistic.

Approach: Compute scaled Schoenfeld residuals (Grambsch-Therneau 1994); regress against a time-transform g(t) under H0 of zero slope; supplement the asymptotic p-value with a graphical residual plot since the test is sensitive to g(t) choice and sample-size dependent.

from lifelines import CoxPHFitter
from lifelines.statistics import proportional_hazard_test

cph = CoxPHFitter()
cph.fit(df, duration_col='time', event_col='event', formula='treatment + age + baseline_score')
cph.print_summary()

# Schoenfeld residuals PH test (lifelines)
results = proportional_hazard_test(cph, df, time_transform='rank')
print(results.summary)

# In R: cox.zph(coxph_fit) returns same with KM transform as default

The g(t) choice trap (Park-Hendry 2015 Political Analysis): the test power depends critically on g(t). KM transform, identity, log, and rank give materially different p-values. With n > 5000 even trivial deviations reject; with n < 100 the test misses meaningful violations.

Critical interpretation rule: a global p > 0.05 does NOT mean PH holds — it means the null cannot be rejected. The graphical diagnostic is more informative — a flat smoothed line is the target. Use cox.zph as a failure detector, not a PH validator.

# Plot scaled Schoenfeld residuals (lifelines)
cph.check_assumptions(df, p_value_threshold=0.05, show_plots=True)

Fixes when PH violated

1. Stratify on the violating covariate (loses inference on it)
2. Time-dependent coefficients: coxph(Surv(t, d) ~ x + tt(x), tt=function(x,t,...) x*log(t)) in R; lifelines: formula='treatment * time'
3. Royston-Parmar flexible parametric (R flexsurv::flexsurvspline or rstpm2::stpm2)
4. RMST as primary; HR as secondary

Restricted Mean Survival Time (RMST) -- The Modern Alternative

Royston-Parmar 2013 + Uno 2014 case: RMST(tau) = E[min(T, tau)] = integral from 0 to tau of S(t) dt = area under KM curve up to tau. The difference RMST_A(tau) - RMST_B(tau) is a time gained in months -- clinically interpretable, requires no PH assumption, always estimable up to the minimum of the largest follow-up across arms.

# Python: rmst via lifelines or manual
from lifelines.utils import restricted_mean_survival_time
from lifelines import KaplanMeierFitter

kmf_A = KaplanMeierFitter().fit(df[df['arm']=='A']['time'], event_observed=df[df['arm']=='A']['event'])
kmf_B = KaplanMeierFitter().fit(df[df['arm']=='B']['time'], event_observed=df[df['arm']=='B']['event'])
tau = 36  # months; pre-specified
rmst_A = restricted_mean_survival_time(kmf_A, t=tau)
rmst_B = restricted_mean_survival_time(kmf_B, t=tau)
print(f'RMST diff: {rmst_A - rmst_B:.2f} months')

# R: survRM2::rmst2(time, status, arm, tau=36) is the standard

The tau (truncation time) choice

• Statistical constraint: tau <= min(largest follow-up time in each arm) to avoid extrapolation (Tian et al 2020 Biostatistics gives rigorous treatment)
• Clinical constraint: tau should reflect a clinically meaningful horizon (5-year OS in adjuvant; 24-month PFS in metastatic)
• Data-dependent tau inflates Type-I error — MUST be pre-specified in SAP. Post-hoc tau tuning to chase significance is p-hacking.

Pseudo-observation regression (Andersen-Klein 2007)

For each subject i, compute jackknife pseudo-value θ_i(tau) = n·RMST(tau) - (n-1)·RMST_{-i}(tau). Regress pseudo-values on covariates via GEE — enables RMST regression WITH covariate adjustment, including time-varying covariates, no PH assumption.

R implementation: pseudo::pseudomean() + GEE via geepack::geeglm().

Lambert critique (postdoc reading): Paul Lambert (stpm3 author) is on record that RMST is "not unambiguously simpler than HR for clinicians." HR has 50-year pedagogy lead. RMST is cumulative (not instantaneous) and cannot detect crossing hazards beyond chosen tau. The argument for RMST is hazard-free interpretability under non-PH; the argument against is communication and tau sensitivity.

Competing Risks -- The Andersen-Keiding Framework

Fine-Gray subdistribution hazard

# Python: scikit-survival or lifelines (limited support); R is the SOTA
# R: library(cmprsk); crr(time, fstatus, cov, failcode=1, cencode=0)
# R: library(riskRegression); FGR(formula, data=df, cause=1)

Fine-Gray 1999 introduced subdistribution hazard lambda^FG(t) = -d/dt log[1 - F_1(t)] where F_1 is CIF for cause 1. The proportional subdistribution hazards model places covariates on this hazard. Subjects who experience competing events remain in the risk set with IPCW weighting — mathematically convenient but conceptually weird.

The Andersen-Keiding 2012 critique (Stat Med 31:1074): Fine-Gray subdistribution hazard violates the three principles for valid hazard functionals:

1. Hazard must be a real instantaneous risk among those truly at risk -> FG keeps dead competing-event subjects "at risk"
2. Covariate effects must be interpretable as causal -> FG coefficients confound the competing-event hazard
3. Hazard must support landmarking (conditioning on survival to s) -> FG does not

The competing-risk confounding trap (Putter-Schumacher-van Houwelingen 2020 Biom J 62:790; the reduction factor decomposition makes it explicit): any covariate that increases the cause-specific hazard of competing event A will decrease the subdistribution hazard for event B simply because A removes subjects from the population at risk of B. A Fine-Gray "protective" effect on B may be an iatrogenic killer via A.

Practical rule (Putter-Fiocco-Geskus 2007)

• For prediction of CIF (cumulative incidence) -> Fine-Gray is fine; report CIF curves and SHR
• For etiology / causal effect on the event -> use cause-specific Cox (treat competing events as censoring); report cause-specific HRs for BOTH event of interest AND competing events
• For multi-state semantics (alive -> relapse -> death) -> use multi-state framework with transition-specific Cox models

The CIF is always estimable

Aalen-Johansen estimator (multi-state generalisation of KM) gives the CIF non-parametrically. The KM estimator is biased upward in the presence of competing risks — it treats competing events as non-informative censoring and overestimates 1 - CIF.

# R: library(mstate); msfit(coxph_fit, newdata, trans); probtrans(msfit_obj)
# R: library(survival); survfit(Surv(time, event_factor) ~ 1) with multi-state Surv

Log-Rank Variants and MaxCombo

from lifelines.statistics import logrank_test, multivariate_logrank_test

# Stratified log-rank (lifelines uses `weightings=` -- note plural)
results = multivariate_logrank_test(
    df['time'], df['arm'], df['event'], weightings='peto'
)
# weightings options (lifelines 0.27+): None, 'wilcoxon', 'tarone-ware',
# 'peto', 'fleming-harrington' (also accepts kwargs for Fleming-Harrington p, q)

The MaxCombo controversy

Magirr-Burman 2021 Stat Biopharm Res 15(2):293: MaxCombo can reject the null in opposite directions on the same dataset. Formally, it rejects the strong null H_0: S_A(t) = S_B(t) for all t, but the rejection direction is determined by the dominant weight, which can flip across portions of the curve. KEYNOTE-042 demonstration: MaxCombo simultaneously favouring pembrolizumab AND chemo depending on weight choice.

Cross-Pharma NPH Working Group recommendation: MaxCombo with directionality constraints — require positive z-statistic at the late-emphasis weight before declaring superiority; report the dominant weight and its direction.

Interval Censoring -- When Standard Right-Censoring Is Wrong

PFS is REALLY interval-censored — events known only between consecutive RECIST scans. The convention is to treat it as right-censored at midpoint or first-PD-scan date. This is a long-standing methodological compromise defensible only when scan intervals are short and balanced across arms.

# R: library(icenReg)
# fit_ic <- ic_par(cbind(left, right) ~ treatment + age, data=df, dist='weibull', model='ph')
# ic_sp() for semi-parametric; ic_npar() for NPMLE

Sun 2006 The Statistical Analysis of Interval-Censored Failure Time Data (Springer) is the canonical reference. NPMLE via Turnbull's algorithm; Cox-like regression via Finkelstein 1986 / Pan 1999 EM algorithms.

When to switch from right-censored midpoint to interval-censored: scan intervals > 4 weeks; scan timing differs between arms (open-label trials with potential differential ascertainment); regulatory submission where 2-3% effect-size shift matters.

Recurrent Events -- AG vs PWP vs WLW

# Python: lifelines does not have native AG; use coxph with (start, stop, event) format
# R: coxph(Surv(start, stop, event) ~ trt + cluster(id), data=long_df) for AG
# R: + strata(enum) for PWP

Box-Steffensmeier critique: WLW is often misused — fitting separate Cox to "time to 2nd event" treats it as a first-event problem rather than conditional on prior history, inflating effect estimates. PWP-gap-time is the cleanest conditional model. AG most efficient when exchangeability holds.

Modern advice (Rogers et al 2016; Cook-Lawless 2007 book): AG with robust variance as default; PWP if events qualitatively heterogeneous; avoid WLW unless events are truly distinct types.

Oncology PFS/OS Estimands -- The 2024 Reality

Per ICH E9(R1), the same PFS dataset yields different HRs depending on censoring rules. Each rule corresponds to a different intercurrent-event strategy:

The 2024 European Journal of Cancer demonstration (PMID 38547775): two sets of censoring rules — FDA-favoured vs trialist-favoured — applied to the same PFS data shifted median PFS from 32 to 43 months in the experimental arm with no change in control. This is the estimand changing, not analytic artefact.

Fleming 2025 argument: PFS should be defined as treatment-policy with progression + death + subsequent therapy as a composite ("treatment failure-free survival") to preserve ITT. Controversial because it loses interpretation as "tumor growth control."

Informative censoring -- detection and handling

The mechanism: standard right-censored Cox / KM assumes censoring is non-informative (the censoring process is unrelated to the underlying failure time after conditioning on observed covariates). When this fails, KM is biased upward in the arm with informative censoring -- patients who drop out due to lack of efficacy or toxicity are precisely the ones who would have failed early.

TROPiCS-02 2023 (Tolaney JCO): "evaporative cooling" of progression events. Patients on the toxic arm discontinue and are censored BEFORE the next protocol-mandated scan that would have captured progression; KM biases PFS upward in the toxic arm. Templeton 2020 Nat Rev Clin Oncol and Campigotto-Weller 2014 JCO are foundational citations.

Detection workflow:

1. Tabulate censoring reasons by arm from CDISC DS (Disposition) or ADaM ADTTE CNSR integer values:

   • CNSR=1 (lost to follow-up): symmetric across arms = OK
   • CNSR=2 (withdrew consent): if asymmetric, investigate why
   • CNSR=3 (admin EoS): symmetric by definition
   • CNSR=4 (subsequent therapy initiated): differential is the canonical informative pattern
   • CNSR=5+ (toxicity discontinuation): differential is highly informative

2. Compare KM curves for censoring distribution by arm. If "time-to-censoring" KM differs by arm in same direction as outcome, suspect informative censoring.

3. Cox-Snell or Schoenfeld residuals on censoring-as-event model: if treatment is significantly associated with censoring hazard, censoring is informative.

Handling strategies (per ICH E9(R1) and Lewis 2023):

IPCW workflow (stabilised weights per Robins-Finkelstein 2000):

# Stabilised IPCW: w(t) = S_num(t) / S_denom(t)
# where S_denom predicts remaining-uncensored given baseline + time-varying covariates,
# and S_num predicts remaining-uncensored given treatment alone (numerator for stabilisation)

# Step 1: fit censoring (NOT event) hazard models
# Note: outcome variable for these models is "1 if censored, 0 if event or still at risk"
denom_cens <- coxph(Surv(time, censored) ~ treatment + age + baseline_severity, data=df)
num_cens   <- coxph(Surv(time, censored) ~ treatment, data=df)

# Step 2: compute survival probabilities (NOT hazards) for "remaining uncensored"
# basehaz + linear predictor -> S(t) per subject; the `survfit` + `summary` API gives S_i(t_i)
S_denom <- summary(survfit(denom_cens, newdata=df), times=df$time)$surv
S_num   <- summary(survfit(num_cens,   newdata=df), times=df$time)$surv

# Step 3: stabilised IPCW weight per subject at their observed time
df$ipcw <- S_num / S_denom

# Step 4: weighted Cox for the event of interest with robust SE
fit_ipcw <- coxph(Surv(time, event) ~ treatment + age + baseline_severity,
                  data=df, weights=df$ipcw, robust=TRUE)

Note: in practice use the ipw::ipwtm (or ipwExt) wrapper which handles the time-varying weights and stabilisation correctly; the pseudocode above shows the underlying machinery. Verify weight distribution: median weight near 1, no extreme values (>10 indicates near-violation of positivity).

Operational rule: for any TTE primary analysis where DS shows differential censoring reasons by arm, the regulatory expectation is (1) tabulate reasons in CSR, (2) report sensitivity under IPCW or composite, (3) discuss whether primary HR/RMST changes under sensitivity.

ADaM ADTTE -- The CNSR Convention Trap

CDISC ADTTE convention: CNSR = 0 for events, positive integers for censoring reasons (1 = lost to follow-up, 2 = withdrew consent, 3 = admin EoS, 4 = subsequent therapy, etc.). Opposite to most stat packages which use 1 = event.

# Convert ADTTE for Python / R
import pandas as pd

adtte = pd.read_csv('ADTTE.csv')
adtte['event'] = (adtte['CNSR'] == 0).astype(int)  # 1 = event for survival packages
adtte['time'] = adtte['AVAL']  # AVAL is in days/months per AVALU

This is a perpetual bug source. See clinical-biostatistics/cdisc-data-handling for the full ADTTE specification.

Per-Method Failure Modes

Cox PH violation undetected

• Trigger: Significant treatment HR reported without cox.zph diagnostic
• Mechanism: Cox HR is a time-averaged log-HR under PH violation; the "the HR" interpretation breaks
• Symptom: Hazard plots show crossing; cox.zph rejects PH; KM curves diverge then converge
• Fix: Report cox.zph result; switch to RMST or time-varying Cox; cite Grambsch-Therneau 1994

Fine-Gray reported as causal effect

• Trigger: Fine-Gray SHR interpreted as "treatment effect on event of interest"
• Mechanism: SHR confounds with competing-event hazard (Andersen-Keiding 2012)
• Symptom: SHR interpretation contradicts cause-specific Cox results; reviewer confusion
• Fix: Use cause-specific Cox for etiology; Fine-Gray for CIF prediction only; report both per Putter-Fiocco-Geskus 2007

KM curve biased upward under competing risks

• Trigger: KM applied to event of interest treating competing events as non-informative censoring
• Mechanism: KM estimates 1 - cause-specific hazard cumulative; competing events deplete denominator
• Symptom: KM survival exceeds Aalen-Johansen 1 - CIF estimate
• Fix: Use Aalen-Johansen estimator for CIF; never KM in competing-risk setting

MaxCombo direction flipping

• Trigger: MaxCombo significant; direction not pre-specified
• Mechanism: MaxCombo's family can include weights that favour opposite directions
• Symptom: Different reports show opposite "winner" depending on weight choice
• Fix: Pre-specify directional constraints; require positive z at late-emphasis weight before claiming superiority; cite Magirr-Burman 2021

Tau chosen post-hoc to favour significance

• Trigger: RMST tau adjusted after seeing data
• Mechanism: Selection bias; tau-tuning is a flavour of p-hacking
• Symptom: Sponsor's RMST result differs from independently re-analysed with pre-specified tau
• Fix: Pre-specify tau in SAP; cite Tian 2020 Biostatistics

Right-censored midpoint for interval-censored PFS

• Trigger: Scan intervals long or differential between arms; standard right-censored Cox applied
• Mechanism: Midpoint imputation is biased; SE underestimates
• Symptom: Replication with interval-censored Cox gives different point estimate and wider CI
• Fix: R icenReg::ic_par or ic_sp; Sun 2006

CNSR convention confusion

• Trigger: ADTTE CNSR=1 passed to R survival::Surv(time, event) expecting event=1
• Mechanism: Censoring/event role reversed
• Symptom: "Event count" matches censoring count from CSR; nonsensical HR
• Fix: Always convert: event = (CNSR == 0).astype(int); cite ADaM IG v1.3

Reconciliation: When Methods Disagree

Quantitative Thresholds

Common Errors

Anticipated Reviewer Pushback

References

• Andersen PK, Keiding N. 2012. Interpretability and importance of functionals in competing risks and multistate models. Stat Med 31:1074-1088.
• Andersen PK, Klein JP. 2007. Regression analysis for the cumulative incidence functions. Scand J Stat 34:3-16.
• Buyse M, Molenberghs G. 1998. Criteria for the validation of surrogate endpoints in randomized experiments. Biometrics 54:1014-1029.
• Cox DR. 1972. Regression models and life-tables. JRSS-B 34:187-220.
• Fine JP, Gray RJ. 1999. A proportional hazards model for the subdistribution of a competing risk. JASA 94:496-509.
• Karrison TG. 2016. Versatile tests for comparing survival curves based on weighted log-rank statistics. Stat J 16:678-690.
• Lakatos E. 1988. Sample sizes based on the log-rank statistic in complex clinical trials. Biometrics 44:229-241.
• Lewis EF et al. 2023. Time-to-event endpoints in oncology under the ICH E9(R1) estimand framework. Pharm Stat (specific cite varies; see oncology estimand working-group publications).
• Magirr D, Burman CF. 2021. The strong null hypothesis and the MaxCombo test. Stat Biopharm Res 15(2):293-298. (Earlier "Cherry-picking in survival analysis" attribution was a blog post, Oct 2022.)
• Mantel N. 1966. Evaluation of survival data and two new rank order statistics arising in its consideration. Cancer Chemotherapy Reports 50:163-170.
• Peduzzi P et al. 1995. Importance of events per independent variable in proportional hazards analysis. J Clin Epidemiol 48:1503-1510.
• Putter H, Fiocco M, Geskus RB. 2007. Tutorial in biostatistics: competing risks and multi-state models. Stat Med 26:2389-2430.
• Putter H, Schumacher M, van Houwelingen HC. 2020. On the relation between the cause-specific hazard and the subdistribution rate for competing risks data: the Fine-Gray model revisited. Biom J 62:790-807.
• Royston P, Parmar MKB. 2013. Restricted mean survival time: an alternative to the hazard ratio. BMC Med Res Methodol 13:152.
• Schoenfeld DA. 1981. The asymptotic properties of nonparametric tests for comparing survival distributions. Biometrika 68:316-319.
• Sun J. 2006. The Statistical Analysis of Interval-Censored Failure Time Data. Springer.
• Grambsch PM, Therneau TM. 1994. Proportional hazards tests and diagnostics based on weighted residuals. Biometrika 81:515-526.
• Tian L et al. 2020. Empirical comparison of the restricted mean survival time. Biostatistics.
• Uno H et al. 2014. Moving beyond the hazard ratio in quantifying the between-group difference in survival analysis. JCO 32:2380-2385.
• Xu R, O'Quigley J. 2000. Estimating average regression effect under non-proportional hazards. Biostatistics 1:423-439.

Related Skills

• clinical-biostatistics/effect-measures - HR vs RMST as effect measures; CI methods
• clinical-biostatistics/trial-reporting - ICH E9(R1) estimand framework for time-to-event
• clinical-biostatistics/missing-data-sensitivity - Informative censoring and tipping-point for survival
• clinical-biostatistics/cdisc-data-handling - ADTTE structure and CNSR convention
• clinical-biostatistics/subgroup-analysis - Subgroup HTE for survival endpoints
• clinical-biostatistics/power-and-sample-size - Schoenfeld and Lakatos sample size for TTE
• clinical-biostatistics/multiplicity-graphical - Co-primary survival endpoints
• machine-learning/survival-analysis - Predictive survival models and ML extensions