Outcome Analysis and Statistical Methods Guide

Outcome Analysis and Statistical Methods Guide

Overview

Rigorous outcome analysis is essential for clinical decision support documents. This guide covers survival analysis, response assessment, statistical testing, and data visualization for patient cohort analyses and treatment evaluation.

Survival Analysis

Kaplan-Meier Method

Overview

• Non-parametric estimator of survival function from time-to-event data
• Handles censored observations (patients alive at last follow-up)
• Provides survival probability at each time point
• Generates characteristic step-function survival curves

Key Concepts

Censoring

• Right censoring: Most common - patient alive at last follow-up or study end
• Left censoring: Rare in clinical studies
• Interval censoring: Event occurred between two assessment times
• Informative vs non-informative: Censoring should be independent of outcome

Survival Function S(t)

• S(t) = Probability of surviving beyond time t
• S(0) = 1.0 (100% alive at time zero)
• S(t) decreases as time increases
• Step decreases at each event time

Median Survival

• Time point where S(t) = 0.50
• 50% of patients alive, 50% have had event
• Reported with 95% confidence interval
• "Not reached (NR)" if fewer than 50% events

Survival Rates at Fixed Time Points

• 1-year survival rate, 2-year survival rate, 5-year survival rate
• Read from K-M curve at specific time point
• Report with 95% CI: S(t) ± 1.96 × SE

Calculation Example

Time  Events  At Risk  Survival Probability
0     0       100      1.000
3     2       100      0.980 (98/100)
5     1       95       0.970 (97/100 × 95/98)
8     3       87       0.936 (94/100 × 92/95 × 84/87)
...

Log-Rank Test

Purpose: Compare survival curves between two or more groups

Null Hypothesis: No difference in survival distributions between groups

Test Statistic

• Compares observed vs expected events in each group at each time point
• Weights all time points equally
• Follows chi-square distribution with df = k-1 (k groups)

Reporting

• Chi-square statistic, degrees of freedom, p-value
• Example: χ² = 6.82, df = 1, p = 0.009
• Interpretation: Significant difference in survival curves

Assumptions

• Censoring is non-informative and independent
• Proportional hazards (constant HR over time)
• If non-proportional, consider time-varying effects

Alternatives for Non-Proportional Hazards

• Gehan-Breslow test: Weights early events more heavily
• Peto-Peto test: Modifies Gehan-Breslow weighting
• Restricted mean survival time (RMST): Difference in area under K-M curve

Cox Proportional Hazards Regression

Purpose: Multivariable survival analysis, estimate hazard ratios adjusting for covariates

Model: h(t|X) = h₀(t) × exp(β₁X₁ + β₂X₂ + ... + βₚXₚ)

• h(t|X): Hazard rate for individual with covariates X
• h₀(t): Baseline hazard function (unspecified)
• exp(β): Hazard ratio for one-unit change in covariate

Hazard Ratio Interpretation

• HR = 1.0: No effect
• HR > 1.0: Increased risk (harmful)
• HR < 1.0: Decreased risk (beneficial)
• HR = 0.50: 50% reduction in hazard (risk of event)

Example Output

Variable              HR      95% CI         p-value
Treatment (B vs A)    0.62    0.43-0.89      0.010
Age (per 10 years)    1.15    1.02-1.30      0.021
ECOG PS (2 vs 0-1)    1.85    1.21-2.83      0.004
Biomarker+ (vs -)     0.71    0.48-1.05      0.089

Proportional Hazards Assumption

• Hazard ratio constant over time
• Test: Schoenfeld residuals, log-minus-log plots
• Violation: Time-varying effects, consider stratification or time-dependent covariates

Multivariable vs Univariable

• Univariable: One covariate at a time, unadjusted HRs
• Multivariable: Multiple covariates simultaneously, adjusted HRs
• Report both: Univariable for all variables, multivariable for final model

Model Selection

• Forward selection: Start with empty model, add significant variables
• Backward elimination: Start with all variables, remove non-significant
• Clinical judgment: Include known prognostic factors regardless of p-value
• Parsimony: Avoid overfitting, rule of thumb 1 variable per 10-15 events

Response Assessment

RECIST v1.1 (Response Evaluation Criteria in Solid Tumors)

Target Lesions

• Select up to 5 lesions total (maximum 2 per organ)
• Measurable: ≥10 mm longest diameter (≥15 mm for lymph nodes short axis)
• Sum of longest diameters (SLD) at baseline

Response Categories

Complete Response (CR)

• Disappearance of all target and non-target lesions
• Lymph nodes must regress to <10 mm short axis
• Confirmation required at ≥4 weeks

Partial Response (PR)

• ≥30% decrease in SLD from baseline
• No new lesions or unequivocal progression of non-target lesions
• Confirmation required at ≥4 weeks

Stable Disease (SD)

• Neither PR nor PD criteria met
• Minimum duration typically 6-8 weeks from baseline

Progressive Disease (PD)

• ≥20% increase in SLD AND ≥5 mm absolute increase from smallest SLD (nadir)
• OR appearance of new lesions
• OR unequivocal progression of non-target lesions

Example Calculation

Baseline SLD: 80 mm (4 target lesions)
Week 6 SLD: 52 mm

Percent change: (52 - 80)/80 × 100% = -35%
Classification: Partial Response (≥30% decrease)

Week 12 SLD: 48 mm (nadir)
Week 18 SLD: 62 mm

Percent change from nadir: (62 - 48)/48 × 100% = +29%
Absolute change: 62 - 48 = 14 mm
Classification: Progressive Disease (>20% AND ≥5 mm increase)

iRECIST (Immune RECIST)

Purpose: Account for atypical response patterns with immunotherapy

Modifications from RECIST v1.1

iUPD (Immune Unconfirmed Progressive Disease)

• Initial increase in tumor burden or new lesions
• Requires confirmation at next assessment (≥4 weeks later)
• Continue treatment if clinically stable

iCPD (Immune Confirmed Progressive Disease)

• Confirmed progression at repeat imaging
• Discontinue immunotherapy

Pseudoprogression

• Initial apparent progression followed by response
• Mechanism: Immune cell infiltration increases tumor size
• Incidence: 5-10% of patients on immunotherapy
• Management: Continue treatment if patient clinically stable

New Lesions

• Record size and location but continue treatment
• Do not automatically classify as PD
• Confirm progression if new lesions grow or additional new lesions appear

Other Response Criteria

Lugano Classification (Lymphoma)

• PET-based: Deauville 5-point scale
  • Score 1-3: Negative (metabolic CR)
  • Score 4-5: Positive (residual disease)
• CT-based: If PET not available
• Bone marrow: Required for staging in some lymphomas

RANO (Response Assessment in Neuro-Oncology)

• Glioblastoma-specific: Accounts for pseudoprogression with radiation/temozolomide
• Enhancing disease: Bidimensional measurements (product of perpendicular diameters)
• Non-enhancing disease: FLAIR changes assessed separately
• Corticosteroid dose: Must document, increase may indicate progression

mRECIST (Modified RECIST for HCC)

• Viable tumor: Enhancing portion only (arterial phase enhancement)
• Necrosis: Non-enhancing areas excluded from measurements
• Application: Hepatocellular carcinoma with arterial enhancement

Outcome Metrics

Efficacy Endpoints

Overall Survival (OS)

• Definition: Time from randomization/treatment start to death from any cause
• Advantages: Objective, not subject to assessment bias, regulatory gold standard
• Disadvantages: Requires long follow-up, affected by subsequent therapies
• Censoring: Last known alive date
• Analysis: Kaplan-Meier, log-rank test, Cox regression

Progression-Free Survival (PFS)

• Definition: Time from randomization to progression (RECIST) or death
• Advantages: Earlier readout than OS, direct treatment effect
• Disadvantages: Requires regular imaging, subject to assessment timing
• Censoring: Last tumor assessment without progression
• Sensitivity Analysis: Assess impact of censoring assumptions

Objective Response Rate (ORR)

• Definition: Proportion of patients achieving CR or PR (best response)
• Denominator: Evaluable patients (baseline measurable disease)
• Reporting: Percentage with 95% CI (exact binomial method)
• Duration: Time from first response to progression (DOR)
• Advantage: Binary endpoint, no censoring complications

Disease Control Rate (DCR)

• Definition: CR + PR + SD (stable disease ≥6-8 weeks)
• Less Stringent: Captures clinical benefit beyond objective response
• Reporting: Percentage with 95% CI

Duration of Response (DOR)

• Definition: Time from first CR or PR to progression (among responders only)
• Population: Subset analysis of responders
• Analysis: Kaplan-Meier among responders
• Reporting: Median DOR with 95% CI

Time to Treatment Failure (TTF)

• Definition: Time from start to discontinuation for any reason (progression, toxicity, death, patient choice)
• Advantage: Reflects real-world treatment duration
• Components: PFS + toxicity-related discontinuations

Safety Endpoints

Adverse Events (CTCAE v5.0)

Grading

• Grade 1: Mild, asymptomatic or mild symptoms, clinical intervention not indicated
• Grade 2: Moderate, minimal/local intervention indicated, age-appropriate ADL limitation
• Grade 3: Severe or medically significant, not immediately life-threatening, hospitalization/prolongation indicated, disabling, self-care ADL limitation
• Grade 4: Life-threatening consequences, urgent intervention indicated
• Grade 5: Death related to adverse event

Reporting Standards

Adverse Event Summary Table:

AE Term (MedDRA)        Any Grade, n (%)  Grade 3-4, n (%)  Grade 5, n (%)
                        Trt A    Trt B    Trt A   Trt B     Trt A   Trt B
─────────────────────────────────────────────────────────────────────────
Hematologic
  Anemia                45 (90%) 42 (84%) 8 (16%) 6 (12%)   0       0
  Neutropenia           35 (70%) 38 (76%) 15 (30%) 18 (36%) 0       0
  Thrombocytopenia      28 (56%) 25 (50%) 6 (12%) 4 (8%)    0       0
  Febrile neutropenia   4 (8%)   6 (12%)  4 (8%)  6 (12%)   0       0

Gastrointestinal
  Nausea                42 (84%) 40 (80%) 2 (4%)  1 (2%)    0       0
  Diarrhea              31 (62%) 28 (56%) 5 (10%) 3 (6%)    0       0
  Mucositis             18 (36%) 15 (30%) 3 (6%)  2 (4%)    0       0

Any AE                  50 (100%) 50 (100%) 38 (76%) 35 (70%) 1 (2%) 0

Serious Adverse Events (SAEs)

• SAE incidence and type
• Relationship to treatment (related vs unrelated)
• Outcome (resolved, ongoing, fatal)
• Causality assessment (definite, probable, possible, unlikely, unrelated)

Treatment Modifications

• Dose reductions: n (%), reason
• Dose delays: n (%), duration
• Discontinuations: n (%), reason (toxicity vs progression vs other)
• Relative dose intensity: (actual dose delivered / planned dose) × 100%

Statistical Analysis Methods

Comparing Continuous Outcomes

Independent Samples t-test

• Application: Compare means between two independent groups (normally distributed)
• Assumptions: Normal distribution, equal variances (or use Welch's t-test)
• Reporting: Mean ± SD for each group, mean difference (95% CI), t-statistic, df, p-value
• Example: Mean age 62.3 ± 8.4 vs 58.7 ± 9.1 years, difference 3.6 years (95% CI 0.2-7.0, p=0.038)

Mann-Whitney U Test (Wilcoxon Rank-Sum)

• Application: Compare medians between two groups (non-normal distribution)
• Non-parametric: No distributional assumptions
• Reporting: Median [IQR] for each group, median difference, U-statistic, p-value
• Example: Median time to response 6.2 [4.1-8.3] vs 8.5 [5.9-11.2] weeks, p=0.042

ANOVA (Analysis of Variance)

• Application: Compare means across three or more groups
• Output: F-statistic, p-value (overall test)
• Post-hoc: If significant, pairwise comparisons with Tukey or Bonferroni correction
• Example: Treatment effect varied by biomarker subgroup (F=4.32, df=2, p=0.016)

Comparing Categorical Outcomes

Chi-Square Test for Independence

• Application: Compare proportions between two or more groups
• Assumptions: Expected count ≥5 in at least 80% of cells
• Reporting: n (%) for each cell, χ², df, p-value
• Example: ORR 45% vs 30%, χ²=6.21, df=1, p=0.013

Fisher's Exact Test

• Application: 2×2 tables when expected count <5
• Exact p-value: No large-sample approximation
• Two-sided vs one-sided: Typically report two-sided
• Example: SAE rate 3/20 (15%) vs 8/22 (36%), Fisher's exact p=0.083

McNemar's Test

• Application: Paired categorical data (before/after, matched pairs)
• Example: Response before vs after treatment switch in same patients

Sample Size and Power

Power Analysis Components

• Alpha (α): Type I error rate, typically 0.05 (two-sided)
• Beta (β): Type II error rate, typically 0.10 or 0.20
• Power: 1 - β, typically 0.80 or 0.90 (80-90% power)
• Effect size: Expected difference (HR, mean difference, proportion difference)
• Sample size: Number of patients or events needed

Survival Study Sample Size

• Events-driven: Need sufficient events (deaths, progressions)
• Rule of thumb: 80% power requires approximately 165 events for HR=0.70 (α=0.05, two-sided)
• Accrual time + follow-up time determines calendar time

Response Rate Study

Example: Detect ORR difference 45% vs 30% (15 percentage points)
- α = 0.05 (two-sided)
- Power = 0.80
- Sample size: n = 94 per group (188 total)
- With 10% dropout: n = 105 per group (210 total)

Data Visualization

Survival Curves

Kaplan-Meier Plot Best Practices

# Key elements for publication-quality survival curve
1. X-axis: Time (months or years), starts at 0
2. Y-axis: Survival probability (0 to 1.0 or 0% to 100%)
3. Step function: Survival curve with steps at event times
4. 95% CI bands: Shaded region around survival curve (optional but recommended)
5. Number at risk table: Below x-axis showing n at risk at time intervals
6. Censoring marks: Vertical tick marks (|) at censored observations
7. Legend: Clearly identify each curve
8. Log-rank p-value: Prominently displayed
9. Median survival: Horizontal line at 0.50, labeled
10. Follow-up: Median follow-up time reported

Number at Risk Table Format

Number at risk
Group A   50    42    35    28    18    10     5
Group B   48    38    29    19    12     6     2
Time      0     6     12    18    24    30    36 (months)

Hazard Ratio Annotation

On plot: HR 0.62 (95% CI 0.43-0.89), p=0.010
Or in caption: Log-rank test p=0.010; Cox model HR=0.62 (95% CI 0.43-0.89)

Waterfall Plots

Purpose: Visualize individual patient responses to treatment

Construction

• X-axis: Individual patients (anonymized patient IDs)
• Y-axis: Best % change from baseline tumor burden
• Bars: Vertical bars, one per patient
  • Positive values: Tumor growth
  • Negative values: Tumor shrinkage
• Ordering: Sorted from best response (left) to worst (right)
• Color coding:
  • Green/blue: CR or PR (≥30% decrease)
  • Yellow: SD (-30% to +20%)
  • Red: PD (≥20% increase)
• Reference lines: Horizontal lines at +20% (PD), -30% (PR)
• Annotations: Biomarker status, response duration (symbols)

Example Annotations

■ = Biomarker-positive
○ = Biomarker-negative
* = Ongoing response
† = Progressed

Forest Plots

Purpose: Display subgroup analyses with hazard ratios and confidence intervals

Construction

• Y-axis: Subgroup categories
• X-axis: Hazard ratio (log scale), vertical line at HR=1.0
• Points: HR estimate for each subgroup
• Horizontal lines: 95% confidence interval
• Square size: Proportional to sample size or precision
• Overall effect: Diamond at bottom, width represents 95% CI

Subgroups to Display

Subgroup                    n     HR (95% CI)          Favors A  Favors B
──────────────────────────────────────────────────────────────────────────
Overall                     300   0.65 (0.48-0.88)         ●────┤
Age
  <65 years                 180   0.58 (0.39-0.86)        ●────┤
  ≥65 years                 120   0.78 (0.49-1.24)          ●──────┤
Sex
  Male                      175   0.62 (0.43-0.90)        ●────┤
  Female                    125   0.70 (0.44-1.12)         ●─────┤
Biomarker Status
  Positive                  140   0.45 (0.28-0.72)      ●───┤
  Negative                  160   0.89 (0.59-1.34)           ●──────┤
                                  p-interaction=0.041

                                  0.25  0.5   1.0   2.0
                                        Hazard Ratio

Interaction Testing

• Test whether treatment effect differs across subgroups
• p-interaction <0.05 suggests heterogeneity
• Pre-specify subgroups to avoid data mining

Spider Plots

Purpose: Display longitudinal tumor burden changes over time for individual patients

Construction

• X-axis: Time from treatment start (weeks or months)
• Y-axis: % change from baseline tumor burden
• Lines: One line per patient connecting assessments
• Color coding: By response category or biomarker status
• Reference lines: 0% (no change), +20% (PD threshold), -30% (PR threshold)

Clinical Insights

• Identify delayed responders (initial SD then PR)
• Detect early progression (rapid upward trajectory)
• Assess depth of response (maximum tumor shrinkage)
• Duration visualization (when lines cross PD threshold)

Swimmer Plots

Purpose: Display treatment duration and response for individual patients

Construction

• X-axis: Time from treatment start (weeks or months)
• Y-axis: Individual patients (one row per patient)
• Bars: Horizontal bars representing treatment duration
• Symbols:
  • ● Start of treatment
  • ▼ Ongoing treatment (arrow)
  • ■ Progressive disease (end of bar)
  • ◆ Death
  • | Dose modification
• Color: Response status (CR=green, PR=blue, SD=yellow, PD=red)

Example

Patient ID    |0   3   6   9   12  15  18  21  24 months
──────────────|──────────────────────────────────────────
Pt-001        ●═══PR═══════════|════════PR══════════▼
Pt-002        ●═══PR═══════════════PD■
Pt-003        ●══════SD══════════PD■
Pt-004        ●PR══════════════════════════════════PR▼
...

Confidence Intervals

Interpretation

95% Confidence Interval

• Range of plausible values for true population parameter
• If study repeated 100 times, 95 of the 95% CIs would contain true value
• Not: 95% probability true value within this interval (frequentist, not Bayesian)

Relationship to p-value

• If 95% CI excludes null value (HR=1.0, difference=0), p<0.05
• If 95% CI includes null value, p≥0.05
• CI provides more information: magnitude and precision of effect

Precision

• Narrow CI: High precision, large sample size
• Wide CI: Low precision, small sample size or high variability
• Example: HR 0.65 (95% CI 0.62-0.68) very precise; HR 0.65 (0.30-1.40) imprecise

Calculation Methods

Hazard Ratio CI

• From Cox regression output
• Standard error of log(HR) → exp(log(HR) ± 1.96×SE)
• Example: HR=0.62, SE(logHR)=0.185 → 95% CI (0.43, 0.89)

Survival Rate CI (Greenwood Formula)

• SE(S(t)) = S(t) × sqrt(Σ[d_i / (n_i × (n_i - d_i))])
• 95% CI: S(t) ± 1.96 × SE(S(t))
• Can use complementary log-log transformation for better properties

Proportion CI (Exact Binomial)

• For ORR, DCR: Use exact method (Clopper-Pearson) for small samples
• Wilson score interval: Better properties than normal approximation
• Example: 12/30 responses → ORR 40% (95% CI 22.7-59.4%)

Censoring and Missing Data

Types of Censoring

Right Censoring

• End of study: Patient alive at study termination (administrative censoring)
• Loss to follow-up: Patient stops attending visits
• Withdrawal: Patient withdraws consent
• Competing risk: Death from unrelated cause (in disease-specific survival)

Handling Censoring

• Assumption: Non-informative - censoring independent of event probability
• Sensitivity Analysis: Assess impact if assumption violated
  • Best case: All censored patients never progress
  • Worst case: All censored patients progress immediately after censoring
  • Actual result should fall between best/worst case

Missing Data

Mechanisms

• MCAR (Missing Completely at Random): Missingness unrelated to any variable
• MAR (Missing at Random): Missingness related to observed but not unobserved variables
• NMAR (Not Missing at Random): Missingness related to the missing value itself

Handling Strategies

• Complete case analysis: Exclude patients with missing data (biased if not MCAR)
• Multiple imputation: Generate multiple plausible datasets, analyze each, pool results
• Maximum likelihood: Estimate parameters using all available data
• Sensitivity analysis: Assess robustness to missing data assumptions

Response Assessment Missing Data

• Unevaluable for response: Baseline measurable disease but post-baseline assessment missing
  • Exclude from ORR denominator or count as non-responder (sensitivity analysis)
• PFS censoring: Last adequate tumor assessment date if later assessments missing

Reporting Standards

CONSORT Statement (RCTs)

Flow Diagram

• Assessed for eligibility (n=)
• Randomized (n=)
• Allocated to intervention (n=)
• Lost to follow-up (n=, reasons)
• Discontinued intervention (n=, reasons)
• Analyzed (n=)

Baseline Table

• Demographics and clinical characteristics
• Baseline prognostic factors
• Show balance between arms

Outcomes Table

• Primary endpoint results with CI and p-value
• Secondary endpoints
• Safety summary

STROBE Statement (Observational Studies)

Study Design: Cohort, case-control, or cross-sectional

Participants: Eligibility, sources, selection methods, sample size

Variables: Clearly define outcomes, exposures, predictors, confounders

Statistical Methods: Describe all methods, handling of missing data, sensitivity analyses

Results: Participant flow, descriptive data, outcome data, main results, other analyses

Reproducible Research Practices

Statistical Analysis Plan (SAP)

• Pre-specify all analyses before data lock
• Primary and secondary endpoints
• Analysis populations (ITT, per-protocol, safety)
• Statistical tests and models
• Subgroup analyses (pre-specified)
• Interim analyses (if planned)
• Multiple testing procedures

Transparency

• Report all pre-specified analyses
• Distinguish pre-specified from post-hoc exploratory
• Report both positive and negative results
• Provide access to anonymized individual patient data (when possible)

Software and Tools

R Packages for Survival Analysis

• survival: Core package (Surv, survfit, coxph, survdiff)
• survminer: Publication-ready Kaplan-Meier plots (ggsurvplot)
• rms: Regression modeling strategies
• flexsurv: Flexible parametric survival models

Python Libraries

• lifelines: Kaplan-Meier, Cox regression, survival curves
• scikit-survival: Machine learning for survival analysis
• matplotlib: Custom survival curve plotting

Statistical Software

• R: Most comprehensive for survival analysis
• Stata: Medical statistics, good for epidemiology
• SAS: Industry standard for clinical trials
• GraphPad Prism: User-friendly for basic analyses
• SPSS: Point-and-click interface, limited survival features