Mimic Sepsis Eval

mimic-sepsis-eval

MIMIC-Sepsis: A Curated Benchmark for Modeling and Learning from Sepsis Trajectories in the ICU — Huang et al. (2025) (arXiv:2510.24500, 2025)

What this evaluates

Evaluates the ability of models to predict clinical outcomes (mortality, length of stay, shock onset) from time-aligned ICU patient trajectories and treatment dynamics.

Datasets

• MIMIC-Sepsis — total 35239; splits: train (-1), val (-1), test (-1); repo https://github.com/yongh7/MIMIC-sepsis

Metrics

• performance (primary) — range: [0, 1]
  • General predictive performance measured via accuracy for binary classification tasks (mortality, shock onset) and MAE for continuous regression (length of stay).

Input / output format

Input: Time-aligned clinical trajectories resampled to 4-hour intervals from 24 hours before to 72 hours after suspected infection onset, including static demographics, longitudinal vitals/labs, and cumulative treatment interventions (vasopressors, fluids, antibiotics, ventilation).

Output: Predictions for three tasks: binary mortality status, continuous length of stay, and binary shock onset status.

Scoring recipe

def score(predictions, gold):
    if task in ['mortality', 'shock']:
        return sum(p == g for p, g in zip(predictions, gold)) / len(gold)
    elif task == 'los':
        return sum(abs(p - g) for p, g in zip(predictions, gold)) / len(gold)
    return None

Common pitfalls

• Imputation strategy varies by missingness level (<5% linear, <80% KNN, >80% excluded), which can bias results if not replicated.
• Temporal alignment is strictly relative to suspected infection onset (Sepsis-3 criteria), not ICU admission.
• Treatment variables are cumulative per 4-hour interval, requiring careful handling of dosing conversions (e.g., norepinephrine-equivalent).

Evidence (verbatim from paper)

Empirical results show that integrating treatment dynamics significantly improves performance of Transformer-based models, especially in early mortality prediction, length-of-stay estimation, and shock onset classification—demonstrating the critical role of treatment-aware modeling in critical care prediction.

Citation

@misc{huang2025mimicsepsis,
  title={MIMIC-Sepsis: A Curated Benchmark for Modeling and Learning from Sepsis Trajectories in the ICU},
  author={Huang et al. (2025)},
  year={2025},
  note={arXiv:2510.24500}
}

• arXiv: 2510.24500