# Mimic Iv Clinical Prediction Eval

> Evaluates clinical prediction models on irregular, sparse time-series data from the ICU module of MIMIC-IV. It probes the ability of models to handle temporal sparsity, missingness, and heterogeneous features for binary classification tasks like in-ICU mortality and length of stay prediction. Use when the user wants to benchmark on MIMIC-IV 2.2, or asks about evaluating this task. Reports AUC-ROC.

- Skill: `qhjqhj00/mimic-iv-clinical-prediction-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/mimic-iv-clinical-prediction-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/mimic-iv-clinical-prediction-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Productivity
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/mimic-iv-clinical-prediction-eval

---


# mimic-iv-clinical-prediction-eval

> Benchmarking with MIMIC-IV, an irregular, spare clinical time series dataset — Bui et al. (2024) (arXiv:2401.15290, 2024)

## What this evaluates

Evaluates clinical prediction models on irregular, sparse time-series data from the ICU module of MIMIC-IV. It probes the ability of models to handle temporal sparsity, missingness, and heterogeneous features for binary classification tasks like in-ICU mortality and length of stay prediction.

## Datasets

- **MIMIC-IV 2.2** — total ?; splits: train (-1), val (-1), test (-1)

## Metrics

- `AUC-ROC` **(primary)** — range: [0, 1]
  - Area under the Receiver Operating Characteristic curve. Measures the trade-off between true positive rate and false positive rate across all classification thresholds.
- `PR-AUC` — range: [0, 1]
  - Area under the Precision-Recall curve. Measures the trade-off between precision and recall across all classification thresholds, particularly useful for imbalanced clinical datasets.

## Input / output format

**Input**: 2-hour resolution time series of lab/vital measurements and ICD-10 diagnosis codes for the first 48 hours of an ICU admission.

**Output**: Binary classification label: 1 for in-ICU mortality or length of stay ≥ 3 days, 0 otherwise.

## Scoring recipe

```python
def compute_auc_roc(y_true, y_pred):
    fpr, tpr, _ = roc_curve(y_true, y_pred)
    return auc(fpr, tpr)

def compute_pr_auc(y_true, y_pred):
    precision, recall, _ = precision_recall_curve(y_true, y_pred)
    return auc(recall, precision)

# 5-fold CV: 80% train, 20% test. 10% of train used for validation.
# Average metrics across all folds.
```

## Common pitfalls

- The dataset uses forward-fill and mean imputation for missing labs/vitals, which can artificially inflate performance if models assume regular sampling.
- Table 3 incorrectly labels the second metric as 'PR-ROC' instead of PR-AUC, which may cause confusion when reproducing results.
- Evaluation is strictly limited to ICU stays for Chronic Kidney Disease patients, so results do not generalize to other wards or conditions.

## Evidence (verbatim from paper)

> Regarding evaluation, we utilize area under the ROC curve (OC-AUC score) and area under precision-recall curve (PR-AUC score). ... We use 5-folded cross validation to verify the model performance, in which the dataset is spit 80% for training, and 20% for testing.

## Citation

```bibtex
@misc{bui2024benchmarking,
  title={Benchmarking with MIMIC-IV, an irregular, spare clinical time series dataset},
  author={Bui et al. (2024)},
  year={2024},
  note={arXiv:2401.15290}
}
```

- arXiv: 2401.15290