Mimic Iv Tu Dataset Eval

mimic-iv-tu-dataset-eval

Self-supervised Representation Learning on Electronic Health Records with Graph Kernel Infomax — Yao et al. (2022) (arXiv:2209.00655, 2022)

What this evaluates

Evaluates self-supervised graph representation learning on electronic health records and general graph classification tasks. It probes the model's ability to learn temporal and structural patient representations without task-specific fine-tuning, and its robustness across clinical and non-clinical domains.

Datasets

• MIMIC-IV — total ?; splits: 10-fold CV (repeated 5 times) (-1)
• TUDataset — total ?; splits: 10-fold CV (repeated 5 times) (-1)

Metrics

• binary classification accuracy (primary) — range: [0, 1]
  • Standard accuracy: the proportion of correctly predicted labels out of the total number of instances. For similarity search, top-K accuracy is used.

Input / output format

Input: Graph-structured Electronic Health Records (nodes: medical codes/visits, edges: temporal/structural relationships) or standard graph classification inputs.

Output: Binary label (mortality prediction), ranked list of similar patients (top-K), or discrete graph class label.

Scoring recipe

def compute_accuracy(predictions, labels):
    correct = sum(1 for p, l in zip(predictions, labels) if p == l)
    return correct / len(labels)

def compute_topk_accuracy(predictions, labels, k=10):
    correct = sum(1 for p, l in zip(predictions, labels) if l in p[:k])
    return correct / len(labels)

Common pitfalls

• Evaluations use a linear probe on frozen representations without any supervised fine-tuning, unlike many baselines that fine-tune end-to-end.
• The pre-training set explicitly excludes patients with the target chronic diseases to prevent data leakage during clinical evaluation.
• Cross-validation is repeated 5 times (5x10-fold) rather than the standard single split, requiring careful aggregation of results.

Evidence (verbatim from paper)

We evaluate GKI under linear evaluation protocol [3]. The unsupervised pre-training is firstly performed on all patient EHRs in the MIMIC-IV dataset with all selected chronic disease patients excluded. The learned representations from the frozen model are directly applied without supervised fine-tuning and evaluated with 5 times repeated 10-fold cross-validation to clinical downstream tasks. We formulate Treatment plan mortality prediction as a binary classification problem with linear logistic regression and patient similarity search as a top-K similarity search by the inner product via K-nearest neighbors where the ground-truth labels are their primary diagnosed disease.

Citation

@misc{yao2022graphkernelinfomax,
  title={Self-supervised Representation Learning on Electronic Health Records with Graph Kernel Infomax},
  author={Yao et al. (2022)},
  year={2022},
  note={arXiv:2209.00655}
}

• arXiv: 2209.00655