Erdes Oculardetachment Eval

erdes-oculardetachment-eval

ERDES: A Benchmark Video Dataset for Retinal Detachment and Macular Status Classification in Ocular Ultrasound — Navard et al. (2025) (arXiv:2508.04735, 2025)

What this evaluates

Evaluates the ability of spatiotemporal deep learning models to classify ocular ultrasound videos into binary diagnostic categories: detecting retinal detachment (RD) versus non-RD, and classifying macular status (intact vs. detached). It probes the model's capacity to learn subtle spatiotemporal patterns in medical ultrasound while handling real-world class imbalance.

Datasets

• ERDES — total 5381; splits: train (3413), val (377), test (945); repo https://github.com/osupcvlab/ERDES

Metrics

• Accuracy — range: [0, 1]
  • Proportion of true results (both true positives and true negatives) among all predictions. Formula: (TP + TN) / (TP + TN + FP + FN).
• Precision — range: [0, 1]
  • Proportion of predicted positive cases that are actually positive. Formula: TP / (TP + FP).
• Sensitivity — range: [0, 1]
  • Proportion of actual positive cases correctly identified. Formula: TP / (TP + FN).
• Specificity — range: [0, 1]
  • Proportion of actual negative cases correctly identified. Formula: TN / (TN + FP).
• F1-Score (primary) — range: [0, 1]
  • Harmonic mean of precision and recall. Formula: 2 * (Precision * Recall) / (Precision + Recall).

Input / output format

Input: Preprocessed ocular ultrasound video clips (MP4) cropped to the region of interest (ROI) using YOLOv8, padded to dimensions divisible by 16, and fed as spatiotemporal volumes to the model.

Output: Binary classification probability score (e.g., P(RD) or P(Macula_Detached)), thresholded at 0.5 to yield a discrete class label (0 or 1).

Scoring recipe

def compute_metrics(y_true, y_pred):
    tp = sum(1 for t, p in zip(y_true, y_pred) if t == 1 and p == 1)
    tn = sum(1 for t, p in zip(y_true, y_pred) if t == 0 and p == 0)
    fp = sum(1 for t, p in zip(y_true, y_pred) if t == 0 and p == 1)
    fn = sum(1 for t, p in zip(y_true, y_pred) if t == 1 and p == 0)
    accuracy = (tp + tn) / (tp + tn + fp + fn)
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0.0
    sensitivity = tp / (tp + fn) if (tp + fn) > 0 else 0.0
    specificity = tn / (tn + fp) if (tn + fp) > 0 else 0.0
    f1 = 2 * precision * sensitivity / (precision + sensitivity) if (precision + sensitivity) > 0 else 0.0
    return {'accuracy': accuracy, 'precision': precision, 'sensitivity': sensitivity, 'specificity': specificity, 'f1': f1}

Common pitfalls

• Severe class imbalance exists (Non-RD: 4,233 vs RD: 502), requiring stratified splits and careful metric selection.
• Posterior vitreous detachment (PVD) clips are explicitly excluded from the Non-RD vs. RD binary task.
• Anatomical subclasses (TD, ND, Bilateral) are grouped under the main RD or Macula labels rather than evaluated separately.
• Models are trained from scratch without external pretraining, which may limit performance compared to pretrained baselines.

Evidence (verbatim from paper)

Performance was evaluated using five standard classification metrics, namely Accuracy, Precision, Sensitivity (Recall), Specificity, and the F1-Score. These metrics provide complementary insights into the performance of model classification, especially in the context of imbalanced datasets.

Citation

@misc{navard2025erdes,
  title={ERDES: A Benchmark Video Dataset for Retinal Detachment and Macular Status Classification in Ocular Ultrasound},
  author={Navard et al. (2025)},
  year={2025},
  note={arXiv:2508.04735}
}

• arXiv: 2508.04735