Hephaestus Minicubes Eval

hephaestus-minicubes-eval

Hephaestus Minicubes: A Global, Multi-Modal Dataset for Volcanic Unrest Monitoring — Papadopoulos et al. (2025) (arXiv:2505.17782, 2025)

What this evaluates

Evaluates models on detecting volcanic ground deformation using multi-modal InSAR data. It probes the ability to classify deformation presence and segment deformation areas from spatiotemporal interferometric time-series, while handling atmospheric noise and class imbalance.

Datasets

• Hephaestus Minicubes — total ?; splits: train (9840), val (2570), test (6501); repo https://github.com/Orion-AI-Lab/Hephaestus-minicubes

Metrics

• F1-score (primary) — range: percent
  • Harmonic mean of precision and recall: 2 * (Prec * Rec) / (Prec + Rec).
• IoU (primary) — range: percent
  • Intersection over Union: size of intersection between predicted and ground truth masks divided by size of their union.
• Precision — range: percent
  • Ratio of true positive predictions to all positive predictions.
• Recall — range: percent
  • Ratio of true positive predictions to all actual positives.
• AUROC — range: [0, 1]
  • Area Under the Receiver Operating Characteristic curve, measuring classification performance across all thresholds.

Input / output format

Input: Multi-channel InSAR datacubes (phase, coherence, DEM, atmospheric variables) cropped to 512x512 pixels. For time-series tasks, inputs are sequences of 3 interferograms sharing the same primary acquisition date but different secondary dates, ordered chronologically by secondary date.

Output: For classification: a binary label (deformation vs. no deformation). For segmentation: a binary mask representing the union of deformation areas across the input sequence.

Scoring recipe

# Classification F1
tp = sum((pred == 1) & (gold == 1))
fp = sum((pred == 1) & (gold == 0))
fn = sum((pred == 0) & (gold == 1))
prec = tp / (tp + fp) if (tp + fp) > 0 else 0
rec = tp / (tp + fn) if (tp + fn) > 0 else 0
f1 = 2 * prec * rec / (prec + rec) if (prec + rec) > 0 else 0

# Segmentation IoU
intersection = sum((pred_mask == 1) & (gold_mask == 1))
union = sum((pred_mask == 1) | (gold_mask == 1))
iou = intersection / union if union > 0 else 0

Common pitfalls

• Class imbalance is addressed via undersampling negatives to match positives during training, which may skew evaluation if not accounted for.
• Temporal split is used (2014-2019 train, 2019 val, 2020-2021 test) rather than random splitting, making results non-transferable to other time periods.
• Time-series labels are aggregated: a sequence is positive if any product shows deformation, and the segmentation mask is the union of all sequence masks.
• Models are evaluated with and without auxiliary atmospheric variables, significantly impacting performance.

Evidence (verbatim from paper)

To enable a fair comparison of future methods for InSAR based volcanic unrest detection, we provide the first benchmark on Hephaestus Minicubes. This benchmark is designed to serve as a strong baseline across two fundamental tasks: binary ground deformation classification and semantic segmentation. ... In Tabs.[3] and[4], we present the classification and segmentation results, respectively, reporting Precision, Recall, F1-score, and Area Under the Receiver Operating Characteristic curve (AUROC) for the classification task, and Precision, Recall, F1-score, and Intersection over Union (IoU) for the segmentation task.

Citation

@misc{papadopoulos2025hephaestus,
  title={Hephaestus Minicubes: A Global, Multi-Modal Dataset for Volcanic Unrest Monitoring},
  author={Papadopoulos et al. (2025)},
  year={2025},
  note={arXiv:2505.17782}
}

• arXiv: 2505.17782