Rgz Od Eval

rgz-od-eval

Optimization of Deep Learning Models for Radio Galaxy Classification — Denzel et al. (2026) (arXiv:2601.04773, 2026)

What this evaluates

Evaluates the ability of deep learning models to classify radio galaxy morphologies and detect radio sources in continuum images. It probes transfer learning, data preprocessing robustness, and handling of class imbalance in a specialized astronomical domain.

Datasets

• RGZ OD — total ?; splits: train (-1), test (-1)

Metrics

• Top-1 accuracy (primary) — range: percent
  • Percentage of samples where the model's highest-confidence prediction matches the ground truth class. Computed as macro-weighted to handle class imbalance.
• Top-2 accuracy — range: percent
  • Percentage of samples where the ground truth class appears among the two highest-confidence predictions. Used to measure performance excluding confusion between specific FR-I/FR-II class pairs.
• mAP — range: percent
  • Mean Average Precision across all classes, averaging the area under the precision-recall curve for each class. Standard object detection metric.
• IoU — range: percent
  • Intersection over Union between predicted and ground truth bounding boxes. Calculated as the area of overlap divided by the area of union.

Input / output format

Input: Radio continuum images of galaxy sources, preprocessed using various scaling techniques (z-scaling, min-max scaling, ZMZStack) and optional data augmentation.

Output: Classification: discrete class labels (e.g., 1_1, 1_2, 1_3, 2_2, 2_3, 3.3). Object Detection: bounding box coordinates and class labels per detected source.

Scoring recipe

# Classification
preds = argmax(model_output, dim=1)
top2_preds = argsort(model_output, k=2)
correct_top1 = sum(1 for p, g in zip(preds, gold) if p == g)
correct_top2 = sum(1 for p2, g in zip(top2_preds, gold) if g in p2)
acc_top1 = macro_weighted(correct_top1 / total)
acc_top2 = macro_weighted(correct_top2 / total)

# Object Detection
ious = [intersection_area(pred_box, gt_box) / union_area(pred_box, gt_box) for pred_box, gt_box in matches]
aps = [average_precision(recall, precision) for class_preds in classes]
mAP = mean(aps)
avg_iou = mean(ious)

Common pitfalls

• Confusion between class pairs 1_2/1_3 and 2_2/2_3 (FR-I/FR-II galaxies) is common for both models and human experts.
• Ground truth bounding boxes are automatically generated and strictly quadratic, causing systematic IoU shifts that penalize well-matched but non-square predictions.
• Class imbalance significantly skews standard accuracy; the paper uses macro-weighted metrics to account for this.

Evidence (verbatim from paper)

To account for imbalanced classes in the dataset, the performance metrics are macro-weighted. ... both top-1 and top-2 accuracies are provided. ... In terms of mean average precision (mAP) (Padilla et al. 2020), the YOLOv8 models could not outperform transformer-based models such as DINO. However, YOLOv8 achieved the best performance in terms of Intersection over Union (IoU).

Citation

@misc{denzel2026optimization,
  title={Optimization of Deep Learning Models for Radio Galaxy Classification},
  author={Denzel et al. (2026)},
  year={2026},
  note={arXiv:2601.04773}
}

• arXiv: 2601.04773