Sphere Exoplanet Detection Eval

sphere-exoplanet-detection-eval

A New Statistical Model of Star Speckles for Learning to Detect and Characterize Exoplanets in Direct Imaging Observations — Bodrito et al. (2025) (arXiv:2503.17117, 2025)

What this evaluates

This evaluation probes an algorithm's ability to detect faint exoplanet signals buried in structured stellar speckle noise and accurately characterize their physical properties in direct imaging observations. It measures detection sensitivity across varying false alarm rates and quantifies regression accuracy for astrophysical parameters like flux and sub-pixel position.

Datasets

• SPHERE — total ?; splits: train (220), test (8)

Metrics

• AUC — range: [0, 1]
  • Area under the receiver operating characteristic curve plotting true positive rate against false discovery rate across varying detection thresholds.
• ARE (primary) — range: [0, 1]
  • Absolute relative error between ground truth and estimated flux: |F_true - F_pred| / F_true.
• RMSE — range: other
  • Root mean square error for sub-pixel localization: sqrt(mean((x_true - x_pred)^2)).

Input / output format

Input: 4-D data cubes (2 spectral channels, 15–300 temporal frames, 256×256 pixels), off-axis PSF, central wavelengths, and parallactic angles.

Output: Detection maps (signal-to-noise scores) and estimated exoplanet parameters (flux and sub-pixel coordinates).

Scoring recipe

def compute_auc(tp_rates, fp_rates):
    return np.trapz(tp_rates, fp_rates)

def compute_are(y_true_flux, y_pred_flux):
    return np.abs(y_true_flux - y_pred_flux) / y_true_flux

def compute_rmse(y_true_pos, y_pred_pos):
    return np.sqrt(np.mean((y_true_pos - y_pred_pos)**2))

# Aggregate over test set
auc_score = compute_auc(tp_rates, fp_rates)
are_score = np.mean(compute_are(gold_flux, pred_flux))
rmse_score = compute_rmse(gold_pos, pred_pos)

Common pitfalls

• Synthetic exoplanet injection is required for ground truth, which may not perfectly replicate real planet PSFs or complex noise statistics.
• AUC is preferred over fixed thresholds because different methods lack statistical grounding, making consistent false alarm rates hard to enforce.
• ADI mode suffers from self-subtraction artifacts, requiring spectral diversity or multi-scale modeling to recover sensitivity.

Evidence (verbatim from paper)

The detection metric used is the area under the receiver operating characteristic curve (AUC), representing the true positive rate against the false discovery rate obtained by varying the detection threshold. Higher AUC values indicate better performance. This standard metric in direct imaging [7, 25, 34] captures the precision-recall tradeoff and allows fair algorithm comparisons, as a common threshold does not ensure consistent false alarm rates due to the lack of statistical grounding in some detection maps, see Sec. 1. The primary metric is the absolute relative error (ARE) between the ground truth and estimated flux, with lower values indicating better performance.

Citation

@misc{bodrito2025speckle,
  title={A New Statistical Model of Star Speckles for Learning to Detect and Characterize Exoplanets in Direct Imaging Observations},
  author={Bodrito et al. (2025)},
  year={2025},
  note={arXiv:2503.17117}
}

• arXiv: 2503.17117