Exoplanet Imaging Challenge Ii Eval

exoplanet-imaging-challenge-ii-eval

Exoplanet Imaging Data Challenge, phase II: Comparison of algorithms in terms of characterization capabilities — Cantalloube et al. (2024) (arXiv:2410.17636, 2024)

What this evaluates

This benchmark evaluates the ability of high-contrast imaging post-processing pipelines to accurately estimate the astrometric position of injected exoplanet signals in multispectral astronomical data. It probes how well algorithms handle varying signal-to-noise ratios, complex residual backgrounds (e.g., diffraction patterns, coronagraphic inner working angles), and different observing conditions.

Datasets

• Exoplanet Imaging Data Challenge Phase II — total ?; splits: test (-1)

Metrics

• D_astro^GT (primary) — range: [0, ∞)
  • Euclidean distance (L2-norm) between the estimated Cartesian coordinates of the injected planetary signal and its ground-truth position.

Input / output format

Input: High-contrast multispectral images from GPI or SPHERE-IFS instruments containing injected planetary signals at known ground-truth locations, along with metadata on observing conditions.

Output: Estimated position (Cartesian coordinates or separation/position angle) for each injected planetary signal, optionally accompanied by 1σ uncertainties and the posterior distribution used for estimation.

Scoring recipe

def compute_astrometry_error(predictions, ground_truth):
    # predictions and ground_truth are 2D arrays of (x, y) coordinates
    diff = predictions - ground_truth
    l2_distances = np.sqrt(np.sum(diff**2, axis=1))
    return l2_distances

Common pitfalls

• Proximity to bright diffraction features (e.g., coronagraph IWA, spider diffraction, low wind effect) significantly distorts flux distribution and biases position estimation, often overriding the impact of global observing conditions.
• Low signal-to-noise ratio (SNR) increases estimation uncertainty and error, but spatial location within the field of view frequently dominates accuracy over overall data quality.
• Classical Gaussian fitting methods (e.g., ANDROMEDA) fail to account for complex residual backgrounds, leading to large astrometric biases that exceed one resolution element.

Evidence (verbatim from paper)

D_astro^GT is the metric chosen for the astrometry (L2-norm distance between the estimation and the ground-truth).

Citation

@misc{cantalloube2024exoplanetimaging,
  title={Exoplanet Imaging Data Challenge, phase II: Comparison of algorithms in terms of characterization capabilities},
  author={Cantalloube et al. (2024)},
  year={2024},
  note={arXiv:2410.17636}
}

• arXiv: 2410.17636