K2 Vetting Eval

k2-vetting-eval

Discovery and Vetting of Exoplanets I: Benchmarking K2 Vetting Tools — Kostov et al. (2019) (arXiv:1901.07459, 2019)

What this evaluates

Evaluates automated exoplanet vetting pipelines on K2 transit lightcurves by comparing their planet candidate versus false positive dispositions against established ground truth from the NASA Exoplanet Archive. It probes the ability of tools to correctly identify true transiting planets while filtering out astrophysical false positives like eclipsing binaries and blended stars.

Datasets

• K2 Planet Candidate Catalog (DAVE Benchmark) — total 772; splits: test (772); repo https://github.com/exoplanetvetting/DAVE

Metrics

• disposition (primary) — range: other
  • Binary classification of each target as either a planet candidate (“PC”) or a false positive (“FP”), evaluated against NExScI ground truth. Calculated as the proportion of targets where the automated disposition matches the confirmed status.

Input / output format

Input: EPIC ID, transit properties, and lightcurves from up to four detrending pipelines (AGP, EVEREST, PDC, SFF) for each K2 target.

Output: Binary disposition (“PC” or “FP”), reason code (e.g., “CO” for centroid offset), and additional comments.

Scoring recipe

def score_disposition(predictions, gold):
    correct = 0
    for p, g in zip(predictions, gold):
        gold_label = 'PC' if g == 'confirmed' else 'FP'
        if p == gold_label:
            correct += 1
    return correct / len(predictions)

Common pitfalls

• Lightcurve pipeline choice significantly affects SNR and dispositions; a target may be classified differently depending on whether AGP, EVEREST, PDC, or SFF data is used.
• Low SNR targets (SNR < 7 or < 4) are automatically flagged as FPs by DAVE but often require visual inspection to avoid false rejections.
• Ground truth from NExScI can be inconsistent, with some targets listed as both PC and FP, or conflicting with recent literature.

Evidence (verbatim from paper)

The catalog lists the EPIC ID of each K2 planet candidate, the transit properties, disposition (planet candidate “PC” or false positive “FP”), the reason for the disposition (e.g. centroid offset during transit, as “CO”), as well as additional comments (for example, presence of field stars in the target’s aperture, as “FSAp”).

Citation

@misc{kostov2019dave,
  title={Discovery and Vetting of Exoplanets I: Benchmarking K2 Vetting Tools},
  author={Kostov et al. (2019)},
  year={2019},
  note={arXiv:1901.07459}
}

• arXiv: 1901.07459