Deepgrav Gw Detection Eval

deepgrav-gw-detection-eval

DeepGrav: Anomalous Gravitational-Wave Detection Through Deep Latent Features — Jianqi Yan et al. (2025) (arXiv:2503.03799, 2025)

What this evaluates

Binary classification capability to distinguish background noise from gravitational-wave signals (specifically BBH and SGLF classes) in time-series data. It probes the model's ability to generalize to unseen gravitational wave anomalies using deep latent features.

Datasets

• HDR A3D3 gravitational-wave dataset — total ?; splits: train (-1), val (-1), test (-1); repo https://github.com/yan123yan/HDR-anomaly-challenge-submission

Metrics

• AUC (primary) — range: [0, 1]
  • Area under the Receiver Operating Characteristic curve, measuring the trade-off between true positive rate and false positive rate across all classification thresholds.
• accuracy — range: [0, 1]
  • Fraction of correctly classified instances out of the total number of instances.

Input / output format

Input: Raw time-series gravitational-wave signal data (background or signal class).

Output: Binary classification label (0 for background, 1 for signal) or predicted probability for the positive class.

Scoring recipe

def compute_auc(y_true, y_pred_proba):
    fpr, tpr, _ = roc_curve(y_true, y_pred_proba)
    return auc(fpr, tpr)

def compute_accuracy(y_true, y_pred_labels):
    return np.mean(y_true == y_pred_labels)

Common pitfalls

• BBH and SGLF classes are merged into a single positive class; evaluating them separately would misrepresent the reported AUC/accuracy.
• Data augmentation (signal averaging) is applied only to the training set; applying it to validation/test sets would invalidate the reported performance metrics.
• The dataset splits are fixed at 70/10/20; ensure the test set is strictly held out and not used during augmentation or tuning.

Evidence (verbatim from paper)

The dataset is divided into three subsets: 70% for training, 10% for validation, and 20% for testing. ... All our models and experimental settings are available on GitHub. ... assessing metrics on the validation set, including the Receiver Operating Characteristic (ROC) curves and AUC.

Citation

@misc{yan2025deepgrav,
  title={DeepGrav: Anomalous Gravitational-Wave Detection Through Deep Latent Features},
  author={Jianqi Yan et al. (2025)},
  year={2025},
  note={arXiv:2503.03799}
}

• arXiv: 2503.03799