Tidmad Denoising Eval

tidmad-denoising-eval

TIDMAD: Time Series Dataset for Discovering Dark Matter with AI Denoising — Fry et al. (2024) (arXiv:2406.04378, 2024)

What this evaluates

Evaluates the ability of traditional and deep learning denoising algorithms to recover sinusoidal dark matter signals from ultra-long, noisy time series data collected by the ABRACADABRA experiment.

Datasets

• TIDMAD — total ?; splits: train (-1), val (-1), test (-1); repo https://github.com/jessicafry/TIDMAD

Metrics

• mean square error (primary) — range: other
  • Average of the squared differences between predicted and ground truth values. (Note: The specific benchmarking metric used for final model comparison is referenced in Section 4 but not detailed in this section.)

Input / output format

Input: Ultra-long time series segments (SQUID and injected signals), processed in frequency-split chunks for most models.

Output: Single floating point number per time step (regression) or 256-class classification decision per time step (mapping to 8-bit integers -128 to 127).

Scoring recipe

def compute_mse(predictions, gold):
    return np.mean((predictions - gold) ** 2)
# Note: Final benchmarking metric is detailed in Sec. 4 of the paper.

Common pitfalls

• Memory constraints require segmenting ultra-long time series before model input.
• Frequency splitting is required for all models except WaveNet due to broad input frequency spectrum.
• Classification models use Focal Loss to handle class-imbalanced 256-class labels.

Evidence (verbatim from paper)

FC-Net outputs a single floating point number at each time step, and the training is conducted by minimizing the mean square error between this floating point number and corresponding ground truth time series at every sample.

Citation

@misc{fry2024tidmad,
  title={TIDMAD: Time Series Dataset for Discovering Dark Matter with AI Denoising},
  author={Fry et al. (2024)},
  year={2024},
  note={arXiv:2406.04378}
}

• arXiv: 2406.04378