Seismic Wavefield Ctf Eval

seismic-wavefield-ctf-eval

The Seismic Wavefield Common Task Framework — Yermakov et al. (2025) (arXiv:2512.19927, 2025)

What this evaluates

Evaluates machine learning models for seismic wavefield forecasting, reconstruction, and generalization under realistic constraints like noise and limited data. It probes model robustness and dynamic learning by comparing performance across multiple tasks against naive baselines.

Datasets

• global wavefields — total ?; splits: test (-1); repo https://github.com/CTF-for-Science/ctf4science
• DAS — total ?; splits: test (-1)
• synthetic 3D crustal wavefields — total ?; splits: test (-1)

Metrics

• multi-metric scoring (primary) — range: [0, 100]
  • A normalized aggregate score across multiple tasks (e.g., forecasting, denoising, reconstruction) scaled to a maximum of 100. Computed by evaluating model predictions against ground truth for each task and aggregating the results.

Input / output format

Input: Seismic wavefield time-series or spatial-temporal data from sparse sensor arrays or Distributed Acoustic Sensing (DAS) systems.

Output: Predicted wavefield values or reconstructed states corresponding to specific tasks (e.g., short-term forecasting, denoising, state reconstruction).

Scoring recipe

scores = []
for task in [E3, E5, E7, E9, ...]:
    error = compute_error(predictions[task], ground_truth[task])
    normalized = scale_to_100(error, baseline_error)
    scores.append(normalized)
final_score = aggregate(scores)
return final_score

Common pitfalls

• Assuming a single metric captures model capability; the framework requires multi-task evaluation to reveal strengths and weaknesses.
• High scores on foundation models may indicate zero-prediction fallback rather than actual dynamic forecasting.
• Overfitting to noise in limited datasets causes complex architectures to fail against naive zero or average baselines.

Evidence (verbatim from paper)

This result also demonstrates how the multi-metric scoring in the CTF is better than a single score. While the RNNs achieve high average scores, they perform exceptionally poorly on tasks E7 and E9, corresponding to short-term prediction on limited data (noiseless and noisy, respectively).

Citation

@misc{yermakov2025seismicwavefieldctf,
  title={The Seismic Wavefield Common Task Framework},
  author={Yermakov et al. (2025)},
  year={2025},
  note={arXiv:2512.19927}
}

• arXiv: 2512.19927