Stead Magnitude Estimation Eval

stead-magnitude-estimation-eval

A Machine-Learning Approach for Earthquake Magnitude Estimation — Mousavi et al. (2019) (arXiv:1911.05975, 2019)

What this evaluates

Evaluates a deep learning model's ability to directly estimate earthquake magnitude (both local ML and duration Md) from raw, unprocessed single-station seismograms without normalization or instrument response correction. It probes the model's robustness to site effects, regional calibration differences, and varying signal-to-noise ratios.

Datasets

• STEAD — total 300000; splits: train (-1), val (-1), test (-1)

Metrics

• mean error (primary) — range: other
  • Average difference between predicted and cataloged magnitude values across the test set.
• standard deviation of prediction error — range: other
  • Standard deviation of the differences between predicted and true magnitudes.
• coefficient of determination — range: [0, 1]
  • R^2 score measuring the proportion of variance in the true magnitudes explained by the model.

Input / output format

Input: Band-pass filtered (1.0-40.0 Hz) single-station seismogram waveforms, ≤ 30 seconds long (from 1 second before P-wave onset to end of S coda), with SNR > 20 dB.

Output: A single floating-point magnitude value (predicting either ML or Md).

Scoring recipe

def compute_metrics(predictions, ground_truth):
    errors = predictions - ground_truth
    mean_error = np.mean(errors)
    std_error = np.std(errors)
    ss_res = np.sum((ground_truth - predictions) ** 2)
    ss_tot = np.sum((ground_truth - np.mean(ground_truth)) ** 2)
    r2 = 1 - (ss_res / ss_tot) if ss_tot != 0 else 0
    return {'mean_error': mean_error, 'std_error': std_error, 'r2': r2}

Common pitfalls

• Performance degrades significantly at magnitude extremes due to fewer training/test samples at the bounds.
• Site effects (surface vs. borehole stations) and regional calibration differences heavily influence accuracy, making global models sensitive to station type.
• Signal-to-noise ratio has a greater impact on performance than network depth, requiring careful data filtering to avoid misleading results.

Evidence (verbatim from paper)

We use a selected portion of STanford EArthquake Dataset (STEAD) 15 for training of the network. STEAD is a global dataset of labeled seismograms including local earthquake and seismic noise waveforms. Here, we only used ~ 300,000 earthquake waveforms recorded at epicentral distances of less than 1 degree, for which their full waveforms (from 1 second before P until the end of the S coda) are equal or less than 30 seconds. ... used 70% of each subset for the training and 10% and 20% for the validation and testing respectively. ... Overall the network is able to predict earthquake magnitudes with a mean error close to zero and standard deviation of ~ 0.2. ... Although the coefficient of determination decreases slightly, due to relatively smaller size of the training set, we see only a small improvement in standard deviation of prediction error.

Citation

@misc{mousavi2019machine,
  title={A Machine-Learning Approach for Earthquake Magnitude Estimation},
  author={Mousavi et al. (2019)},
  year={2019},
  note={arXiv:1911.05975}
}

• arXiv: 1911.05975