Ground Motion Synthesis Eval

ground-motion-synthesis-eval

Broadband Ground Motion Synthesis via Generative Adversarial Neural Operators: Development and Validation — Shi et al. (2023) (arXiv:2309.03447, 2023)

What this evaluates

Evaluates the ability of a generative model to synthesize realistic 3-component broadband ground motion acceleration time histories conditioned on seismic parameters. It probes the model's capacity to match empirical spectral intensities (PSA, FAS, EAS) and capture aleatory variability across different frequency bands and tectonic settings.

Datasets

• BBP dataset — total ?; splits: train (-1), test (-1)
• Kik-net dataset — total ?; splits: train (-1), test (-1)

Metrics

• Normalized model residual (epsilon) (primary) — range: other
  • epsilon = (ln(IM_observed) - mean(ln(IM_synthetic))) / std(ln(IM_synthetic)). The mean of epsilon quantifies model bias, and the standard deviation quantifies whether the synthetic ensemble's variability matches the empirical dataset.
• PSA — range: other
  • 5%-damped pseudo-spectral acceleration. Calculated as rotation-independent (RotD50) for horizontal components and as the 5%-damped response spectrum of the vertical component, over periods 0.01-10 sec.
• FAS — range: other
  • Fourier Amplitude Spectrum. Horizontal FAS is the power mean spectrum of the two horizontal components; vertical FAS is calculated directly from the vertical component.
• EAS — range: other
  • Effective Amplitude Spectrum. Calculated by smoothing the power-mean FAS (horizontal) or vertical FAS using a Konno-Ohmachi smoothing window with bandwidth 0.0333.

Input / output format

Input: Conditional parameters (moment magnitude M, rupture distance R_rup, V_S30, faulting style f_type) and a 1D Gaussian Random Field sample.

Output: 3-component normalized acceleration time series waveforms and three normalization constants (peak ground accelerations).

Scoring recipe

def evaluate(predictions, gold, conditional_vars, dataset):
    # 1. Generate 100 synthetic realizations per scenario
    synthetics = generate_100_samples(predictions, conditional_vars)
    # 2. Compute Intensity Measures (IM: PSA, FAS, or EAS)
    im_obs = compute_IM(gold)
    im_synth = [compute_IM(s) for s in synthetics]
    # 3. Filter usable frequency range (Kik-net only)
    if dataset == 'Kik-net':
        im_obs, im_synth = filter_by_noise_ratio(im_obs, im_synth)
    # 4. Compute log-space statistics of synthetic IMs
    log_im_synth = np.log(im_synth)
    mean_log = np.mean(log_im_synth)
    std_log = np.std(log_im_synth)
    # 5. Compute normalized residual epsilon
    log_im_obs = np.log(im_obs)
    epsilon = (log_im_obs - mean_log) / std_log
    return epsilon.mean(), epsilon.std()

Common pitfalls

• Frequency range filtering must be applied to Kik-net data based on noise-to-signal ratio (LUF/HUF) before computing IMs, whereas BBP uses the full frequency range.
• PSA is computed as rotation-independent (RotD50) for horizontal components, not the standard geometric mean.
• Residuals are computed in log-space; a well-calibrated model should yield mean epsilon ~ 0 and std epsilon ~ 1.

Evidence (verbatim from paper)

The statistical evaluation of the generated ground motions was performed (i) by comparing the PSA and FAS, for BBP dataset, or EAS, for Kik-net dataset, ordinates of the observed and synthetically generated ground motions for specific scenario bins and (ii) by performing residual analysis of the PSA and FAS or EAS of the training and testing datasets, for both the horizontal and vertical directions. ... The comparison is performed by: (i) generating 100 synthetic 3-component realizations with the same (M, R_rup, V_S30, f_type) conditional variable values of the observed ground motion, (ii) computing the relevant IM from the observed and synthetic ground motions (IM^observed, IM^cGM-GANO, respectively), (iii) calculating the mean and standard deviation of IM^cGM-GANO in log space, and (iv) computing the normalized model residual, epsilon, for each observation as: epsilon = (ln(IM^observed) - mean(ln(IM^cGM-GANO))) / std(ln(IM^cGM-GANO))

Citation

@misc{shi2023gano,
  title={Broadband Ground Motion Synthesis via Generative Adversarial Neural Operators: Development and Validation},
  author={Shi et al. (2023)},
  year={2023},
  note={arXiv:2309.03447}
}

• arXiv: 2309.03447