Openfwi Eval

openfwi-eval

Edge-InversionNet: Enabling Efficient Inference of InversionNet on Edge Devices — Wang et al. (2023) (IMAGE '23 (under review) / arXiv:2310.09667, 2023)

What this evaluates

Evaluates the predictive accuracy and computational efficiency of a pruned deep learning model for seismic full waveform inversion. It measures how well the model reconstructs subsurface velocity maps and quantifies inference latency and resource consumption on edge hardware.

Datasets

• OpenFWI — total 151000; splits: test (-1)

Metrics

• MAE (primary) — range: other
  • Average of absolute differences between predicted and ground-truth velocity map values.
• RMSE — range: other
  • Square root of the average of squared differences between predicted and ground-truth velocity map values.
• SSIM — range: [0, 1]
  • Structural similarity index measuring luminance, contrast, and structure correlation between predicted and ground-truth maps.

Input / output format

Input: Tensor of shape (5, 1000, 1, 70) representing number of sources × time steps × receivers width × receivers length.

Output: Tensor of shape (70, 1, 70) representing depth × width × length of the predicted velocity map.

Scoring recipe

def compute_metrics(pred, gold):
    mae = np.mean(np.abs(pred - gold))
    rmse = np.sqrt(np.mean((pred - gold) ** 2))
    ssim = compute_ssim(pred, gold)
    return mae, rmse, ssim

Common pitfalls

• The paper plots (1-SSIM) instead of raw SSIM to align error direction with MAE/RMSE, which can cause confusion when comparing figures to tables.
• Latency measurements are hardware-specific (Raspberry Pi) and averaged over 50 runs; they do not generalize to other edge devices or software backends.
• Only 6 out of 10 OpenFWI datasets are visualized in figures due to page limits; full numerical results are only reported in Table I.

Evidence (verbatim from paper)

Here we use three metrics for evaluation: mean absolute error (MAE), rooted mean squared error (RMSE) and structural similarity (SSIM). MAE and RMSE quantify the numerical disparity between the predicted and true velocity maps, while SSIM captures the perceptual similarity of the two maps.

Citation

@misc{wang2023edgeinversionnet,
  title={Edge-InversionNet: Enabling Efficient Inference of InversionNet on Edge Devices},
  author={Wang et al. (2023)},
  year={2023},
  note={IMAGE '23 (under review) / arXiv:2310.09667}
}

• arXiv: 2310.09667