Relative Decrease In Mae

relative-decrease-in-mae

Machine-Learned Preconditioners for Linear Solvers in Geophysical Fluid Flows — Ackmann et al. (2020) (arXiv:2010.02866, 2020)

What this evaluates

Evaluates the effectiveness of machine-learned preconditioners for linear solvers in semi-implicit shallow-water models by measuring error reduction in the first solver iteration and overall convergence rates during free-running simulations.

Datasets

• Shallow-Water Model Simulation — total ?; splits: train (-1), val (-1), test (-1); repo https://github.com/JanAckmann/MLPrecon

Metrics

• Relative decrease in MAE (primary) — range: other
  • Calculated as the reduction in Mean Absolute Error at each latitude for the first solver iteration, comparing the simulation using the implicit Richardson preconditioner to the machine-learned preconditioner. Lower values indicate better performance.
• Convergence rate — range: other
  • Measured by tracking the normalized residual norms $| r_{\nu} |{\infty} / (\epsilon | r_0 |{\infty})$ across solver iterations during free-running simulations.

Input / output format

Input: Local stencil of residuals $r_0$ (e.g., 5x5 grid or optimized 5-point coordinates) from the shallow-water model state.

Output: Predicted correction term $\Delta \tilde{\Phi}$ (or linear regression coefficients $A^{ij}, B^i$).

Scoring recipe

def compute_relative_mae_decrease(mae_richardson, mae_ml):
    return mae_ml / mae_richardson  # Lower is better

def compute_convergence_rate(residual_nu, residual_0, epsilon):
    norm_nu = np.linalg.norm(residual_nu, ord=np.inf)
    norm_0 = np.linalg.norm(residual_0, ord=np.inf)
    return norm_nu / (epsilon * norm_0)

# Track convergence_rate over solver iterations for free-running simulation

Common pitfalls

• Preconditioners may actually increase error for very small $\Delta \Phi$ values due to incomplete information.
• Training data only covers the first solver iteration, but evaluation tests all iterations in free-running simulations.
• Computational savings depend heavily on balancing iteration count against per-application FLOPs.

Evidence (verbatim from paper)

To get a first impression about the quality of results, we compare the relative decrease in Mean Absolute Error (MAE) at each latitude for the first iteration of the solver between the simulation using the implicit Richardson and the Neural Network preconditioner in Figure 2(a). Lower values mean better performance, i.e. higher error reduction. The shown maxima, minima, and median values of the residual norms $| r_{\nu} |{\infty}$ are normalized by $\epsilon | r_0 |{\infty}$.

Citation

@misc{ackmann2020mlprecon,
  title={Machine-Learned Preconditioners for Linear Solvers in Geophysical Fluid Flows},
  author={Ackmann et al. (2020)},
  year={2020},
  note={arXiv:2010.02866}
}

• arXiv: 2010.02866