Relative Mean Square Error

relative-mean-square-error

Learning fast, accurate, and stable closures of a kinetic theory of an active fluid — Maddu et al. (2023) (arXiv:2308.06675, 2023)

What this evaluates

Evaluates the prediction accuracy of data-driven neural network closures for kinetic theory of active fluids against reference kinetic simulations and traditional analytical closures. It probes the model's ability to capture rotational symmetries, generalize across parameter regimes, and resolve topological defects in fluid dynamics.

Datasets

• Kinetic simulation data for active fluid closures — total ?; splits: train (-1), test (-1)

Metrics

• RMSE (primary) — range: [0, 1]
  • Relative Mean Square Error computed as the squared L2 norm of the difference between reference and predicted closure terms, divided by the squared L2 norm of the reference term. Formula: RMSE(ζ) = ||S:T(ζ) - S̃:T(ζ,θ)||₂² / ||S:T(ζ)||₂².

Input / output format

Input: Invariant quantities I_s(D,E) or independent tensor components {d11,d12,d22,e11,e12,e22}, along with alignment strength ζ and other parameters (α, β, dT, dR).

Output: Predicted closure term S̃:T (tensor contraction value).

Scoring recipe

def compute_rmse(reference, prediction):
    num = np.linalg.norm(reference - prediction)**2
    den = np.linalg.norm(reference)**2
    return num / den

Common pitfalls

• FNO architectures show significantly higher error with component-wise learning but improve by two orders of magnitude when using invariant representations.
• Extrapolation accuracy drops sharply when testing on ζ values outside the training range due to overfitting and changes in fluid flow characteristics.
• Traditional closures (Linear, Quadratic) and FNO struggle to resolve prediction errors near topological defects compared to MLP/CNN with invariant features.

Evidence (verbatim from paper)

We use relative mean-square error (RMSE) as the performance metric: RMSE(ζ) = ||S:T(ζ) - S̃:T(ζ,θ)||₂² / ||S:T(ζ)||₂². From Fig. 3, we report that, irrespective of the network architecture, learning based on invariant representation leads to better performance in terms of prediction accuracy compared to component-wise learning.

Citation

@misc{maddu2023learning,
  title={Learning fast, accurate, and stable closures of a kinetic theory of an active fluid},
  author={Maddu et al. (2023)},
  year={2023},
  note={arXiv:2308.06675}
}

• arXiv: 2308.06675