Multiphysics Parameter Estimation Eval

multiphysics-parameter-estimation-eval

Multiphysics-decision tree learning for improved variably saturated subsurface parameter estimation and reduced-order simulation — Friedel et al. (2023) (arXiv:2312.10213, 2023)

What this evaluates

Evaluates the accuracy of a joint multiphysics-decision tree learning framework in estimating subsurface transport parameters and simulating state variables (pressure head, temperature, concentration) under stochastic boundary conditions. It benchmarks the reduced-order surrogate models against a full numerical multiphysics inversion baseline.

Datasets

• Stochastic managed aquifer recharge dataset — total ?; splits: train (31428), test (7857)

Metrics

• Nash-Sutcliffe efficiency coefficient (primary) — range: [-∞, 1]
  • One minus the ratio of the error variance of the modeled time-series divided by the variance of the observed time-series. A value of 1 indicates a perfect model fit, while values ≤ 0 indicate the model performs no better than the mean of the observations.
• parameter estimation percentage difference — range: percent
  • The average percentage difference between actual and estimated multiphysics property values across model layers. Lower absolute values indicate higher estimation accuracy.

Input / output format

Input: State variables (pressure head, temperature, concentration) and predictor variables (saturated hydraulic conductivity, thermal conductivity, bulk density, longitudinal dispersivity, time, depth).

Output: Predicted values for pressure head, temperature, and concentration.

Scoring recipe

def nash_sutcliffe(observed, predicted):
    mean_obs = sum(observed) / len(observed)
    numerator = sum((o - p)**2 for o, p in zip(observed, predicted))
    denominator = sum((o - mean_obs)**2 for o in observed)
    return 1 - (numerator / denominator)

def param_pct_diff(actual, estimated):
    return sum((a - e) / a for a, e in zip(actual, estimated)) / len(actual) * 100

Common pitfalls

• The dataset is synthetically generated via stochastic boundary conditions rather than field measurements, so results may not directly transfer to real-world aquifer sites without retraining.
• The Nash-Sutcliffe efficiency can be negative if the model is worse than the mean baseline, but the paper only reports values > 0.93, potentially obscuring failure cases.
• Confusion between the supervised reduced-order models (Random Forest/Gradient Boosting for state variables) and the unsupervised Self-Organized Map evaluation splits.

Evidence (verbatim from paper)

Comparison of the numerical multiphysics pressure head, temperature, and concentration simulations (left panel) with the supervised reduced-order model (pressure head predicted by Random Forest, and temperature and concentration predicted by Ensemble Gradient Boosting) simulations (right panel) reveals excellent correspondence (Figure 8). This visual finding is supported quantitatively by the corresponding Nash-Sutcliffe model efficiency coefficients (Nash and Sutcliffe, 1970) calculated (Table 8) among the multiphysics numerical and reduced-order model simulations for state variables > 0.93 (e.g. pressure head = 0.95, temperature = 0.98, concentration = 0.93). The Nash-Sutcliffe efficiency is calculated as one minus the ratio of the error variance of the modeled time-series divided by the variance of the observed time-series.

Citation

@misc{friedel2023multiphysics,
  title={Multiphysics-decision tree learning for improved variably saturated subsurface parameter estimation and reduced-order simulation},
  author={Friedel et al. (2023)},
  year={2023},
  note={arXiv:2312.10213}
}

• arXiv: 2312.10213