Critical Point Uncertainty Eval

critical-point-uncertainty-eval

Uncertainty Visualization of Critical Points of 2D Scalar Fields for Parametric and Nonparametric Probabilistic Models — Athawale et al. (2024) (arXiv:2407.18015, 2024)

What this evaluates

Evaluates the correctness and computational efficiency of closed-form algorithms for computing critical point probabilities in 2D scalar fields under various parametric and nonparametric noise models. The protocol compares these analytical solutions against Monte Carlo sampling baselines across synthetic and real-world scientific datasets to validate accuracy and speed.

Datasets

• Ackley function (synthetic) — total ?; splits: test (-1)
• Gaussian mixture model (synthetic) — total ?; splits: test (-1)
• E3SM climate data — total ?; splits: test (-1)
• Red Sea oceanology data — total ?; splits: test (-1)

Metrics

• RMSE (primary) — range: [0, 1]
  • Root Mean Squared Error between Monte Carlo probability estimates and closed-form algorithm outputs: sqrt(mean((p_MC - p_closed)^2)). Lower values indicate higher correctness.
• Error_max — range: [0, 1]
  • Maximum absolute probability difference between MC and closed-form solutions across all grid vertices: max(|p_MC - p_closed|). Captures worst-case deviation.
• Speed-up — range: other
  • Ratio of Monte Carlo runtime to closed-form algorithm runtime. Higher values indicate greater computational efficiency.

Input / output format

Input: 2D scalar field grid data with ensemble members or error bounds defining uncertainty at each vertex. Noise models (uniform, Epanechnikov, Gaussian, multivariate Gaussian, histogram) are applied to estimate local probability distributions.

Output: Probability maps for local minima, local maxima, and saddle points at each grid vertex, along with quantitative correctness metrics (RMSE, Error_max) and performance metrics (runtime, speed-up) compared to MC baselines.

Scoring recipe

def evaluate(mc_probs, closed_probs, mc_time, closed_time):
    rmse = np.sqrt(np.mean((mc_probs - closed_probs) ** 2))
    error_max = np.max(np.abs(mc_probs - closed_probs))
    speedup = mc_time / closed_time
    return {'RMSE': rmse, 'Error_max': error_max, 'Speed-up': speedup}

Common pitfalls

• MC sampling accuracy heavily depends on sample size; low samples (e.g., 100) produce noisy probability estimates and require careful convergence checking.
• Parametric models like Gaussian assume fixed distribution shapes, which can misrepresent multimodal or skewed noise and assign high probabilities to outlier features.
• Real-world datasets lack ground-truth critical points, requiring reliance on qualitative visual consistency and cross-model agreement for validation rather than exact numerical benchmarks.

Evidence (verbatim from paper)

The RMSE and maximum probability difference (Error_max) between the MC and our closed-form solutions shown in Fig. 5d and a convergence curve in Fig. 5e confirm the correctness of our derivations and algorithms.

Citation

@misc{athawale2024uncertaintyvisualization,
  title={Uncertainty Visualization of Critical Points of 2D Scalar Fields for Parametric and Nonparametric Probabilistic Models},
  author={Athawale et al. (2024)},
  year={2024},
  note={arXiv:2407.18015}
}

• arXiv: 2407.18015