uMLIP-high-temp-mof-eval

uMLIP-high-temp-mof-eval

Benchmarking Universal Machine-Learned Interatomic Potentials for High-Temperature Metal-Organic Framework Chemistry — Edwards et al. (2026) (arXiv:2604.25262, 2026)

What this evaluates

Evaluates the accuracy of universal machine-learned interatomic potentials (uMLIPs) in predicting energy, forces, and stress tensors during high-temperature molecular dynamics simulations of metal-organic frameworks (MOFs), including their stability and thermal decomposition behavior.

Datasets

• High-Temperature MOF AIMD Benchmark — total 9; splits: test (9)

Metrics

• energy MAE (primary) — range: other
  • Mean absolute error between predicted and reference (AIMD) energy values, reported in meV/atom.
• force MAE — range: other
  • Mean absolute error between predicted and reference (AIMD) force vectors, reported in meV/Å.
• stress MAE — range: other
  • Mean absolute error between predicted and reference (AIMD) stress tensors, reported in MPa.

Input / output format

Input: Atomic configurations of nine zinc- and zirconium-based MOFs at 300, 1000, and 2000 K, extracted from 40 ps ab initio molecular dynamics (AIMD) trajectories.

Output: Predicted scalar energy (meV/atom), force vectors (meV/Å), and stress tensors (MPa) for each configuration.

Scoring recipe

def compute_mae(predictions, references):
    return np.mean(np.abs(predictions - references))
# Applied independently to energy, force, and stress components.
# Final metric = mean across all 9 MOFs and 3 temperatures.

Common pitfalls

• Static validation metrics (MAE) significantly underestimate errors that accumulate during long-timescale dynamic simulations (generative errors).
• Force and stress predictions are consistently less accurate than energy predictions, even near equilibrium, which can mislead evaluations relying solely on energy MAE.
• Model rankings on this benchmark correlate with MATBench, but performance is highly system-dependent, with some MOFs showing systematically higher errors regardless of the model.

Evidence (verbatim from paper)

The error was broken down into energy, force and stress components and averaged across all MOFs and temperatures (Table 1). ORB-v3 and fairchem OMAT show low energy mean absolute error (MAE) at 3.59 and 3.21 meV atom-1, respectively, while MACE models and fairchem ODAC23 show errors exceeding 10 meV atom-1.

Citation

@misc{edwards2026benchmarking,
  title={Benchmarking Universal Machine-Learned Interatomic Potentials for High-Temperature Metal-Organic Framework Chemistry},
  author={Edwards et al. (2026)},
  year={2026},
  note={arXiv:2604.25262}
}

• arXiv: 2604.25262