Matq Eval

matq-eval

A Fast, Accurate, and Reactive Equivariant Foundation Potential — Ko et al. (2025) (arXiv:2511.07249, 2025)

What this evaluates

Evaluates the accuracy of a charge-equilibrated equivariant foundation potential in predicting atomic energies, forces, partial charges, and bulk mechanical/thermal properties across diverse crystalline, molecular, and ionic systems.

Datasets

• MatQ — total 120000; splits: test (-1)
• Custom Model Systems (C10H2/C10H3+, Ag3+/−, Na8/9Cl8+, Au2-MgO(001)) — total ?; splits: test (-1)

Metrics

• RMSE (primary) — range: other
  • Root mean squared error calculated as sqrt(mean((predicted - reference)^2)). Applied separately to energies, forces, and charges.
• MAE (primary) — range: other
  • Mean absolute error calculated as mean(|predicted - reference|). Applied to energies, forces, stresses, charges, bulk modulus, shear modulus, and heat capacity.

Input / output format

Input: Atomic numbers (Z_i), atomic positions (R_i), and total system charge (Q_tot).

Output: Atomic energies (E_i), atomic forces, atomic partial charges (q_i), and total energy.

Scoring recipe

import numpy as np
def compute_rmse(pred, gold):
    return np.sqrt(np.mean((pred - gold) ** 2))
def compute_mae(pred, gold):
    return np.mean(np.abs(pred - gold))
# Apply per property (energy, force, charge, stress, modulus, heat capacity)
# Reference values are DFT-computed (PBE) or DDEC6 partitioned charges.

Common pitfalls

• Comparing models trained on vastly different dataset sizes (e.g., MatQ ~120k vs OMat24 ~100M) without accounting for data scale effects on performance.
• Using traditional Ewald summation for long-range electrostatics in benchmarks, which contradicts the model's linear-scaling cutoff-based design and inflates computational cost.
• Confusing DDEC6 partitioned charges with other schemes like Hirshfeld or Bader, which yield different absolute values and should not be mixed in error calculations.

Evidence (verbatim from paper)

The test MAEs in energies, forces and stresses of the QET-MatQ FP are 33meV atoms−1, 108meV Å−1 and 0.452 GPa, respectively. The MAE of the charges of the QET-MatQ FP is approximately 36.4 me, which is only slightly larger than the MAEs for the custom QET MLIPs for the model systems in Table 1.

Citation

@misc{ko2025qet,
  title={A Fast, Accurate, and Reactive Equivariant Foundation Potential},
  author={Ko et al. (2025)},
  year={2025},
  note={arXiv:2511.07249}
}

• arXiv: 2511.07249