Xas Os Cn Prediction Eval

xas-os-cn-prediction-eval

Advancing AI-Driven Analysis in X-ray Absorption Spectroscopy: Spectral Domain Mapping and Universal Models — Cao et al. (2025) (arXiv:2510.15167, 2025)

What this evaluates

Evaluates a machine learning model's ability to predict local chemical descriptors (oxidation state and coordination number) from experimental X-ray absorption spectra, specifically testing how well spectral domain mapping bridges the gap between simulated training data and real experimental measurements.

Datasets

• Combinatorial Zinc Titanate Thin Film XANES — total 100; splits: test (100)

Metrics

• OS/CN prediction accuracy (primary) — range: other
  • Predicted OS and CN are reported as the mean of an ensemble of 32 neural networks, with standard deviation as error. Evaluation compares the predicted mean values and spatial trends against expected physical ground truth (e.g., constant OS of 4.0 for Ti4+ under oxygen-rich conditions, and CN trend 1-P_UD derived from first-principles calculations).

Input / output format

Input: 1D vector of absorption coefficients across a common energy grid (Ti K-edge XANES spectra), down-sampled.

Output: Two real-valued numbers per spectrum: predicted oxidation state (OS) and coordination number (CN), along with ensemble standard deviation.

Scoring recipe

def evaluate(predictions, gold_os, gold_cn_trend):
    # predictions: list of (os_pred, cn_pred) per spectrum
    # gold_os: float (expected constant OS, e.g., 4.0)
    # gold_cn_trend: array of expected CN values per position
    mean_os = sum(p[0] for p in predictions) / len(predictions)
    os_error = abs(mean_os - gold_os)
    cn_correlation = pearsonr([p[1] for p in predictions], gold_cn_trend)[0]
    return {'os_error': os_error, 'cn_correlation': cn_correlation}

Common pitfalls

• Models trained purely on simulated spectra fail to generalize to experimental data due to distribution shift (edge position shifts, peak height differences).
• Outputs are continuous averages of local motifs, not discrete site-specific labels, requiring careful physical interpretation.
• Evaluation relies heavily on qualitative trend matching against first-principles ground truth rather than a single scalar metric.

Evidence (verbatim from paper)

The predicted CN and OS from using the SDM-transformed spectra as input are also shown in Figure[5]. The predicted CN trends are similar with or without SDM, and both show qualitative agreement with the expected trend (e.g. the shape of the curve) obtained in Ref.[9]. However, SDM qualitatively changes RankAAE’s OS prediction. Without the SDM, RankAAE predicts that OS decreases sharply in the low f_Ti region, and the resulting broad distribution of OS contradicts the fact that Ti sites should have a constant OS under the oxygen-rich synthesis conditions.

Citation

@misc{cao2025xas,
  title={Advancing AI-Driven Analysis in X-ray Absorption Spectroscopy: Spectral Domain Mapping and Universal Models},
  author={Cao et al. (2025)},
  year={2025},
  note={arXiv:2510.15167}
}

• arXiv: 2510.15167