Dvcs Cff Extraction Eval

dvcs-cff-extraction-eval

Benchmarks for a Global Extraction of Information from Deeply Virtual Exclusive Scattering — Almaeen et al. (2022) (arXiv:2207.10766, 2022)

What this evaluates

This benchmark evaluates machine learning models' ability to extract Compton Form Factors (CFFs) from deeply virtual Compton scattering (DVCS) cross-section data. It specifically probes how well models adhere to quantum chromodynamics (QCD) constraints, generalize across kinematic regions, and accurately quantify both aleatoric and epistemic uncertainties during the extraction process.

Datasets

• DVCS unpolarized proton target data — total ?; splits: train (-1), test (-1)

Metrics

• predictive uncertainty (primary) — range: other
  • Measures whether the underlying physics ground truth lies within the predicted error bands, quantifying aleatoric and epistemic uncertainties propagated through the ML training process using methods like Monte Carlo dropout or ensemble variance.
• CFFs comparison — range: other
  • Compares extracted CFF values at set kinematic points against theoretical predictions or experimental data to assess fitting accuracy and theoretical tension across different cross-section formalisms.

Input / output format

Input: Kinematic variables and measured cross-section values (and asymmetries) from DVES processes, specifically unpolarized scattering off an unpolarized proton target.

Output: Extracted Compton Form Factors (CFFs) along with propagated statistical error bands/uncertainty estimates.

Scoring recipe

def evaluate_cff_extraction(predictions, gold_cffs, predicted_uncertainties):
    # Check if ground truth lies within predicted error bands
    in_bands = np.all((gold_cffs >= predicted_uncertainties[:, 0]) & 
                      (gold_cffs <= predicted_uncertainties[:, 1]))
    # Compare extracted CFFs to theoretical/experimental values
    mse = np.mean((predictions - gold_cffs) ** 2)
    return {'coverage': float(in_bands), 'mse': float(mse)}

Common pitfalls

• Confusing theoretical tensions from different cross-section formalisms or kinematic dependencies with actual model performance.
• Failing to properly disentangle irreducible data-dependent uncertainties from model-dependent training errors.
• Overlooking the $Q^2$ dependence and PQCD evolution when generalizing extracted CFFs to regions outside the training data.

Evidence (verbatim from paper)

The predictive uncertainty of the deep ML models is of particular interest, which is the result of data- and model-dependent uncertainties propagated through the ML training process at different stages. ... We list below a standard set of metrics with which comparisons of CFFs in literature can be done.

Citation

@misc{almaeen2022benchmarks,
  title={Benchmarks for a Global Extraction of Information from Deeply Virtual Exclusive Scattering},
  author={Almaeen et al. (2022)},
  year={2022},
  note={arXiv:2207.10766}
}

• arXiv: 2207.10766