qhjqhj00 jet-tagging-resilience-eval
The Pareto Frontier of Resilient Jet Tagging — Gambhir et al. (2025) (arXiv:2509.19431, 2025)
What this evaluates
Evaluates the trade-off between classification performance (AUC) and model resilience (robustness to Monte Carlo simulation variations) in quark/gluon and top-quark jet tagging. It probes whether complex neural architectures generalize better to different physics simulators compared to simpler, physics-informed models.
Datasets
- Pythia 8 / Herwig 7 Jet Samples — total ?; splits: train (-1), test (-1)
Metrics
AUC(primary) — range: [0, 1]- Area Under the Receiver Operating Characteristic Curve, measuring binary classification performance across all decision thresholds.
Resilience— range: percent- Percent difference in AUC between models evaluated on Pythia-generated test samples versus Herwig-generated test samples. Calculated as (AUC_Pythia - AUC_Herwig) / AUC_Pythia * 100.
Input / output format
Input: Jet event features or raw particle data (e.g., angularities, energy-flow maps, or particle lists) representing quark/gluon or top/bottom/light jets.
Output: Binary classification probability or score indicating the likelihood of a jet belonging to a specific flavor (e.g., quark vs. gluon, or top vs. background).
Scoring recipe
auc_pythia = roc_auc_score(y_true_pythia, y_pred_pythia)
auc_herwig = roc_auc_score(y_true_herwig, y_pred_herwig)
resilience_pct = (auc_pythia - auc_herwig) / auc_pythia * 100
return {'AUC': auc_pythia, 'Resilience': resilience_pct}
Common pitfalls
- Assuming higher AUC on a single simulator guarantees better physics analysis results, ignoring simulation dependence.
- Treating resilience as a fixed property rather than a trade-off; the Pareto frontier shows no model maximizes both AUC and resilience simultaneously.
- Expecting knowledge distillation to break the performance-resilience trade-off, as the study shows it fails to surpass the frontier.
Evidence (verbatim from paper)
Figure[1] shows the classifier AUC vs. resilience (AUC %-difference for Pythia vs. Herwig samples), for each of the models trained and for each set of hyperparameters considered: ‘optimal’ performance is at the lower-right corner of the figure. The Pareto frontier connecting models that optimize the AUC-resiliency tradeoff is highlighted, and models in the shaded region are Pareto-excluded.
Citation
@misc{gambhir2025pareto,
title={The Pareto Frontier of Resilient Jet Tagging},
author={Gambhir et al. (2025)},
year={2025},
note={arXiv:2509.19431}
}
- arXiv: 2509.19431