Quark Gluon Jet Discrimination Eval

quark-gluon-jet-discrimination-eval

Quark-Gluon Jet Discrimination Using Convolutional Neural Networks — Lee et al. (2020) (arXiv:2012.02531, 2020)

What this evaluates

This benchmark evaluates a model's ability to classify high-energy physics particle jets as originating from quarks or gluons using pixelized detector data. It probes feature extraction and binary classification performance across different input channel configurations and jet transverse momentum ranges.

Datasets

• Simulated CMS LHC Jet Data (DELPHES) — total 100000; splits: (unstated)

Metrics

• AUC (primary) — range: [0, 1]
  • Area Under the Receiver Operating Characteristic Curve, measuring the trade-off between quark jet efficiency and gluon jet efficiency across all classification thresholds.
• Gluon efficiency at fixed quark efficiency — range: percent
  • The fraction of gluon jets correctly identified ($\epsilon_g$) when the quark jet identification efficiency ($\epsilon_q$) is fixed at 50% or 70%.

Input / output format

Input: Pixelized jet images in (η,φ) space of size 33×33, with 3 to 10 channels representing transverse momentum ($p_T$) and particle multiplicity for categories like charged hadrons, neutral hadrons, electrons, muons, and photons.

Output: Binary classification score/probability indicating quark vs. gluon jet origin.

Scoring recipe

def compute_metrics(y_true, y_score):
    auc = roc_auc_score(y_true, y_score)
    efficiencies = {}
    for target_q in [0.5, 0.7]:
        thresh = find_threshold(y_true[y_true==1], y_score[y_true==1], target_q)
        g_eff = recall_score(y_true[y_true==0], y_score[y_true==0], pos_label=0, threshold=thresh)
        efficiencies[f'gluon_eff_at_q{int(target_q*100)}'] = g_eff
    return auc, efficiencies

Common pitfalls

• Confusing particle-level simulation data with detector-level (DELPHES) data, which have different noise and resolution characteristics.
• Failing to account for the strong dependence of classification performance on the jet transverse momentum ($p_T$) range, as performance degrades at higher $p_T$.
• Overlooking that channel configuration (3 vs 10 channels) significantly impacts performance, with saturation observed beyond 6 channels.

Evidence (verbatim from paper)

The various convolutional networks were trained on 10-channel samples with $p_{T}$ in from 100 to 200 GeV, and the resulting test AUC distributions are shown in Figure[2]... Table[1], shows our results obtained from detector level events produced with DELPHES as previously described. For comparison, we also show the efficiency previously reported at particle level... Gluon jet efficiencies at 50% and 70% quark jet efficiencies on the test set for a 200 GeV jet.

Citation

@misc{lee2020quarkgluon,
  title={Quark-Gluon Jet Discrimination Using Convolutional Neural Networks},
  author={Lee et al. (2020)},
  year={2020},
  note={arXiv:2012.02531}
}

• arXiv: 2012.02531