Jet Classification Eval

jet-classification-eval

FastML Science Benchmarks: Accelerating Real-Time Scientific Edge Machine Learning — Duarte et al. (2022) (arXiv:2207.07958, 2022)

What this evaluates

Evaluates ultra-low-latency supervised classification of particle physics jet signatures on edge hardware. It probes the ability to distinguish rare boson/top-quark jets from common quark/gluon jets under strict microsecond latency and pipeline interval constraints.

Datasets

• LHC Jet Classification Dataset — total ?; splits: test (-1); repo https://github.com/fastmachinelearning/fastml-science

Metrics

• classification accuracy (primary) — range: percent
  • Percentage of correctly classified jet events out of the total test set.
• FPR at TPR of 50% — range: [0, 1]
  • False positive rate measured at the operating point where the true positive rate for Z jets is exactly 50%.

Input / output format

Input: Either 16 expert-designed features (16-bit fixed-point) or a point cloud of up to 100 particles represented by their 3D momentum vectors (px, py, pz) quantized to 16-bit integers.

Output: 5-class probability distribution or hard label over {light quark (q), gluon (g), top quark (t), W boson, Z boson}.

Scoring recipe

# For accuracy
accuracy = sum(pred == gold for pred, gold in zip(predictions, labels)) / len(labels)

# For FPR@TPR50
# 1. Compute TPR and FPR across classification thresholds for Z jet class
# 2. Interpolate to find FPR where TPR == 0.5
# 3. Return interpolated FPR value

Common pitfalls

• Latency must be measured end-to-end including pipeline interval, not just raw inference time.
• Point cloud inputs require zero-padding to exactly 100 particles and specific 16-bit quantization (LSB=0.0625 GeV, range ±2048 GeV).
• The benchmark requires FPGA synthesis constraints (DSPs, LUTs, FFs) to be reported alongside algorithmic metrics.

Evidence (verbatim from paper)

The two we will focus on are (1) classification accuracy and (2) FPR at TPR of 50% for the signal being Z jet

Citation

@misc{duarte2022fastml,
  title={FastML Science Benchmarks: Accelerating Real-Time Scientific Edge Machine Learning},
  author={Duarte et al. (2022)},
  year={2022},
  note={arXiv:2207.07958}
}

• arXiv: 2207.07958