Kitchen Sink Anomaly Detection Eval

kitchen-sink-anomaly-detection-eval

Kitchen Sink Anomaly Detection — Das et al. (2026) (arXiv:2604.20965, 2026)

What this evaluates

Evaluates the robustness and sensitivity of various jet substructure feature sets (Energy Flow Polynomials, subjettiness, and their combination) for model-agnostic resonant anomaly detection in high-energy physics dijet events. It compares performance across Ideal Anomaly Detection (IAD) and CWoLa hunting setups using multiple Beyond Standard Model signal topologies.

Datasets

• LHCO & BSM dijet signals — total ?; splits: train (-1), test (-1)

Metrics

• max(SIC) (primary) — range: other
  • Maximum Significance Improvement Characteristic; measures the peak discovery significance improvement over the background-only expectation across varying signal injections ($N_{sig}$).
• sigma_0,min (primary) — range: other
  • Minimal initial signal significance required to achieve a 3σ evidence or 5σ discovery threshold, derived by linearly interpolating the median significance curve across 10 training runs.
• regret (r_f) — range: other
  • Relative performance gap of a feature set compared to the best-performing set, calculated as (metric_best - metric_f) / metric_best.

Input / output format

Input: High-level jet substructure feature vectors (e.g., EFPs, N-subjettiness, baseline observables) computed for dijet events, optionally labeled as signal or background.

Output: Classifier decision scores or binary predictions from Gradient Boosted Decision Trees (GBDTs), used to compute discovery significance metrics.

Scoring recipe

# For each of 10 training samples:
#   train GBDT on features, predict on test set
#   compute Asimov significance for each signal injection N_sig
#   take max significance -> SIC
# Compute median SIC across 10 runs and interpolate
# Find initial significance where interpolated SIC >= 3σ or 5σ -> sigma_0,min
# Regret r_f = (metric_best - metric_f) / metric_best

Common pitfalls

• Confusing the IAD setup (idealized, known signal distribution for training) with CWoLa hunting (unlabeled data, relies on background-to-signal ratio estimation).
• Assuming the 'Random' feature set represents random noise; it actually refers to random subsampling of features per tree to reduce training cost while maintaining performance.
• Overlooking that metrics are aggregated across multiple distinct BSM signal topologies, so a feature set's ranking can vary significantly depending on the specific decay topology (e.g., 2-prong vs. 4-prong).

Evidence (verbatim from paper)

For a 5σ discovery, we present the average and maximum of the minimal initial significance eq. (4) and regret eq. (5) for the different feature sets, over the investigated signal models in Table 3.

Citation

@misc{das2026kitchensink,
  title={Kitchen Sink Anomaly Detection},
  author={Das et al. (2026)},
  year={2026},
  note={arXiv:2604.20965}
}

• arXiv: 2604.20965