# AUC

> Evaluates machine learning classifiers on their ability to distinguish signal from background in particle physics simulations, measuring how well algorithms rank signal events above background ones using the AUC metric.

- Skill: `qhjqhj00/auc` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/auc`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/auc/raw
- Safety review: PASS (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML, Data & Analytics, Model Training & Fine-tuning
- Tags: Auc, Classification, Evaluation, Machine Learning, Particle Physics, Roc Curve
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-08-22
- Page: https://skillmd.com/skills/qhjqhj00/auc

---


# AUC

> How to use Machine Learning to improve the discrimination between signal and background at particle colliders — Cid Vidal et al. (2021) (arXiv:2110.15099, 2021)

## What this evaluates

This evaluation probes the capability of machine learning classifiers to distinguish signal events from background noise in high-energy particle physics simulations. It measures how well different algorithms and feature sets rank positive (signal) instances higher than negative (background) ones across varying data statistics.

## Datasets

- **LHCb B-meson decay simulation** — total ?; splits: train (-1), test (-1)

## Metrics

- `AUC` **(primary)** — range: [0, 1]
  - Area Under the Receiver Operating Characteristic curve. Computed by integrating the true positive rate against the false positive rate across all classification thresholds.

## Input / output format

**Input**: Numerical feature vectors representing particle decay events, provided at low-level, high-level, or combined feature sets.

**Output**: Continuous probability score or decision function value indicating the likelihood of the event being signal.

## Scoring recipe

```python
def compute_auc(y_true, y_pred_scores):
    fpr, tpr, _ = roc_curve(y_true, y_pred_scores)
    return auc(fpr, tpr)
```

## Common pitfalls

- AUC can mask poor calibration or suboptimal performance at specific operating points relevant to physics analyses.
- Adding more features or statistics does not guarantee higher AUC due to redundancy or statistical fluctuations in simulations.
- Comparisons across libraries must control for hyperparameter tuning, as default settings may not yield optimal discrimination.

## Evidence (verbatim from paper)

> Once we have calculated the ROC curves and obtained the AUC scores for each of the algorithms, we proceed to compare them all for each channel.

## Citation

```bibtex
@misc{cidvidal2021signalbackground,
  title={How to use Machine Learning to improve the discrimination between signal and background at particle colliders},
  author={Cid Vidal et al. (2021)},
  year={2021},
  note={arXiv:2110.15099}
}
```

- arXiv: 2110.15099