# Uncertainty

> Evaluates a CNN's ability to predict stellar atmospheric parameters and chemical abundances from low-resolution spectra, measuring both internal consistency across model runs and agreement with established spectroscopic pipeline measurements. Use when the user has predictions and gold and needs to compute Uncertainty.

- Skill: `qhjqhj00/uncertainty` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/uncertainty`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/uncertainty/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: DevOps & Infra
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/uncertainty

---


# uncertainty

> The Gaia-ESO Survey: Chemical evolution of Mg and Al in the Milky Way with Machine-Learning — Ambrosch et al. (2022) (arXiv:2208.08872, 2022)

## What this evaluates

Evaluates a CNN's ability to predict stellar atmospheric parameters and chemical abundances from low-resolution spectra, measuring both internal consistency across model runs and agreement with established spectroscopic pipeline measurements.

## Datasets

- **Gaia-ESO Survey (GES)** — total 30000; splits: train (-1), test (-1), observed (-1)

## Metrics

- `Uncertainty` **(primary)** — range: K or dex
  - Standard deviation of predictions across eight independently trained CNN models with different random weight initializations.
- `Bias & Scatter` — range: K or dex
  - Mean difference and standard deviation around the 1:1 relation between CNN predictions and Gaia-ESO pipeline measurements.

## Input / output format

**Input**: GIRAFFE low-resolution stellar spectra (flux as a function of wavelength).

**Output**: Five continuous values: T_eff (K), log g (dex), [Mg/Fe] (dex), [Al/Fe] (dex), [Fe/H] (dex).

## Scoring recipe

```python
# preds: list of 8 model predictions per star
# gold: GES pipeline measurements
internal_uncertainty = np.std(preds, axis=0)
bias = np.mean(preds[0] - gold)
scatter = np.std(preds[0] - gold)
```

## Common pitfalls

- Internal uncertainty measures model initialization variance, not data noise or true prediction error.
- Performance degrades significantly for low S/N spectra (<30) and extreme abundance values due to limited training data.
- Strong label correlations in training data can cause the network to learn spurious correlations instead of physical spectral features.

## Evidence (verbatim from paper)

> We define the uncertainties of our results as the dispersion between the label predictions from the eight CNN models. In Fig. 8 we display the distribution of the label uncertainties σ(Label) relative to the predicted label values of our five labels. ... The mean uncertainties of the label predictions are small: 24K for T_eff, 0.03 for log(g), 0.02 dex for [Mg/Fe], 0.03 dex for [Al/Fe], and 0.02 dex for [Fe/H].

## Citation

```bibtex
@misc{ambrosch2022gaia,
  title={The Gaia-ESO Survey: Chemical evolution of Mg and Al in the Milky Way with Machine-Learning},
  author={Ambrosch et al. (2022)},
  year={2022},
  note={arXiv:2208.08872}
}
```

- arXiv: 2208.08872