# Neuralgcm Climate Eval

> Evaluates the model's ability to simulate historical global temperature trends and spatial temperature biases over multi-decadal climate simulations, and tests its generalization to warmer climate scenarios. It probes long-term stability, physical consistency, and response to prescribed sea surface temperature forcing. Use when the user wants to benchmark on ERA5, or asks about evaluating this task. Reports RMSB (850hPa temperature).

- Skill: `qhjqhj00/neuralgcm-climate-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/neuralgcm-climate-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/neuralgcm-climate-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/neuralgcm-climate-eval

---


# neuralgcm-climate-eval

> Neural General Circulation Models for Weather and Climate — Kochkov et al. (2023) (arXiv:2311.07222, 2023)

## What this evaluates

Evaluates the model's ability to simulate historical global temperature trends and spatial temperature biases over multi-decadal climate simulations, and tests its generalization to warmer climate scenarios. It probes long-term stability, physical consistency, and response to prescribed sea surface temperature forcing.

## Datasets

- **ERA5** — total ?; splits: test (-1)

## Metrics

- `RMSB (850hPa temperature)` **(primary)** — range: other
  - Root Mean Square Bias calculated as the square root of the mean squared difference between predicted and observed 850hPa temperatures, averaged over the 1981-2014 period.

## Input / output format

**Input**: Initial atmospheric state and prescribed sea surface temperatures (SST) for AMIP-like simulations.

**Output**: Time-series of 3D atmospheric temperature fields, specifically aggregated to 850hPa temperature and vertical tropical profiles.

## Scoring recipe

```python
def compute_rmsb(predictions, observations):
    # predictions, observations: arrays of 850hPa temperature over 1981-2014
    diff = predictions - observations
    rmsb = np.sqrt(np.mean(diff ** 2))
    return rmsb
```

## Common pitfalls

- Analysis is restricted to only the 22 out of 37 runs that remained stable for the full 40-year period; unstable trajectories are excluded from the main evaluation.
- Baseline comparisons shift significantly when global temperature bias is removed from CMIP6 runs (AMIP vs AMIP*), affecting relative performance claims.
- Generalization to +4K SST scenarios exhibits climate drift and diverges from expectations, limiting robustness claims for extreme warming.

## Evidence (verbatim from paper)

> When comparing spatial biases averaged over 1981-2014 we find that all 22 NeuralGCM- 2.8° runs have smaller bias than CMIP6 AMIP runs... (c) The root mean square bias (RMSB) of the 850hPa temperature averaged between 1981-2014 for 22 NeuralGCM- 2.8° AMIP runs (labeled NGCM), 22 CMIP6 AMIP experiments (labeled AMIP) and debiased 22 CMIP6 AMIP experiments (labeled as AMIP*).

## Citation

```bibtex
@misc{kochkov2023neuralgcm,
  title={Neural General Circulation Models for Weather and Climate},
  author={Kochkov et al. (2023)},
  year={2023},
  note={arXiv:2311.07222}
}
```

- arXiv: 2311.07222