Wind Power Ensemble Eval

wind-power-ensemble-eval

Evaluating Ensemble Post-Processing for Wind Power Forecasts — Phipps et al. (2020) (arXiv:2009.14127, 2020)

What this evaluates

Evaluates the calibration, sharpness, and accuracy of probabilistic wind power forecasts under different ensemble post-processing strategies (raw, weather-only, power-only, and joint weather-power post-processing). It probes whether correcting biases at the weather stage alone is sufficient, or if direct post-processing of the final power ensemble is required to handle non-linear power curve biases.

Datasets

• Benchmark Data — total ?; splits: train (-1), test (-1); repo https://github.com/KIT-IAI/EvaluatingEnsemblePostProcessing
• Swedish Data Set — total ?; splits: train (-1), test (-1); repo https://github.com/KIT-IAI/EvaluatingEnsemblePostProcessing

Metrics

• CRPS (primary) — range: other (MW)
  • Continuous Ranked Probability Score; measures the integrated squared difference between the predictive cumulative distribution function and the Heaviside step function at the observed value. Lower values indicate better accuracy.
• CRPSS — range: percent
  • CRPS Skill Score; calculated as 1 - (CRPS_postprocessed / CRPS_raw), where CRPS_raw is the baseline raw ensemble score. Positive values indicate improvement over the raw ensemble.
• PIT histogram — range: [0, 1]
  • Probability Integral Transform histogram; plots the cumulative probability of the observed value under the predictive distribution. A well-calibrated forecast yields a uniform histogram on [0, 1].

Input / output format

Input: Ensemble weather forecasts (e.g., TIGGE or ECMWF EPS variables like wind speed, temperature, pressure) and historical wind power generation data.

Output: Probabilistic wind power forecasts represented as predictive distributions (e.g., truncated normal) for specified forecast horizons (3h to 24h).

Scoring recipe

def compute_crps(forecast_cdf, observation):
    return integral((forecast_cdf(x) - (x >= observation))**2 dx)

def compute_crpss(crps_postprocessed, crps_raw):
    return 1.0 - (crps_postprocessed / crps_raw)

def compute_pit_histogram(forecast_cdf, observations):
    pits = [forecast_cdf(obs) for obs in observations]
    return histogram(pits, bins=10)

Common pitfalls

• Post-processing weather ensembles alone does not consistently improve wind power forecast calibration because the non-linear wind power curve introduces additional biases that require direct power-level correction.
• CRPSS is calculated relative to the raw ensemble baseline, not a deterministic or climatology baseline, so negative values indicate worse performance than raw ensembles.
• The benchmark dataset uses synthetically generated wind power data, which may not capture real-world structural changes like capacity upgrades or maintenance outages present in the Swedish dataset.

Evidence (verbatim from paper)

We first perform the different post-processing strategies on the benchmark data set and evaluate the calibration and sharpness based on the above introduced CRPS and PIT/ verification rank histogram for both forecasting models with varying forecast horizon.

Citation

@misc{phipps2020evaluating,
  title={Evaluating Ensemble Post-Processing for Wind Power Forecasts},
  author={Phipps et al. (2020)},
  year={2020},
  note={arXiv:2009.14127}
}

• arXiv: 2009.14127