Somoml Eval

somoml-eval

Global soil moisture from in-situ measurements using machine learning -- SoMo.ml — Sungmin O et al. (2020) (arXiv:2010.02374, 2020)

What this evaluates

Evaluates a model's ability to extrapolate daily soil moisture dynamics across three depth layers using in-situ ground measurements and meteorological forcing. It probes temporal fidelity and absolute accuracy against independent station data.

Datasets

• ISMN & CEMADEN in-situ soil moisture measurements — total ?; splits: train (-1), test (-1)

Metrics

• NRMSE (primary) — range: other
  • Root-mean-square error (RMSE) divided by the mean of the ground truth values.
• anomaly correlation coefficient — range: [0, 1]
  • Pearson correlation between soil moisture anomalies computed over a 5-week window (P = [d-17, d+17]) for predictions and ground truth.

Input / output format

Input: Daily time series of meteorological forcing from t-364 to t, plus static spatial features (precipitation, aridity, elevation, soil type, land cover).

Output: Daily soil moisture values at 0.25° spatial resolution for three depth layers (0–10 cm, 10–30 cm, 30–50 cm).

Scoring recipe

def compute_nrmse(pred, gold):
    rmse = np.sqrt(np.mean((pred - gold) ** 2))
    return rmse / np.mean(gold)

def compute_anomaly_corr(pred, gold, window=17):
    pred_anom = rolling_mean(pred, window) - np.mean(pred)
    gold_anom = rolling_mean(gold, window) - np.mean(gold)
    return np.corrcoef(pred_anom, gold_anom)[0, 1]

Common pitfalls

• Reference datasets (ESA-CCI, GLEAM) have missing values in space and time, complicating direct comparison.
• In-situ sensor calibrations vary across stations, requiring mean/std adjustment to ERA5 before training/evaluation.
• Model performance degrades in data-sparse hydro-climatic regions (e.g., very arid or high-latitude areas) due to lack of training observations.

Evidence (verbatim from paper)

Cross-validation (5-fold) is made through a direct grid-to-point comparison between the SoMo.ml* and the in-situ measurements as done in many previous studies[42-46]. ... Established skill scores such as normalised root-mean-square error (NRMSE), relative bias, and correlation coefficient are used to quantify the agreement with the ground truth data. NRMSE is defined as the RMSE divided by the means of ground truth.

Citation

@misc{o2020somoml,
  title={Global soil moisture from in-situ measurements using machine learning -- SoMo.ml},
  author={Sungmin O et al. (2020)},
  year={2020},
  note={arXiv:2010.02374}
}

• arXiv: 2010.02374