Remote Sensing Eval

remote-sensing-eval

In-domain representation learning for remote sensing — Neumann et al. (2019) (arXiv:1911.06721, 2019)

What this evaluates

Evaluates in-domain representation learning and scene classification across diverse remote sensing modalities (optical, SAR, aerial) and spatial resolutions. It probes model robustness to varying class balances, visual similarities, and label domains in Earth observation.

Datasets

• BigEarthNet — total 590326; splits: train (-1), val (-1), test (-1)
• EuroSAT — total 27000; splits: train (-1), val (-1), test (-1)
• RESISC-45 — total 31500; splits: train (-1), val (-1), test (-1)
• So2Sat — total 376000; splits: train (-1), val (-1), test (-1)
• UC Merced — total 2100; splits: train (-1), val (-1), test (-1)

Metrics

• accuracy (primary) — range: [0, 1]
  • Fraction of correctly predicted class labels out of total samples. For multi-label datasets, per-sample accuracy is computed as the ratio of correctly predicted labels to total labels.

Input / output format

Input: Remote sensing image patches (Sentinel-2 multi-spectral, Sentinel-1 SAR, or aerial RGB) with varying spatial resolutions (0.2–60 m) and pixel dimensions (32x32 to 256x256).

Output: Class label(s) corresponding to land-use/land-cover, urban structures, or ecological zones (multi-class or multi-label).

Scoring recipe

correct = 0
total = 0
for pred, gold in zip(predictions, gold_labels):
    if isinstance(gold, list):
        if set(pred) == set(gold): correct += 1
    else:
        if pred == gold: correct += 1
    total += 1
return correct / total

Common pitfalls

• Datasets have highly varying spatial resolutions and spectral bands, requiring careful preprocessing or architecture choices to avoid resolution bias.
• Label imbalance is severe in BigEarthNet, making standard accuracy misleading without macro-averaging or threshold tuning.
• So2Sat uses a non-standard split derived from the original validation set (25% val, 75% test), which differs from the 60/20/20 ratio used for other datasets.

Evidence (verbatim from paper)

For reproducability and a common evaluation framework, standard train, validation, and test splits using the 60%, 20%, and 20% ratios, respectively, were generated for all datasets except So2Sat. ... Because the classes are quite distinctive, very high accuracies can be achieved when using the entire dataset for training.

Citation

@misc{neumann2019indomain,
  title={In-domain representation learning for remote sensing},
  author={Neumann et al. (2019)},
  year={2019},
  note={arXiv:1911.06721}
}

• arXiv: 1911.06721