qhjqhj00 pastis-hd-eval
OmniSat: Self-Supervised Modality Fusion for Earth Observation — Astruc et al. (2024) (arXiv:2404.08351, 2024)
What this evaluates
Tests agricultural land cover mapping and crop-type classification by evaluating models on high-resolution satellite imagery combined with optical and radar time series.
Datasets
- PASTIS-HD — total 2433; splits: train (-1), test (-1); repo https://github.com/gastruc/OmniSat
Metrics
macro-averaged F1-score(primary) — range: percent- Unweighted mean of per-class F1 scores, treating all 18 crop types equally regardless of class frequency.
Input / output format
Input: Satellite patches of 1280x1280m containing Sentinel-2 optical time series, Sentinel-1 radar time series, and contemporary VHR SPOT 6-7 images.
Output: Multi-class classification label indicating one of 18 crop types per patch.
Scoring recipe
def compute_macro_f1(gold, pred):
from sklearn.metrics import f1_score
return f1_score(gold, pred, average='macro', zero_division=0) * 100
Common pitfalls
- Ignoring the irregular cloud occlusion in optical time series, which requires robust date-filtering or masking.
- Evaluating on single-date images instead of the full time series, missing critical temporal dynamics.
Evidence (verbatim from paper)
We report the macro-averaged F1-score for crop-type multi-class classification on the PASTIS-HD dataset.
Citation
@misc{astruc2024omnisat,
title={OmniSat: Self-Supervised Modality Fusion for Earth Observation},
author={Astruc et al. (2024)},
year={2024},
note={arXiv:2404.08351}
}
- arXiv: 2404.08351