Geobench Eval

geobench-eval

GeoMeld: Toward Semantically Grounded Foundation Models for Remote Sensing — Hasan et al. (2026) (arXiv:2604.10591, 2026)

What this evaluates

Evaluates the transferability and semantic grounding of remote sensing foundation models on image-level classification and pixel-level semantic segmentation tasks across multiple geospatial benchmarks.

Datasets

• GeoBench — total ?; splits: test (-1)

Metrics

• F1 (primary) — range: [0, 1]
  • Harmonic mean of precision and recall, computed per class and averaged.
• Accuracy — range: [0, 1]
  • Ratio of correctly predicted samples to total samples.
• IoU — range: [0, 1]
  • Intersection over union between predicted and ground-truth segmentation masks.
• Recall@5 — range: [0, 1]
  • Fraction of relevant items retrieved within the top-5 ranked results for cross-modal retrieval.

Input / output format

Input: Spatially aligned 128×128 remote sensing image tiles (primarily Sentinel-2 12-band) paired with task-specific ground-truth labels (class vectors for classification, pixel masks for segmentation).

Output: Predicted class labels or probabilities for classification tasks; predicted pixel-wise segmentation masks for segmentation tasks.

Scoring recipe

def compute_metrics(preds, golds, task_type):
    if task_type == 'classification':
        acc = (preds == golds).mean()
        f1 = f1_score(golds, preds, average='macro')
        return {'Accuracy': acc, 'F1': f1}
    elif task_type == 'segmentation':
        ious = []
        for p, g in zip(preds, golds):
            inter = np.logical_and(p, g).sum()
            union = np.logical_or(p, g).sum()
            ious.append(inter / (union + 1e-6))
        return {'IoU': np.mean(ious)}
    elif task_type == 'retrieval':
        # Recall@5: fraction of queries where at least one relevant item is in top-5
        pass

Common pitfalls

• Confusing linear probing (frozen encoder, 50 epochs) with full fine-tuning (all parameters updated, 50 epochs for classification, 100 for segmentation).
• Not specifying macro vs. micro averaging for F1 score, which can significantly change reported values.
• Mixing up pretraining multi-modal inputs (S2, SAR, DEM) with downstream task inputs, which are task-specific.

Evidence (verbatim from paper)

Table 2: Downstream evaluation results on GeoBench dataset. FT = full fine-tuning, LP = linear probing. | Pretrain data | BigEarthNet20k (F1↑)FT/LP | So2Sat20k (Acc.↑)FT/LP | Cashew1k (IoU↑)FT | SAcrop3k (IoU↑)FT |

Citation

@misc{hasan2026geomeld,
  title={GeoMeld: Toward Semantically Grounded Foundation Models for Remote Sensing},
  author={Hasan et al. (2026)},
  year={2026},
  note={arXiv:2604.10591}
}

• arXiv: 2604.10591