Oem Gfss Eval

oem-gfss-eval

Generalized Few-Shot Semantic Segmentation in Remote Sensing: Challenge and Benchmark — Broni-Bediako et al. (2024) (arXiv:2409.11227, 2024)

What this evaluates

Evaluates a model's ability to perform generalized few-shot semantic segmentation on remote sensing imagery. It tests whether a model can accurately segment both previously seen (base) and new (novel) land cover classes simultaneously using only a few support examples (5-shot), probing generalization and resistance to class forgetting in low-data regimes.

Datasets

• OEM-GFSS — total 408; splits: train (258), val (50), test (100); repo https://github.com/cliffbb/OEM-Fewshot-Challenge

Metrics

• mIoU (primary) — range: [0, 1]
  • Mean Intersection over Union across all 15 classes. Computed as the average of per-class IoU, where IoU = true positives / (true positives + false positives + false negatives).

Input / output format

Input: 1024×1024 pixel remote sensing images (0.25–0.5m resolution). Each evaluation instance provides a support set with pixel-level masks for novel classes and a query set containing images and masks for both base and novel classes.

Output: Pixel-wise segmentation mask for each query image, assigning one of 15 fine-grained land cover class labels or background.

Scoring recipe

def compute_miou(predictions, ground_truth, num_classes=15):
    ious = []
    for c in range(num_classes):
        pred_c = (predictions == c)
        gt_c = (ground_truth == c)
        intersection = np.logical_and(pred_c, gt_c).sum()
        union = np.logical_or(pred_c, gt_c).sum()
        ious.append(intersection / union if union > 0 else 1.0)
    return np.mean(ious)

Common pitfalls

• Evaluating only on novel classes instead of jointly on base and novel classes, which violates the generalized few-shot setting.
• Ignoring the 5-shot constraint for the support set or using more examples than specified.
• Failing to correctly handle the background class, as undefined objects are explicitly labeled as background (RGB 0,0,0) and should be included in the evaluation.

Evidence (verbatim from paper)

The validation and test sets contain images and labels of the val-novel and test-novel classes, respectively, and both consist of a support set and a query set for GFSS task of a 5-shot with 4-novel and 7-base classes.

Citation

@misc{broni-bediako2024oem,
  title={Generalized Few-Shot Semantic Segmentation in Remote Sensing: Challenge and Benchmark},
  author={Broni-Bediako et al. (2024)},
  year={2024},
  note={arXiv:2409.11227}
}

• arXiv: 2409.11227