Ris Lad Eval

ris-lad-eval

RIS-LAD: A Benchmark and Model for Referring Low-Altitude Drone Image Segmentation — Ye et al. (2025) (arXiv:2507.20920, 2025)

What this evaluates

Evaluates referring image segmentation on low-altitude drone imagery, probing the model's ability to accurately localize and segment referred objects despite challenges like category drift (tiny objects) and object drift (dense same-category scenes).

Datasets

• RIS-LAD — total ?; splits: val (-1), test (-1); repo https://github.com/AHideoKuzeA/RIS-LAD-A-Benchmark-and-Model-for-Referring-Low-Altitude-Drone-Image-Segmentation

Metrics

• oIoU (primary) — range: percent
  • Overall Intersection-over-Union: the average IoU across all pixels in the image between the predicted mask and the ground truth mask.
• mIoU (primary) — range: percent
  • Mean Intersection-over-Union: the average IoU across all object instances in the image.
• P@X — range: percent
  • Precision at IoU threshold X: the percentage of predicted masks that achieve an IoU ≥ X with the ground truth.

Input / output format

Input: A low-altitude drone image paired with a referring text expression describing the target object.

Output: A binary segmentation mask indicating the pixels belonging to the referred object.

Scoring recipe

def compute_metrics(pred_masks, gt_masks, X):
    ious = []
    for pred, gt in zip(pred_masks, gt_masks):
        intersection = np.logical_and(pred, gt).sum()
        union = np.logical_or(pred, gt).sum()
        ious.append(intersection / union if union > 0 else 0.0)
    oIoU = np.mean(ious) * 100
    mIoU = np.mean(ious) * 100
    P_at_X = sum(1 for iou in ious if iou >= X) / len(ious) * 100
    return oIoU, mIoU, P_at_X

Common pitfalls

• P@X may overestimate performance on small objects due to their ease of enclosure.
• Methods relying on frozen pre-trained encoders (e.g., CLIP) may suffer negative transfer due to domain gaps with drone imagery.
• Boundary overgeneralization can lead to high P@X but low oIoU/mIoU.

Evidence (verbatim from paper)

For metrics, we report Precision@0.5–0.9 (P@X), Overall Intersection-over-Union (oIoU), and Mean Intersection-over-Union (mIoU). While P@X highlights accurate predictions, it may overestimate performance on small objects due to their ease of enclosure. Thus, oIoU and mIoU are used as primary metrics to evaluate segmentation performance.

Citation

@misc{ye2025rislad,
  title={RIS-LAD: A Benchmark and Model for Referring Low-Altitude Drone Image Segmentation},
  author={Ye et al. (2025)},
  year={2025},
  note={arXiv:2507.20920}
}

• arXiv: 2507.20920