Refaerial Eval

refaerial-eval

RefAerial: A Benchmark and Approach for Referring Detection in Aerial Images — Hu et al. (2026) (arXiv:2604.20543, 2026)

What this evaluates

Evaluates a model's ability to localize a target object in an aerial image based on a fine-grained natural language description. It probes cross-modal alignment, scale-invariant object detection, and handling of complex backgrounds with numerous distractors.

Datasets

• RefAerial — total ?; splits: test (-1)
• RefCOCO — total ?; splits: test (-1)
• RefCOCO+ — total ?; splits: test (-1)
• RefCOCOg — total ?; splits: test (-1)

Metrics

• mP (average P@0.5/0.6/0.7/0.8) (primary) — range: percent
  • Precision at IoU threshold $\theta$, denoted P@$\theta$. A prediction is correct if IoU(pred, gt) > $\theta$. mP is the arithmetic mean of P@0.5, P@0.6, P@0.7, and P@0.8.

Input / output format

Input: Aerial image and a natural language referring expression.

Output: A single bounding box (e.g., [x_min, y_min, x_max, y_max]) localizing the target object.

Scoring recipe

def compute_precision(predictions, ground_truths, iou_threshold):
    correct = 0
    for pred_box, gt_box in zip(predictions, ground_truths):
        iou = calculate_iou(pred_box, gt_box)
        if iou > iou_threshold:  # strictly greater than
            correct += 1
    return (correct / len(predictions)) * 100

Common pitfalls

• IoU threshold comparison is strictly greater than (>) the threshold, not greater-than-or-equal-to (≥).
• Precision is computed per referring expression (1-to-1 mapping), not per image, as each expression corresponds to exactly one ground-truth box.
• Models must be re-trained or fine-tuned on the dataset; direct transfer of ground-view checkpoints causes severe performance drops due to aerial scale variability.

Evidence (verbatim from paper)

Following the standard protocol for referring detection in [20, 38, 7]*, we use precision (P@$\theta$) as the evaluation metric, where the prediction is deemed correct if its IoU with the ground-truth box is larger than threshold $\theta$. In detail, we utilize P@0.5, P@0.6, P@0.7, P@0.8, and mP (average pecision across these thresholds) except specially mentioned.

Citation

@misc{hu2026refaerial,
  title={RefAerial: A Benchmark and Approach for Referring Detection in Aerial Images},
  author={Hu et al. (2026)},
  year={2026},
  note={arXiv:2604.20543}
}

• arXiv: 2604.20543