Pap 12k Eval

pap-12k-eval

Panoramic Affordance Prediction — Zhang et al. (2026) (arXiv:2603.15558, 2026)

What this evaluates

Evaluates a model's ability to predict affordance regions in 360° panoramic imagery. It probes spatial reasoning, handling of extreme scale variations, and robustness to geometric distortions inherent in equirectangular projection formats.

Datasets

• PAP-12K — total 12000; splits: test (-1); repo https://github.com/EnVision-Research/PAP

Metrics

• gIoU (primary) — range: percent
  • Average Intersection-over-Union across all test samples, measuring overall segmentation quality.
• cIoU — range: percent
  • Cumulative intersection over cumulative union across the entire dataset, providing a dataset-level quality measure less sensitive to individual object sizes.
• P@50 — range: percent
  • Percentage of predictions with an IoU score exceeding 0.5, evaluating the model's ability to generate high-quality predictions.
• P@50:95 — range: percent
  • Average precision calculated across a range of IoU thresholds from 0.5 to 0.95 with 0.05 increments, providing a stricter and more fine-grained assessment of segmentation accuracy.

Input / output format

Input: Ultra-high-resolution (12K) 360° panoramic images in equirectangular projection format, accompanied by a textual affordance query/prompt.

Output: Binary affordance mask or segmentation map indicating the predicted region for the queried affordance.

Scoring recipe

def compute_metrics(pred_masks, gt_masks):
    ious = [iou(p, g) for p, g in zip(pred_masks, gt_masks)]
    gIoU = sum(ious) / len(ious) * 100
    cIoU = (sum(p.sum() for p in pred_masks) & sum(g.sum() for g in gt_masks)) / \
           (sum(p.sum() for p in pred_masks) | sum(g.sum() for g in gt_masks)) * 100
    P50 = sum(1 for i in ious if i > 0.5) / len(ious) * 100
    thresholds = [0.5 + 0.05 * i for i in range(10)]
    P50_95 = mean([sum(1 for i in ious if i >= t) / len(ious) for t in thresholds]) * 100
    return gIoU, cIoU, P50, P50_95

Common pitfalls

• Standard 2D perspective models suffer severe performance degradation due to geometric distortion in equirectangular panoramic images.
• Objects split across the left/right image boundaries are frequently missed or incorrectly segmented by baseline methods.
• Extreme scale variations (objects <0.1% or >30% of image area) require multi-scale routing to avoid localization failure.

Evidence (verbatim from paper)

Following the standard evaluation protocol in affordance prediction (Wang et al., 2025a; Zhang et al., 2025c), we adopt four complementary metrics to comprehensively assess prediction quality, including: 1) $gIoU$ (Generalized IoU): The average Intersection-over-Union across all test samples, measuring the overall segmentation quality of the predicted affordance regions. 2) $cIoU$ (Cumulative IoU): The cumulative intersection over cumulative union across the entire dataset, providing a dataset-level quality measure that is less sensitive to the size of individual objects. 3) $P@50$ (Precision at IoU=0.5): The percentage of predictions with an IoU score exceeding 0.5, evaluating the model's ability to generate high-quality predictions. 4) $P@50:95$ : The average precision calculated across a range of IoU thresholds from 0.5 to 0.95 with 0.05 increments, providing a stricter and more fine-grained assessment of segmentation accuracy.

Citation

@misc{zhang2026panoramic,
  title={Panoramic Affordance Prediction},
  author={Zhang et al. (2026)},
  year={2026},
  note={arXiv:2603.15558}
}

• arXiv: 2603.15558