Gseval Pixel Grounding Eval

gseval-pixel-grounding-eval

GroundingSuite: Measuring Complex Multi-Granular Pixel Grounding — Hu et al. (2025) (arXiv:2503.10596, 2025)

What this evaluates

Evaluates a model's ability to perform open-vocabulary, fine-grained pixel grounding by generating accurate segmentation masks from complex, long-form referring expressions across multiple granularities (stuff, part, multi-object, single-object).

Datasets

• GSEval — total 3800; splits: test (-1)
• gRefCOCO — total ?; splits: test (-1)
• RefCOCOm — total ?; splits: test (-1)
• RefCOCO — total ?; splits: test (-1)
• RefCOCOg — total ?; splits: test (-1)

Metrics

• cIoU / gIoU (primary) — range: [0, 1]
  • Intersection over Union (IoU) between the predicted segmentation mask and the ground-truth mask. The paper reports cIoU (centered IoU) for gRefCOCO and gIoU (generalized IoU) for RefCOCOm, with higher values indicating better pixel-level alignment.

Input / output format

Input: An image paired with a natural language referring expression (text description).

Output: A binary segmentation mask (pixel-level prediction) corresponding to the object(s) described in the text.

Scoring recipe

def compute_iou(pred_mask, gt_mask):
    intersection = np.logical_and(pred_mask, gt_mask).sum()
    union = np.logical_or(pred_mask, gt_mask).sum()
    return intersection / union if union > 0 else 0.0
# Metric is averaged over all test instances.

Common pitfalls

• Evaluating on multi-granular targets (stuff, part, multi-object, single-object) requires consistent mask generation across varying object complexities.
• Long-form referring expressions (averaging 16 words) demand fine-grained semantic understanding, making models prone to hallucination or misalignment without robust training data.
• Zero-shot evaluation on open-source methods must use public code and weights to ensure fair comparison, as noted in the figure captions.

Evidence (verbatim from paper)

The resulting GSEval benchmark (3,800 images) offers zero overlap with existing datasets and evaluates open-vocabulary, fine-grained, and complex scene grounding, enabling state-of-the-art performance (cIoU 68.9 on gRefCOCO, gIoU 55.3 on RefCOCOm) and setting new benchmarks for pixel grounding across multi-granular and open-domain scenarios.

Citation

@misc{hu2025groundingsuite,
  title={GroundingSuite: Measuring Complex Multi-Granular Pixel Grounding},
  author={Hu et al. (2025)},
  year={2025},
  note={arXiv:2503.10596}
}

• arXiv: 2503.10596