# Gres Eval

> Evaluates a model's ability to segment arbitrary numbers of target objects (including zero) in an image based on a natural language expression. It probes multi-target localization, no-target rejection, and robustness to complex linguistic structures like counting and compound relations. Use when the user wants to benchmark on gRefCOCO, or asks about evaluating this task. Reports generalized IoU (gIoU).

- Skill: `qhjqhj00/gres-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/gres-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/gres-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/gres-eval

---


# gres-eval

> GRES: Generalized Referring Expression Segmentation — Liu et al. (2023) (arXiv:2306.00968, 2023)

## What this evaluates

Evaluates a model's ability to segment arbitrary numbers of target objects (including zero) in an image based on a natural language expression. It probes multi-target localization, no-target rejection, and robustness to complex linguistic structures like counting and compound relations.

## Datasets

- **gRefCOCO** — total 278232; splits: train (-1), val (-1), test (-1)

## Metrics

- `generalized IoU (gIoU)` **(primary)** — range: [0, 1]
  - Extends mean IoU to all samples, including no-target expressions. For no-target cases, the ground truth is empty, so any non-empty prediction yields an IoU of 0. Computed as the average IoU across all test samples.
- `cumulative IoU (cIoU)` — range: [0, 1]
  - Standard RES metric computing the mean Intersection over Union across all target instances in the dataset. For multi-target expressions, IoU is computed per target and averaged.
- `Precision@X` — range: [0, 1]
  - Pixel-wise precision at a given IoU threshold X. The exact threshold value is not specified in the text and follows standard RES conventions.
- `No-target-accuracy (N-acc.)` — range: percent
  - Percentage of no-target expressions correctly predicted as empty masks.
- `Target-accuracy (T-acc.)` — range: percent
  - Percentage of target expressions correctly predicted with a non-empty mask above a standard threshold.

## Input / output format

**Input**: An image I and a natural language expression T.

**Output**: A binary segmentation mask M over the image. For no-target expressions, M must be entirely empty (all negative).

## Scoring recipe

```python
def compute_gres_metrics(pred_masks, gold_masks, is_no_target_flags):
    ious = []
    n_acc = 0
    t_acc = 0
    for pred, gt, is_no in zip(pred_masks, gold_masks, is_no_target_flags):
        if is_no:
            ious.append(0.0)
            if not pred.any(): n_acc += 1
        else:
            iou_val = np.sum(pred & gt) / np.sum(pred | gt)
            ious.append(iou_val)
            if iou_val >= 0.5: t_acc += 1
    gIoU = np.mean(ious)
    cIoU = np.mean([np.sum(p & g) / np.sum(p | g) for p, g in zip(pred_masks, gold_masks) if not is_no])
    return {'gIoU': gIoU, 'cIoU': cIoU, 'N-acc.': n_acc/len(pred_masks), 'T-acc.': t_acc/len(pred_masks)}
```

## Common pitfalls

- Models often fail to output an empty mask for no-target expressions, incorrectly segmenting background or irrelevant objects.
- Multi-target expressions require aggregating IoU across multiple instances; failing to correctly sum or average per-instance IoUs leads to inaccurate cIoU/gIoU scores.
- Counting expressions (e.g., 'two people') and ordinal expressions (e.g., 'second person') are easily confused, causing wrong instance selection and degraded precision.

## Evidence (verbatim from paper)

> Besides the regular RES performance metric cumulative IoU (cIoU) and Precision@X, we further propose a new metric called generalized IoU (gIoU), which extends the mean IoU to all samples including no-target ones. Moreover, No-target performance is also separately evaluated by computing No-target-accuracy (N-acc.) and Target-accuracy (T-acc.).

## Citation

```bibtex
@misc{liu2023gres,
  title={GRES: Generalized Referring Expression Segmentation},
  author={Liu et al. (2023)},
  year={2023},
  note={arXiv:2306.00968}
}
```

- arXiv: 2306.00968