Seeds Superpixel Eval

seeds-superpixel-eval

SEEDS: Superpixels Extracted via Energy-Driven Sampling — Van den Bergh et al. (2013) (arXiv:1309.3848, 2013)

What this evaluates

Evaluates the quality of superpixel segmentation algorithms by measuring how well superpixel boundaries align with ground-truth object boundaries and how accurately superpixels can be used as indivisible units for downstream segmentation tasks.

Datasets

• Berkeley Segmentation Dataset (BSD) — total 500; splits: train (200), val (100), test (200)

Metrics

• under-segmentation error (UE) (primary) — range: [0, 1]
  • Sum of pixels in superpixels that do not overlap with their corresponding ground-truth segment, divided by total ground-truth area. Lower is better.
• corrected under-segmentation error (CUE) — range: [0, 1]
  • Each superpixel is matched to the ground-truth segment with the largest overlap. The number of pixels outside this matched segment is summed and divided by total ground-truth area. Lower is better.
• boundary recall (BR) — range: [0, 1]
  • Percentage of ground-truth boundary pixels that have at least one superpixel boundary pixel within a tolerance of epsilon=2 pixels. Higher is better.
• achievable segmentation accuracy (ASA) — range: [0, 1]
  • Maximum possible segmentation accuracy if superpixels are treated as indivisible units. Computed by assigning each superpixel to its most overlapping ground-truth label and dividing correctly labeled pixels by total area. Higher is better.

Input / output format

Input: RGB or LAB color image and corresponding ground-truth segmentation map.

Output: Superpixel segmentation map where each pixel is assigned a superpixel ID.

Scoring recipe

def compute_ue(sp, gt):
    total_gt = sum(len(g) for g in gt)
    err = sum(len(s - g) for s in sp for g in gt if s & g)
    return err / total_gt

def compute_cue(sp, gt):
    total_gt = sum(len(g) for g in gt)
    err = sum(len(s - max(gt, key=lambda g: s & g)) for s in sp)
    return err / total_gt

def compute_br(sp, gt, eps=2):
    gt_b = get_boundaries(gt)
    sp_b = get_boundaries(sp)
    matches = sum(1 for p in gt_b if min_dist(p, sp_b) < eps)
    return matches / len(gt_b)

def compute_asa(sp, gt):
    total_gt = sum(len(g) for g in gt)
    correct = sum(max(len(s & g) for g in gt) for s in sp)
    return correct / total_gt

Common pitfalls

• Tolerance handling for boundary pixels varies across papers (e.g., 5% margin vs. removing borders).
• UE penalizes boundary pixels equally on both sides, which can unfairly penalize grid-based initializations.
• BR uses a fixed tolerance epsilon=2 pixels, which may not match other implementations.

Evidence (verbatim from paper)

We compute the standard metrics used to evaluate the performance of superpixel algorithms, which are under-segmentation error (UE), boundary recall (BR) and achievable segmentation accuracy (ASA). Additionally, we introduce a new metric, which is a corrected under-segmentation error (CUE).

Citation

@misc{vandenberg2013seeds,
  title={SEEDS: Superpixels Extracted via Energy-Driven Sampling},
  author={Van den Bergh et al. (2013)},
  year={2013},
  note={arXiv:1309.3848}
}

• arXiv: 1309.3848