qhjqhj00 moving-object-segmentation-eval
Learning to Segment Moving Objects in Videos — Fragkiadaki et al. (2014) (arXiv:1412.6504, 2014)
What this evaluates
Evaluates a model's ability to generate and rank spatio-temporal proposals for moving objects in video. It probes motion-based segmentation quality, proposal coverage, and ranking accuracy on both rigid and non-rigid motion across diverse scenes.
Datasets
- VSB100 — total 100; splits: train (40), test (60)
- Moseg — total 59; splits: test (59)
Metrics
Average best overlap(primary) — range: [0, 1]- For each ground-truth segment, compute the maximum IoU across all proposals. Average these maximum IoU values across all ground-truth segments.
Coverage(primary) — range: [0, 1]- Weighted average of IoU scores, where weights correspond to the area of each ground-truth segment. Larger segments contribute more to the score.
Detection rate at 50%— range: percent- Percentage of ground-truth segments that achieve a maximum IoU above 0.5 with any proposal.
Detection rate at 70%— range: percent- Percentage of ground-truth segments that achieve a maximum IoU above 0.7 with any proposal.
Anytime best overlap/detection— range: [0, 1] | percent- Variant of the above metrics where, instead of evaluating per-frame segments, the best overlap/detection is computed across the entire lifespan of each ground-truth tube.
Input / output format
Input: RGB video frames, optical flow fields, and static image segment proposals.
Output: Spatio-temporal tubes or per-frame segment proposals, each associated with a bounding box and a moving objectness score.
Scoring recipe
def compute_metrics(proposals, gt_segments):
ious = {g: max(iou(p, g) for p in proposals) for g in gt_segments}
avg_best_overlap = mean(ious.values())
coverage = sum(ious[g] * area(g) for g in gt_segments) / sum(area(g) for g in gt_segments)
det_50 = mean(v > 0.5 for v in ious.values())
det_70 = mean(v > 0.7 for v in ious.values())
return avg_best_overlap, coverage, det_50, det_70
# For 'anytime best' metrics, replace per-frame segments with spatio-temporal tubes and evaluate across the full lifespan.
Common pitfalls
- Confusing per-frame IoU with spatio-temporal tube IoU, as the paper evaluates both separately and notes different failure modes (e.g., temporal fragmentation vs. background leakage).
- Using mean IoU instead of 'best overlap' or 'coverage', which specifically require taking the maximum IoU per ground-truth object or weighting by ground-truth area.
- Overlooking the 'anytime best' metric, which evaluates proposal quality across an object's entire lifespan rather than frame-by-frame.
Evidence (verbatim from paper)
We consider the following four widely used static image segmentation metrics: a) Average best overlap: the average (across all 2D ground-truth segments in our dataset) of the best IoU score of a ground-truth object with all segment proposals. b) Coverage: the weighted average of IoU scores, weighted by the area of the ground-truth segments (larger segments matter more). c) Detection rate at 50%:the percentage of ground-truth segments that have IoU above 50% with a segment proposal. d) Detection rate at 70%.
Citation
@misc{fragkiadaki2014learning,
title={Learning to Segment Moving Objects in Videos},
author={Fragkiadaki et al. (2014)},
year={2014},
note={arXiv:1412.6504}
}
- arXiv: 1412.6504