Rgb Event Segmentation Eval

rgb-event-segmentation-eval

Learning Flow-Guided Registration for RGB-Event Semantic Segmentation — Yao et al. (2025) (arXiv:2505.01548, 2025)

What this evaluates

Evaluates the capability of RGB-Event fusion models to perform accurate per-pixel semantic segmentation under challenging conditions such as fast motion, varying lighting, and spatiotemporal misalignment between asynchronous modalities.

Datasets

• DDD17 — total 19840; splits: train (15950), test (3890)
• DSEC — total 10891; splits: test (10891)
• DELIVER — total ?; splits: test (-1)
• M3ED — total 7997; splits: train (5516), test (2481)

Metrics

• mIoU (primary) — range: percent
  • Mean Intersection over Union across all semantic classes, computed as the average of IoU per class.
• Accuracy — range: percent
  • Pixel accuracy, calculated as the ratio of correctly classified pixels to the total number of pixels.

Input / output format

Input: Paired RGB images and asynchronous event data (represented as Motion-Enhanced Event Tensor, voxel grids, or other event representations).

Output: Per-pixel semantic segmentation mask assigning one of the dataset-specific class labels to each pixel.

Scoring recipe

def compute_miou(preds, gold, num_classes):
    ious = []
    for c in range(num_classes):
        tp = np.sum((preds == c) & (gold == c))
        fp = np.sum((preds == c) & (gold != c))
        fn = np.sum((preds != c) & (gold == c))
        iou = tp / (tp + fp + fn + 1e-6)
        ious.append(iou)
    return np.mean(ious) * 100

Common pitfalls

• Comparisons must use the same backbone (MiT-B2) to be fair, as performance heavily depends on it.
• Event representation format (e.g., MET vs. Voxel Grid) significantly impacts results and must be explicitly controlled or reported.
• M3ED dataset splits are manually curated from the Urban Day subset, not a standard public split, requiring careful reproduction.

Evidence (verbatim from paper)

As shown in Table [2], BRENet achieves 78.56% mIoU and 96.61% accuracy on DDD17 and 74.94% mIoU and 95.85% accuracy on DSEC using the MiT-B2 backbone. ... The table indicates that BRENet achieves 63.13% and 67.28% mIoU on DELIVER and M3ED.

Citation

@misc{yao2025learning,
  title={Learning Flow-Guided Registration for RGB-Event Semantic Segmentation},
  author={Yao et al. (2025)},
  year={2025},
  note={arXiv:2505.01548}
}

• arXiv: 2505.01548