Few Shot Segmentation Eval

few-shot-segmentation-eval

Learning Point Embeddings from Shape Repositories for Few-Shot Segmentation — Sharma et al. (2019) (arXiv:1910.01269, 2019)

What this evaluates

Evaluates a model's ability to perform semantic segmentation on 3D point clouds using only a few labeled examples per category. It probes how well learned point embeddings generalize to unseen shapes when supervision is extremely limited, testing both few-shot (few labeled shapes) and few-point (few labeled points per shape) scenarios.

Datasets

• ShapeNet segmentation dataset — total ?; splits: train (-1), test (-1)

Metrics

• mIOU (primary) — range: [0, 1]
  • Mean Intersection over Union computed across all part labels and all shapes in the test split. Calculated as the average of IoU scores for each class across all test shapes.

Input / output format

Input: 3D point clouds (shapes) represented as sets of points with coordinates and/or features.

Output: Per-point semantic part label probabilities or hard segmentation masks for each point in the input shape.

Scoring recipe

ious = []
for shape in test_shapes:
    for class_id in all_part_labels:
        pred = (predictions[shape] == class_id)
        gold = (gold[shape] == class_id)
        inter = np.logical_and(pred, gold).sum()
        union = np.logical_or(pred, gold).sum()
        ious.append(inter / union if union > 0 else 1.0)
return np.mean(ious)

Common pitfalls

• Confusing the few-shot setting (varying number of labeled training shapes) with the few-point setting (varying number of labeled points per shape).
• Failing to exclude shapes used for pre-training (part hierarchy/tag datasets) from the test split, which causes data leakage and inflates mIOU.
• Computing mIOU over object categories instead of fine-grained part labels, which underestimates segmentation granularity.

Evidence (verbatim from paper)

The performance is measured as the mean intersection over union (mIOU) across all part labels and shapes in the test splits. We exclude the shapes existing in our part hierarchy and tag datasets used for pre-training PEN from the test splits.

Citation

@misc{sharma2019fewshotsegmentation,
  title={Learning Point Embeddings from Shape Repositories for Few-Shot Segmentation},
  author={Sharma et al. (2019)},
  year={2019},
  note={arXiv:1910.01269}
}

• arXiv: 1910.01269