Cps3d Seg Eval

cps3d-seg-eval

Point Cloud Segmentation of Integrated Circuits Package Substrates Surface Defects Using Causal Inference: Dataset Construction and Methodology — Guo et al. (2025) (arXiv:2511.05853, 2025)

What this evaluates

Evaluates 3D point cloud segmentation models for detecting surface defects on integrated circuit package substrates. It probes the model's ability to accurately classify high-density point clouds into normal and defect categories under industrial inspection conditions.

Datasets

• CPS3D-Seg — total ?; splits: train (-1), test (-1); repo https://github.com/Bingyang0410/CPS3D-Seg

Metrics

• mIoU (primary) — range: [0, 1]
  • Mean Intersection over Union: average of IoU (intersection over union) computed per class across all test samples. It is the headline metric used to rank methods.
• mAP — range: [0, 1]
  • Mean Average Precision: average of AP scores across all defect classes, typically computed at a fixed IoU threshold.
• OA — range: [0, 1]
  • Overall Accuracy: ratio of correctly classified points to total points in the dataset.
• Normal/Abnormal IoU & Precision — range: [0, 1]
  • Class-specific IoU and Precision calculated separately for normal (background) and abnormal (defect) categories to address severe class imbalance in industrial defect detection.

Input / output format

Input: 3D point cloud data (typically ~16K points per object) containing spatial coordinates and potentially intensity/color features.

Output: Per-point segmentation labels (binary or multi-class defect classification masks).

Scoring recipe

def compute_metrics(predictions, ground_truth, num_classes=2):
    ious = []
    for c in range(num_classes):
        pred_c = (predictions == c)
        gt_c = (ground_truth == c)
        intersection = np.sum(pred_c & gt_c)
        union = np.sum(pred_c | gt_c)
        ious.append(intersection / union if union > 0 else 0.0)
    mIoU = np.mean(ious)
    OA = np.sum(predictions == ground_truth) / len(ground_truth)
    return mIoU, OA

Common pitfalls

• Models are compared using a fixed data split ratio, but the exact train/val/test split sizes and shuffling strategy are not disclosed, hindering direct reproducibility.
• The dataset exhibits severe class imbalance between normal and defective points; reporting only aggregate mIoU can mask poor defect detection performance, hence the paper emphasizes separate normal/abnormal metrics.
• High point density (~16K points/object) requires careful memory management; naive implementations may fail or require sampling, altering the evaluation protocol.

Evidence (verbatim from paper)

Our goal is to provide a fair and systematic performance analysis on CPS3D-Seg dataset. To ensure fairness in comparison, all models were trained and tested in the same computing environment. During the training process, we used the same ratio of data to ensure comparability of the results. ... Compared to the above methods, CINet achieved the best results in all metrics. Specifically, CINet outperforms PTV3 by approximately 4.16% on the most important mIoU metric.

Citation

@misc{guo2025cps3d,
  title={Point Cloud Segmentation of Integrated Circuits Package Substrates Surface Defects Using Causal Inference: Dataset Construction and Methodology},
  author={Guo et al. (2025)},
  year={2025},
  note={arXiv:2511.05853}
}

• arXiv: 2511.05853