Unseen Object 6d Pose Eval

unseen-object-6d-pose-eval

Unseen Object 6D Pose Estimation: A Benchmark and Baselines — Gou et al. (2022) (arXiv:2206.11808, 2022)

What this evaluates

Evaluates a model's ability to estimate the 6D pose (rotation and translation) of novel, unseen 3D objects in real-world scenes without retraining, using only their mesh models and partial RGBD inputs. It specifically probes robustness to pose ambiguity, partial observability, and real-world noise.

Datasets

• GraspNet-1Billion — total ?; splits: Seen (-1), Similar (-1), Novel (-1)
• YCB-Video — total ?; splits: test (-1)

Metrics

• IADD (primary) — range: other
  • Infimum of Average Distance: $\frac{1}{m}\sum_{v\in\mathcal{V}}\inf_{R^{},T^{}}| (Rv+T) - (R^{}v+T^{}) |$. Computes the minimum average vertex distance over all valid ground-truth poses, handling infinite pose ambiguities via angle sampling or center distance.
• AUC@0.5d — range: [0, 1]
  • Area Under the Curve of IADD (or ADD/ADD-S) error scores, integrated up to a threshold of 0.5 times the target object's diagonal length.

Input / output format

Input: Partial RGBD scene point cloud and a 3D mesh model of a target object.

Output: 6D pose parameters: rotation matrix R and translation vector T.

Scoring recipe

def compute_iadd(pred_R, pred_T, gt_poses, mesh_verts):
    min_add = float('inf')
    for R_gt, T_gt in gt_poses:
        add = mean(norm((pred_R @ v + pred_T) - (R_gt @ v + T_gt)) for v in mesh_verts)
        min_add = min(min_add, add)
    return min_add

# For infinite symmetry, sample angles or use center distance.
# Then compute AUC of scores up to threshold = 0.5 * object_diagonal.

Common pitfalls

• Pose ambiguity for symmetric objects makes standard ADD misleading; IADD is required for fair comparison across object types.
• AUC threshold is object-specific (0.5x diagonal), not a fixed global value like 5cm or 10cm.
• Partial observability and noise in real-world scenes significantly degrade correspondence-based baselines, requiring robust matching.

Evidence (verbatim from paper)

We propose a new metric named Infimum of ADD(IADD). ... The number is the area under curve(AUC) score of each methed using each metric. The upper bound of AUC is set to $0.5\times$ diagonal of the target object.

Citation

@misc{gou2022unseen,
  title={Unseen Object 6D Pose Estimation: A Benchmark and Baselines},
  author={Gou et al. (2022)},
  year={2022},
  note={arXiv:2206.11808}
}

• arXiv: 2206.11808