Crowd Pose Estimation Eval

crowd-pose-estimation-eval

Rethinking pose estimation in crowds: overcoming the detection information-bottleneck and ambiguity — Mu Zhou et al. (2023) (arXiv:2306.07879, 2023)

What this evaluates

Evaluates the ability of pose estimation models to accurately predict 2D keypoints for humans and animals in crowded, occluded, and multi-instance scenarios. It probes robustness to detection ambiguity, overlapping instances, and the transferability of conditional pose inputs from bottom-up detectors to top-down refiners.

Datasets

• CrowdPose — total 20000; splits: trainval (12000), test (8000)
• OCHuman — total 4731; splits: test (4731)
• COCO — total 82000; splits: train (57000), val (5000), test-dev (20000)
• Multi-Animal (SchoolingFish, Marmosets, Tri-Mouse) — total ?; splits: test (-1)

Metrics

• AP (primary) — range: [0, 1]
  • Average Precision computed over IoU thresholds (typically 0.5:0.95) for matching predicted keypoints to ground truth within a normalized distance threshold. Variants include APeasy/APmed/APhard (crowd density/occlusion splits) and APM/APL (object size splits).
• APeasy — range: [0, 1]
  • AP computed only on easy instances (low crowd density/occlusion).
• APmed — range: [0, 1]
  • AP computed only on medium instances.
• APhard — range: [0, 1]
  • AP computed only on hard instances (high crowd density/occlusion).
• APM — range: [0, 1]
  • AP computed only on medium-sized objects.
• APL — range: [0, 1]
  • AP computed only on large-sized objects.

Input / output format

Input: RGB images of crowded scenes, optionally paired with conditional bounding boxes or 2D pose predictions from a bottom-up model.

Output: Predicted 2D keypoints (e.g., 14 for humans) with instance IDs and confidence scores for each detected person or animal.

Scoring recipe

def compute_pose_ap(predictions, ground_truth, iou_thresh=0.5):
    matches = []
    for gt in ground_truth:
        best_score = 0
        for pred in predictions:
            if pred.instance_id == gt.instance_id:
                iou = calculate_keypoint_iou(pred, gt)
                if iou >= iou_thresh:
                    best_score = max(best_score, pred.confidence)
        matches.append(best_score)
    return average_precision(matches)

Common pitfalls

• Using standard object detector bounding boxes instead of bottom-up pose predictions as conditional inputs significantly degrades performance in crowded scenes.
• Training on COCO and evaluating on OCHuman introduces a domain shift that requires careful validation, as models may overfit to COCO's less crowded distribution.
• Failing to account for generative vs. empirical sampling during training leads to poor generalization when swapping bottom-up model inputs at inference time.

Evidence (verbatim from paper)

We report standard metrics AP, APeasy, APmed and APhard as defined in[[26]]. We compared our method, that derives bounding boxes from a bottom-up model (see Methods), with baselines that used bounding boxes obtained by a Faster R-CNN detector[[37]].

Citation

@misc{zhou2023rethinking,
  title={Rethinking pose estimation in crowds: overcoming the detection information-bottleneck and ambiguity},
  author={Mu Zhou et al. (2023)},
  year={2023},
  note={arXiv:2306.07879}
}

• arXiv: 2306.07879