Person Detection Eval

person-detection-eval

Preventing Errors in Person Detection: A Part-Based Self-Monitoring Framework — Schwaiger et al. (2023) (arXiv:2307.04533, 2023)

What this evaluates

Evaluates person and body part detection accuracy using standard object detection metrics, while assessing a self-monitoring framework's ability to reduce false negatives and false positives through part-based plausibility checks.

Datasets

• DensePose — total 15698; splits: train (13483), val (2215)
• MS-COCO — total 11691; splits: val (11691)
• Pascal VOC — total 2971; splits: trainval (2971)

Metrics

• mAP — range: [0, 1]
  • Mean Average Precision across all classes, computed by averaging the Area Under the Precision-Recall curve for each class.
• AP@0.5 (primary) — range: [0, 1]
  • Average Precision computed using a fixed Intersection over Union (IoU) threshold of 0.5 between predicted and ground-truth bounding boxes.

Input / output format

Input: RGB images containing persons and body parts.

Output: Bounding boxes and class labels for 'person' and 8 body parts (Torso, Hand, Foot, Upper Leg, Lower Leg, Upper Arm, Lower Arm, Head).

Scoring recipe

1. Match predicted boxes to ground-truth boxes using IoU >= 0.5.
2. Classify matches as TP, unmatched predictions as FP, unmatched GT as FN.
3. Compute Precision-Recall curve per class.
4. Calculate AP per class by integrating the PR curve.
5. Average AP across classes to get mAP. Report AP@0.5 as the headline metric.

Common pitfalls

• IoU threshold is fixed at 0.5, not the standard COCO 0.5:0.95 range.
• Evaluation images are filtered to exclude persons with bounding box area < 2247 pixels^2.
• Cross-dataset evaluation uses the Pascal VOC trainval split, not the standard test split.

Evidence (verbatim from paper)

The performance for class person in terms of mAP and Average Precision (AP) with an IOU threshold of 0.5 is presented for each model in Table [I]... To determine the detection sets derived from the ground-truth annotations D_TP_gt, D_FP_gt, and D_FN_gt, we set the value of the IOU threshold to 0.5.

Citation

@misc{schwaiger2023preventing,
  title={Preventing Errors in Person Detection: A Part-Based Self-Monitoring Framework},
  author={Schwaiger et al. (2023)},
  year={2023},
  note={arXiv:2307.04533}
}

• arXiv: 2307.04533