Wider Face Eval

wider-face-eval

WIDER FACE: A Face Detection Benchmark — Yang et al. (2015) (arXiv:1511.06523, 2015)

What this evaluates

Evaluates face detection algorithms on real-world images with extreme variations in scale, pose, occlusion, and event context. It probes the ability of detectors to handle small faces, heavy occlusion, and atypical poses under standard bounding box matching criteria.

Datasets

• WIDER FACE — total 32203; splits: easy (-1), medium (-1), hard (-1)

Metrics

• Average Precision (AP) (primary) — range: [0, 1]
  • Computed following the PASCAL VOC convention. Predictions are matched to ground truth bounding boxes based on Intersection over Union (IoU) threshold (typically 0.5 for face detection). Precision-recall curve is interpolated to compute the area under the curve, yielding a score between 0 and 1.

Input / output format

Input: RGB images containing faces with varying scales, poses, and occlusion levels. Ground truth annotations include bounding box coordinates and difficulty/attribute labels (scale, occlusion, pose, event category).

Output: List of predicted bounding boxes per image, each with coordinates (x, y, width, height) and a confidence score.

Scoring recipe

def compute_ap(preds, gts, iou_thresh=0.5):
    # Sort predictions by confidence descending
    tp, fp = [], []
    matched_gts = set()
    for pred in sorted(preds, key=lambda x: x.conf, reverse=True):
        best_iou, best_gt_idx = 0, -1
        for i, gt in enumerate(gts):
            if i not in matched_gts:
                iou = calculate_iou(pred.box, gt.box)
                if iou > best_iou:
                    best_iou, best_gt_idx = iou, i
        if best_iou >= iou_thresh:
            tp.append(1); fp.append(0); matched_gts.add(best_gt_idx)
        else:
            tp.append(0); fp.append(1)
    # Compute precision/recall and interpolate PR curve
    # Return area under interpolated PR curve
    return ap_score

Common pitfalls

• Performance heavily degrades on faces smaller than 50 pixels or with >30% occlusion, so reporting only overall AP masks critical failures.
• Evaluating detectors trained on external datasets (Scenario-Ext) without retraining on WIDER FACE yields artificially low baselines.
• Attribute-specific subsets (scale, occlusion, pose) require strict filtering of ground truth before evaluation to avoid cross-contamination.

Evidence (verbatim from paper)

We employ PASCAL VOC evaluation metric for the evaluation. Following previous work, we conduct linear transformation for each method to fit the annotation of WIDER FACE. The average precision (AP) of most methods are over 60%, but none of them surpasses 75%. The performance drops 10% for all methods on the medium set. The hard set is even more challenging. The performance quickly decreases, with a AP below 30% for all methods.

Citation

@misc{yang2015widerface,
  title={WIDER FACE: A Face Detection Benchmark},
  author={Yang et al. (2015)},
  year={2015},
  note={arXiv:1511.06523}
}

• arXiv: 1511.06523