Mobiface Eval

mobiface-eval

MobiFace: A Novel Dataset for Mobile Face Tracking in the Wild — Lin et al. (2018) (arXiv:1805.09749, 2018)

What this evaluates

This benchmark evaluates the capability of visual tracking algorithms to maintain robust face localization in unconstrained, mobile-captured video sequences. It specifically probes resilience to challenging real-world conditions such as rapid camera motion, out-of-plane rotations, scale changes, and partial occlusions.

Datasets

• iBUG MobiFace — total 50736; splits: test (50736)

Metrics

• AUC (primary) — range: [0, 1]
  • Area Under the Curve of the success plot, which measures the percentage of frames where the overlap ratio between predicted and ground-truth bounding boxes exceeds a given threshold (typically integrated over thresholds from 0 to 1).
• Precision@20px — range: [0, 1]
  • The percentage of frames where the Euclidean distance between the center of the predicted bounding box and the ground-truth center is within 20 pixels.
• FPS — range: other
  • Frames per second processed by the tracker, computed as total frames divided by total inference time.

Input / output format

Input: Sequential video frames captured from smartphones, along with ground-truth bounding box annotations for the target face in each frame.

Output: Predicted bounding box coordinates (e.g., top-left x, y and width, height) for the target face in each frame.

Scoring recipe

def compute_metrics(predictions, ground_truths, frame_times):
    overlaps = []
    center_errors = []
    for pred, gt in zip(predictions, ground_truths):
        overlaps.append(intersection_over_union(pred, gt))
        center_errors.append(center_distance(pred, gt))
    # AUC: integrate success curve over thresholds [0, 1]
    auc = np.trapz([sum(o >= t) / len(o) for t in np.linspace(0, 1, 100)], np.linspace(0, 1, 100))
    # Precision@20px
    precision = sum(e <= 20 for e in center_errors) / len(center_errors)
    # FPS
    fps = len(predictions) / sum(frame_times)
    return {'AUC': auc, 'Precision@20px': precision, 'FPS': fps}

Common pitfalls

• Forcing trackers to output a bounding box even when the target is completely missing, which unfairly penalizes re-identification capabilities compared to methods that correctly output null/zero.
• Ignoring online model adaptation; trackers with fixed weights (e.g., SiamFC) degrade significantly compared to those that update parameters during tracking.
• Evaluating speed on desktop GPUs (GTX 1060) rather than actual mobile hardware, making FPS comparisons less representative of real-world deployment constraints.

Evidence (verbatim from paper)

For success plot, the trackers' name is shown with their corresponding AUC. For precision plot, the score at 20 pixel threshold is shown.

Citation

@misc{lin2018mobiface,
  title={MobiFace: A Novel Dataset for Mobile Face Tracking in the Wild},
  author={Lin et al. (2018)},
  year={2018},
  note={arXiv:1805.09749}
}

• arXiv: 1805.09749