Pixel Face Eval

pixel-face-eval

Pixel-Face: A Large-Scale, High-Resolution Benchmark for 3D Face Reconstruction — Lyu et al. (2020) (arXiv:2008.12444, 2020)

What this evaluates

Evaluates the capability of 3D face reconstruction models to predict accurate 3D meshes and facial landmarks from 2D RGB images. It probes how well models generalize to high-resolution, in-the-wild faces across diverse ages and expressions, highlighting domain gaps from synthetic training data.

Datasets

• Pixel-Face — total 24000; splits: train (-1), test (-1)

Metrics

• NME — range: other (normalized units)
  • Normalized Mean Error: the average Euclidean distance between predicted and ground-truth 3D facial landmarks, divided by the bounding box size of the ground-truth landmarks.
• ARMSE (primary) — range: other (normalized units)
  • Average Root Mean Square Error: evaluates mesh similarity after normalizing the ground-truth interocular distance to 1, aligning the predicted mesh to the ground-truth via facial landmarks, and shifting the origin to the nose tip. Vertices beyond a crop radius r (0.6–1.0) are discarded, and the metric computes the bidirectional closest point-to-mesh distance.

Input / output format

Input: High-resolution RGB image of a face.

Output: Predicted 3D facial mesh vertices and/or 3D facial landmarks.

Scoring recipe

def compute_nme(pred_lms, gt_lms):
    errors = [l2_dist(p, g) for p, g in zip(pred_lms, gt_lms)]
    bbox_size = max(gt_lms) - min(gt_lms)
    return mean(errors) / bbox_size

def compute_armse(pred_mesh, gt_mesh, gt_lms, r):
    interocular = l2_dist(gt_lms['left_eye'], gt_lms['right_eye'])
    gt_norm = gt_mesh / interocular
    pred_aligned = align_landmarks(pred_mesh, gt_lms)
    pred_shifted = pred_aligned - pred_aligned['nose_tip']
    gt_shifted = gt_norm - gt_norm['nose_tip']
    valid_pred = pred_shifted[dist(pred_shifted, pred_shifted['nose_tip']) <= r]
    valid_gt = gt_shifted[dist(gt_shifted, gt_shifted['nose_tip']) <= r]
    return (chamfer_dist(valid_pred, valid_gt) + chamfer_dist(valid_gt, valid_pred)) / 2

Common pitfalls

• ARMSE scores are highly sensitive to the crop radius r (0.6–1.0); results must explicitly state the r value used.
• Interocular normalization and landmark-based alignment are mandatory for ARMSE; skipping them breaks metric comparability.
• Models trained on synthetic data suffer significant domain gaps on Pixel-Face, often requiring fine-tuning on real data to achieve competitive scores.

Evidence (verbatim from paper)

The Normalized Mean Error (NME) is defined as the average of landmark errors normalized by the bounding box sizes. The Average Root Mean Square Error (ARMSE) is employed to evaluate the similarity between reconstructed 3D meshes and ground truth meshes. Following the setting of 2nd 3DFAW challenge, we first normalize the interocular distance of ground truth to 1. Then we align the reconstructed 3D meshes to the ground truth by facial landmarks. The origin is set to be the nose tip. Given a crop radius which is marked as r, we discard vertices whose distance between nose tip is higher than r. The ARMSE computes the closet point-to-mesh distance between the ground-truth and reconstructed 3D meshes and vice versa.

Citation

@misc{lyu2020pixelface,
  title={Pixel-Face: A Large-Scale, High-Resolution Benchmark for 3D Face Reconstruction},
  author={Lyu et al. (2020)},
  year={2020},
  note={arXiv:2008.12444}
}

• arXiv: 2008.12444