qhjqhj00 mobiface-lfw-megaface-eval
MobiFace: A Lightweight Deep Learning Face Recognition on Mobile Devices — Chi Nhan Duong et al. (2018) (arXiv:1811.11080, 2018)
What this evaluates
Evaluates the accuracy and robustness of lightweight face recognition models on unconstrained face verification tasks, specifically measuring pair-wise verification accuracy and identification rates under extreme distractor conditions.
Datasets
- Labeled Faces in the Wild (LFW) — total 13233; splits: test (6000)
- MegaFace — total ?; splits: test (100200)
Metrics
Accuracy(primary) — range: percent- Percentage of correctly verified face pairs out of the 6000 fixed test pairs.
TAR@FAR=10^-6(primary) — range: percent- True Accepted Rate at a False Accepted Rate of 10^-6, measuring identification accuracy when allowing one false match per million probe-gallery comparisons.
Input / output format
Input: Aligned and cropped face images of size 112x112x3, normalized to [-1, 1] by subtracting 127.5 and dividing by 128. Input consists of face pairs for verification or probe-gallery pairs for identification.
Output: Verification score or binary decision per pair; identification rank or match decision per probe image against a gallery.
Scoring recipe
def score_lfw(predictions, gold):
correct = sum(1 for p, g in zip(predictions, gold) if p == g)
return correct / 6000 * 100
def score_megaface(scores, probes, gallery):
far_threshold = compute_threshold_for_far(scores, 1e-6)
correct = sum(1 for p in probes if p.id in gallery and model(p, gallery) > far_threshold)
return correct / len(probes) * 100
Common pitfalls
- LFW protocol uses exactly 6000 fixed pairs; accuracy must be computed over this specific set, not the full 13k images.
- MegaFace evaluation is highly sensitive to the 1M distractor gallery; TAR@FAR=1e-6 requires precise threshold calibration to avoid inflated rates due to false matches.
Evidence (verbatim from paper)
According to the testing protocol of LFW, there are 6000 face pairs where half of them are positive pairs. ... The accuracy is reported on the True Accepted Rate (TAR) at the False Accepted Rate (FAR) of 10^-6.
Citation
@misc{duong2018mobiface,
title={MobiFace: A Lightweight Deep Learning Face Recognition on Mobile Devices},
author={Chi Nhan Duong et al. (2018)},
year={2018},
note={arXiv:1811.11080}
}
- arXiv: 1811.11080