Densemarks Eval

densemarks-eval

Densemarks: Learning Canonical Embeddings for Human Heads Images via Point Tracks — Dmitrii Pozdeev et al. (arXiv:2511.02830, 2025)

What this evaluates

Evaluates a model's ability to learn dense, pose-robust 3D canonical embeddings for human head images. It probes geometric fidelity in point matching, semantic consistency across identities, and robustness to occlusions and extreme poses.

Datasets

• CelebV-HQ — total 32000; splits: train (31900), test (100)
• Nersemble — total ?; splits: test (-1)

Metrics

• MAE (primary) — range: pixels
  • Mean absolute error between predicted and ground-truth 2D pixel coordinates of matched mesh vertices.
• RMSE — range: pixels
  • Root mean square error of the same 2D coordinate matching errors.
• ArcFace — range: [0, 1]
  • Cosine similarity of face recognition features between the warped source image and the target image.
• Met3R — range: [0, 1]
  • View-consistency metric measuring geometric alignment across views.

Input / output format

Input: Pairs of head images (source and target), resized to 512×512.

Output: Per-pixel 3D canonical embeddings (3-channel feature map) at 512×512 resolution.

Scoring recipe

# Same-person matching quality
gt_uv = project_mesh_vertices(gt_mesh)
pred_embs = model(img)
matched_uv = nearest_neighbor_match(pred_embs, gt_uv)
mae = mean(abs(matched_uv - gt_uv))
rmse = sqrt(mean((matched_uv - gt_uv)**2))

# Cross-person consistency
warped_src = warp(source_img, matched_uv)
arcface = compute_arcface(warped_src, target_img)
met3r = compute_met3r(warped_src, target_img)

Common pitfalls

• Ground truth correspondences are derived from GS2Mesh mesh projections rather than manual pixel annotations, which may introduce projection inaccuracies.
• Cross-person evaluation relies on image warping via nearest-neighbor embedding matching, which can be sensitive to occlusions and embedding dimensionality.
• Results are averaged over only 10 people from Nersemble, which may limit generalization claims.

Evidence (verbatim from paper)

To evaluate the quality of the mapping, we estimate face recognition similarity based on ArcFace between the source image and the mapping result, as well as the view-consistency metric Met3R

Citation

@misc{pozdeev2025densemarks,
  title={Densemarks: Learning Canonical Embeddings for Human Heads Images via Point Tracks},
  author={Dmitrii Pozdeev et al.},
  year={2025},
  note={arXiv:2511.02830}
}

• arXiv: 2511.02830