qhjqhj00 video-to-4d-mesh-eval
ActionMesh: Animated 3D Mesh Generation with Temporal 3D Diffusion — Sabathier et al. (2026) (arXiv:2601.16148, 2026)
What this evaluates
Evaluates a model's ability to generate temporally consistent, animated 3D meshes from input videos. It probes per-frame geometric reconstruction accuracy, overall 4D sequence fidelity, and motion transfer quality while maintaining topology consistency across frames.
Datasets
- Objaverse — total 32; splits: test (32)
- Consistent4D — total ?; splits: test (-1)
- DAVIS — total ?; splits: test (-1)
Metrics
CD-3D(primary) — range: other (lower is better)- Per-frame Chamfer Distance. Aligns the predicted mesh to the ground-truth mesh for each frame using Iterative Closest Point (ICP), then computes the bidirectional chamfer distance between corresponding points.
CD-4D— range: other (lower is better)- 4D sequence Chamfer Distance. Aligns the entire predicted mesh sequence to the ground-truth sequence using a global ICP on the first frame, then averages the chamfer distance across all frames.
CD-M— range: other (lower is better)- Motion Chamfer Distance. After global ICP alignment, establishes nearest-neighbor correspondences using the first frame's mesh. Computes the bidirectional chamfer distance between corresponding points across all remaining frames to quantify motion fidelity.
Input / output format
Input: A video sequence (temporal frames of an object or scene)
Output: A sequence of 3D meshes (one per frame) with consistent topology representing the animated object
Scoring recipe
def compute_metrics(pred_meshes, gt_meshes):
# CD-3D: per-frame ICP alignment & chamfer
cd3d = sum(chamfer_distance(icp_align(p, g), g) for p, g in zip(pred_meshes, gt_meshes)) / len(pred_meshes)
# CD-4D: global ICP on first frame, then average chamfer
p_aligned = global_icp_align(pred_meshes, gt_meshes[0])
cd4d = sum(chamfer_distance(p, g) for p, g in zip(p_aligned, gt_meshes)) / len(pred_meshes)
# CD-M: motion fidelity via first-frame correspondences
cd_m = sum(chamfer_distance(p_aligned[i], gt_meshes[i]) for i in range(1, len(pred_meshes))) / (len(pred_meshes) - 1)
return cd3d, cd4d, cd_m
Common pitfalls
- ICP alignment is sensitive to initialization and topology mismatches; improper alignment can artificially inflate CD scores.
- The primary quantitative benchmark (Objaverse subset) contains only 32 scenes and is in-house, limiting generalizability and cross-paper comparability.
- CD-M assumes stable point correspondences from the first frame; severe self-occlusion or non-rigid deformation can break this assumption.
Evidence (verbatim from paper)
First, we evaluate the per-frame 3D reconstruction quality by aligning, for each frame, the predicted mesh with ICP and computing the chamfer distance between ground-truth and prediction (CD-3D). Second, the 4D reconstruction quality is evaluated by aligning the predicted mesh sequence with a global ICP applied on the first mesh, and averaging the chamfer distance (CD-4D). Third, we evaluate motion fidelity with a chamfer-like distance tailored to quantify motion (CD-M). Specifically, after aligning the mesh sequence with a global ICP, we establish nearest neighbor correspondences using the first mesh. Then, for each remaining frame, we evaluate the bidirectional distance between corresponding points.
Citation
@misc{sabathier2026actionmesh,
title={ActionMesh: Animated 3D Mesh Generation with Temporal 3D Diffusion},
author={Sabathier et al. (2026)},
year={2026},
note={arXiv:2601.16148}
}
- arXiv: 2601.16148