Movicam Eval

movicam-eval

Physics-based Human Pose Estimation from a Single Moving RGB Camera — Aytekin et al. (2025) (arXiv:2507.17406, 2025)

What this evaluates

Evaluates monocular 3D human pose and trajectory estimation in global coordinates, specifically probing a model's ability to maintain physical plausibility (e.g., avoiding scene penetration, minimizing foot sliding) while tracking dynamic camera motion on complex, non-flat terrain.

Datasets

• MoviCam — total ?; splits: test (-1); repo https://github.com/aidilayce/physdynpose

Metrics

• MPJPE (primary) — range: mm
  • Mean Per Joint Position Error in mm. Computed as the average Euclidean distance between predicted and ground-truth 3D joint positions.
• PA-MPJPE — range: mm
  • Procrustes-aligned MPJPE in mm. MPJPE computed after rigid alignment (rotation + translation) of predictions to ground truth.
• W-MPJPE — range: mm
  • World-aligned MPJPE in mm. MPJPE after aligning the initial frames of predictions and ground-truth data in world coordinates.
• WA-MPJPE — range: mm
  • World-aligned All-trajectories MPJPE in mm. MPJPE after aligning all trajectories in world coordinates.
• RTE — range: percent
  • Root Translation Error in %. Normalized by the subject’s actual displacement, calculated over the entire trajectory after rigid alignment.
• Jitter — range: mm/s
  • Temporal smoothness error in mm/s. Measures frame-to-frame velocity changes of the pose.
• FS — range: mm
  • Foot Sliding in mm. Average toe joint displacement during contact phases.
• % frames with scene penetration — range: percent
  • Percentage of frames where the predicted human mesh intersects the ground-truth scene geometry.
• Average penetration depth — range: mm
  • Average penetration depth in mm. Mean intersection depth per frame where penetration occurs.
• Average height above scene — range: mm
  • Average height above the scene in mm. Mean vertical distance between the subject and the ground-truth height map.

Input / output format

Input: Monocular RGB video frames from a moving camera.

Output: Per-frame 3D human joint positions, root trajectory, and mesh vertices in global/world coordinates.

Scoring recipe

def compute_metrics(pred_joints, gt_joints, pred_root, gt_root, pred_mesh, gt_scene):
    mpjpe = np.mean(np.linalg.norm(pred_joints - gt_joints, axis=-1))
    pa_mpjpe = procrustes_align(pred_joints, gt_joints)
    w_mpjpe = align_initial_frames(pred_joints, gt_joints)
    wa_mpjpe = align_all_trajectories(pred_joints, gt_joints)
    rte = np.linalg.norm(pred_root - gt_root) / np.linalg.norm(gt_root[-1] - gt_root[0]) * 100
    jitter = compute_temporal_smoothness(pred_joints)
    fs = compute_foot_sliding(pred_joints, contact_mask)
    penetration_frames = detect_mesh_scene_intersection(pred_mesh, gt_scene)
    penetration_depth = compute_intersection_depth(pred_mesh, gt_scene)
    height_above = compute_vertical_distance(pred_mesh, gt_scene)
    return {
        'MPJPE': mpjpe, 'PA-MPJPE': pa_mpjpe, 'W-MPJPE': w_mpjpe,
        'WA-MPJPE': wa_mpjpe, 'RTE': rte, 'Jitter': jitter, 'FS': fs,
        '% penetration': np.mean(penetration_frames) * 100,
        'Avg penetration': np.mean(penetration_depth),
        'Avg height': np.mean(height_above)
    }

Common pitfalls

• Confusing PA-MPJPE (local alignment) with W-MPJPE/WA-MPJPE (global/world alignment), as the latter are critical for evaluating trajectory drift and physical plausibility in moving-camera settings.
• Ignoring physical plausibility metrics (penetration, foot sliding, height above scene) in favor of raw pose accuracy, since high PA-MPJPE does not guarantee realistic human-scene interaction.
• Failing to account for camera motion when evaluating global trajectory errors, as SLAM inaccuracies on flat terrain can artificially inflate RTE and W-MPJPE.

Evidence (verbatim from paper)

We evaluate the performance of the methods in two parts: (a) 3D reconstruction errors and (b) physical plausibility. 3D Reconstruction Errors. To evaluate 3D human pose and trajectory estimation accuracy, we compute Mean Per Joint Position Error (MPJPE) and Procrustes-aligned MPJPE (PA-MPJPE) in mm. We also report W-MPJPE, which is MPJPE after aligning the initial frames of predictions and ground-truth data, and WA-MPJPE, which is after aligning all trajectories. Additionally, we follow [[26]] in reporting Root Translation Error (RTE) as %, normalized by the subject’s actual displacement, calculated over the entire trajectory after rigid alignment.

Citation

@misc{aytekin2025physdynpose,
  title={Physics-based Human Pose Estimation from a Single Moving RGB Camera},
  author={Aytekin et al. (2025)},
  year={2025},
  note={arXiv:2507.17406}
}

• arXiv: 2507.17406