Hand Pose Estimation Eval

hand-pose-estimation-eval

A Comparison of Directional Distances for Hand Pose Estimation — Tzionas and Gall (2017) (arXiv:1704.00492, 2017)

What this evaluates

Evaluates the accuracy of 3D hand pose estimation methods by measuring joint localization error on isolated frame pairs. It probes how well different directional distance metrics handle orientation information and varying temporal offsets between frames.

Datasets

• Synthetic dataset — total ?; splits: test (-1)
• Realistic dataset — total ?; splits: test (-1)

Metrics

• average joint error (primary) — range: mm
  • Average Euclidean distance (in mm) between estimated and ground-truth 3D joint positions across all joints per frame.
• percentage frames below threshold — range: percent
  • Percentage of test frames where the average joint error falls below a specified threshold value.

Input / output format

Input: Hand silhouettes/images and ground truth 3D joint coordinates. The benchmark evaluates isolated frame pairs with varying temporal offsets (1, 5, 10, 15 frames) to control difficulty.

Output: Estimated 3D joint coordinates for the hand.

Scoring recipe

def compute_metrics(predictions, ground_truth, threshold=5.0):
    errors = []
    for pred, gt in zip(predictions, ground_truth):
        errors.append(np.mean(np.linalg.norm(pred - gt, axis=1)))
    mean_err = np.mean(errors)
    std_err = np.std(errors)
    pct_below = np.mean(np.array(errors) < threshold) * 100
    return mean_err, std_err, pct_below

Common pitfalls

• Using unsigned circular distance (180) for orientation instead of signed (360) significantly degrades performance.
• Assuming synthetic dataset performance directly predicts real-world accuracy; the paper notes synthetic data can be misleading (e.g., CH performs best on synthetic but worst on real).
• Ignoring frame differences/temporal offsets when comparing methods, as performance degrades with larger offsets (1 to 15 frames).

Evidence (verbatim from paper)

As measure, we use the average joint error per test frame and compute the percentage of frames with an error below a given threshold. ... Table 1: Mean error±std.dev.(mm), av.time (sec) for 1,5,10,15 frame differences.

Citation

@misc{tzionas2017comparison,
  title={A Comparison of Directional Distances for Hand Pose Estimation},
  author={Tzionas and Gall (2017)},
  year={2017},
  note={arXiv:1704.00492}
}

• arXiv: 1704.00492