Utd Mhad Eval

utd-mhad-eval

Improving Human Motion Prediction Through Continual Learning — Yasar et al. (2021) (arXiv:2107.00544, 2021)

What this evaluates

Evaluates a model's ability to predict future human joint positions over a 15-frame horizon using past observations, while testing continual learning capabilities across different subjects and curriculum-based fine-tuning.

Datasets

• UTD-MHAD — total ?; splits: train (-1), val (-1), test (-1)

Metrics

• MSE (primary) — range: other
  • The $l_2$ distance between ground-truth and predicted poses at each timestep, averaged over the number of joints and sequence length. Formula: $\mathcal{L}(X,\hat{X}) = \frac{1}{T \cdot K} \sum_{t=1}^{T} \sum_{i=1}^{K} (x_t^i - \hat{x}_t^i)^2$, where T is total frames and K is total joints.

Input / output format

Input: 15-frame sequence of 20-joint skeleton coordinates.

Output: 15-frame sequence of 20-joint skeleton coordinates.

Scoring recipe

def compute_mse(predictions, ground_truth):
    # predictions, ground_truth: shape (T=15, K=20)
    diff = predictions - ground_truth
    mse = np.mean(diff ** 2)
    return mse

Common pitfalls

• The evaluation uses a strict cross-subject protocol (odd-numbered subjects for train/val, even-numbered for test) rather than random sequence splitting, which is essential for claiming generalization.
• Only skeleton data is used; RGB/depth modalities are explicitly excluded despite being available in the dataset.
• The zero-velocity baseline (predicting the last observed pose for all future frames) is a strong baseline for short-term prediction and must be reported alongside MSE to gauge movement dynamics.

Evidence (verbatim from paper)

We evaluated the performance of all models using the Mean Squared Error (MSE), which is the $l_{2}$ distance between the ground-truth and the predicted poses at each timestep, averaged over the number of joints and sequence length, in line with prior work. The MSE is calculated as: $\mathcal{L(X,\hat{X})}=\frac{1}{T.K}\sum^{T}{t=1}\sum^{K}{i=1}(x_{t}^{i}-\hat{x}_{t}^{i})^{2}$ where, T and K are the total number of frame and joints respectively.

Citation

@misc{yasar2021improving,
  title={Improving Human Motion Prediction Through Continual Learning},
  author={Yasar et al. (2021)},
  year={2021},
  note={arXiv:2107.00544}
}

• arXiv: 2107.00544