Rehab Pile Eval

rehab-pile-eval

Deep Learning for Skeleton Based Human Motion Rehabilitation Assessment: A Benchmark — Ali Ismail-Fawaz et al. (2025) (arXiv:2507.21018, 2025)

What this evaluates

This benchmark evaluates deep learning models on skeleton-based human motion rehabilitation assessment. It probes both classification (e.g., motion type or health status) and extrinsic regression tasks using standardized cross-subject splits and min-max normalization to prevent data leakage.

Datasets

• Rehab-Pile — total ?; splits: train (-1), test (-1); repo https://github.com/MSD-IRIMAS/DeepRehabPile

Metrics

• accuracy (primary) — range: [0, 1]
  • Proportion of correctly predicted samples: (1/N_test) * sum(1[y_i == y_hat_i])
• balanced accuracy — range: [0, 1]
  • Average recall across all classes: (1/C) * sum_c (correct_c / total_c)
• RMSE — range: [0, inf)
  • Root Mean Squared Error: sqrt((1/N_test) * sum((y_i - y_hat_i)^2))
• MAE — range: [0, inf)
  • Mean Absolute Error: (1/N_test) * sum(|y_i - y_hat_i|)

Input / output format

Input: 3D tensor of shape (L, J, D) representing skeleton sequences (frames, joints, dimensions), typically flattened to (L, J*D) multivariate time series. Features are min-max normalized using training set statistics.

Output: Classification: predicted class label (argmax of output probabilities). Regression: continuous value bounded to [0.0, 1.0] during inference, then de-normalized by multiplying by the dataset-specific maximum value.

Scoring recipe

def compute_metrics(y_true, y_pred, is_classification=True):
    if is_classification:
        y_pred_cls = np.argmax(y_pred, axis=1)
        acc = np.mean(y_true == y_pred_cls)
        balanced_acc = np.mean([np.mean(y_true[y_true==c] == y_pred_cls[y_true==c]) for c in np.unique(y_true)])
        return acc, balanced_acc
    else:
        rmse = np.sqrt(np.mean((y_true - y_pred)**2))
        mae = np.mean(np.abs(y_true - y_pred))
        return rmse, mae

Common pitfalls

• Normalization parameters must be computed exclusively on the training set to prevent data leakage into the test set.
• Splits must follow a strict cross-subject protocol; no subject's data can appear in both training and testing folds.
• Regression metrics require de-normalizing predictions by multiplying with the dataset-specific maximum value before calculation.
• Final reported scores must average predictions across 5 random initializations per fold, then average the resulting metrics across all folds.

Evidence (verbatim from paper)

We use both accuracy and balanced accuracy to account for potential class imbalance. ... We use both the Root Mean Squared Error (RMSE) and the Mean Absolute Error (MAE) as evaluation metrics for regression tasks.

Citation

@misc{ismailfawaz2025deeprehabpile,
  title={Deep Learning for Skeleton Based Human Motion Rehabilitation Assessment: A Benchmark},
  author={Ali Ismail-Fawaz et al. (2025)},
  year={2025},
  note={arXiv:2507.21018}
}

• arXiv: 2507.21018