Fabl Eval

fabl-eval

Simultaneous Feature and Body-Part Learning for Real-Time Robot Awareness of Human Behaviors — Han et al. (2017) (arXiv:1702.07474, 2017)

What this evaluates

Evaluates a joint learning framework (FABL) for real-time human behavior recognition using 3D skeletal data from depth sensors. It tests the method's ability to simultaneously select discriminative body parts and features for action classification across public benchmarks and a custom robot-interaction task.

Datasets

• MSR Action3D Dataset — total ?; splits: train (-1), test (-1)
• Cornell Activity Dataset 60 (CAD-60) — total ?; splits: train (-1), test (-1)
• Baxter Robot Serving Drinks Task — total 320; splits: train (160), test (160)

Metrics

• average recognition accuracy (primary) — range: percent
  • Percentage of correctly classified sequences out of the total number of test sequences. Computed as (number of correct predictions / total number of predictions) * 100.

Input / output format

Input: 3D skeletal joint coordinates per frame, transformed into four feature modalities: spatial joint displacement, temporal joint displacement, long-term temporal joint displacement, and spatial joint distance. Histograms of these features are computed to form a feature vector per sequence.

Output: A single discrete class label corresponding to the recognized human behavior or robot-interaction gesture.

Scoring recipe

def compute_accuracy(predictions, ground_truth):
    correct = sum(1 for p, g in zip(predictions, ground_truth) if p == g)
    total = len(ground_truth)
    return (correct / total) * 100 if total > 0 else 0.0

Common pitfalls

• Subject-wise split is strictly enforced: training data contains zero instances from subjects used in testing. Standard random splits will invalidate the evaluation.
• The custom Baxter robot dataset uses only 2 subjects and 20 repetitions each, yielding lower accuracy (~77%) than public benchmarks; results are not directly comparable in scale or generalization.
• Features are hand-crafted histograms of simple skeletal displacements/distance, not raw coordinates. The evaluation relies on this specific feature extraction pipeline.

Evidence (verbatim from paper)

When combined both structured sparsity-inducting norms to perform simultaneous feature and skeletal joint learning, our FABL method obtains an accuracy of 91.67%,

Citation

@misc{han2017fabl,
  title={Simultaneous Feature and Body-Part Learning for Real-Time Robot Awareness of Human Behaviors},
  author={Han et al. (2017)},
  year={2017},
  note={arXiv:1702.07474}
}

• arXiv: 1702.07474