Hri30 Action Surface Eval

hri30-action-surface-eval

Sociable and Ergonomic Human-Robot Collaboration through Action Recognition and Augmented Hierarchical Quadratic Programming — Tassi et al. (2022) (arXiv:2207.03435, 2022)

What this evaluates

Evaluates a vision-based framework for classifying object surfaces and recognizing human actions, and tests its integration into a human-robot collaboration controller for ergonomic task execution and subjective workload assessment.

Datasets

• HRI30 — total 2940; splits: train (-1), test (-1)
• Custom Surface Dataset — total ?; splits: train (-1), val (-1), test (-1)

Metrics

• accuracy (primary) — range: [0, 1]
  • Number of correctly classified instances divided by the total number of instances in the evaluation set.
• ergonomics_score ($e_s$) — range: other
  • Proprietary ergonomic risk metric computed by the AHQP controller based on human pose and robot end-effector constraints; lower values indicate better posture and lower ergonomic risk.
• NASA-TLX — range: other
  • Subjective workload questionnaire measuring six subscales (temporal demand, mental demand, effort, physical demand, performance, frustration) on a 0-20 scale.

Input / output format

Input: 224×224 pixel images for surface classification; 256×256 pixel video frames sampled at 1/4 rate for action recognition; real-time RGB-D camera feeds and human kinematic cues for HRC task execution.

Output: Binary surface classification label; action class label; robot end-effector trajectory and ergonomics score; NASA-TLX questionnaire responses.

Scoring recipe

def compute_accuracy(predictions, gold):
    correct = sum(1 for p, g in zip(predictions, gold) if p == g)
    return correct / len(gold)

def compute_ergonomics_score(human_pose, robot_ee_pose):
    # Proprietary AHQP internal metric
    return ahqp_controller.calculate_risk(human_pose, robot_ee_pose)

def compute_nasa_tlx(responses):
    # responses: dict of 6 subscales, each 0-20
    return {k: v for k, v in responses.items()}

Common pitfalls

• Datasets are custom/internal and not publicly released, preventing direct replication or external benchmarking.
• Ergonomics score $e_s$ is a proprietary metric specific to the AHQP framework, not a standard biomechanical measure like REBA or RULA.
• NASA-TLX performance (PE) scale is inverted (higher score = better performance), which can cause misinterpretation if not explicitly noted.

Evidence (verbatim from paper)

The comparison shows that the baseline Resnet-152 outperforms the ResNet-50, given the number of layers that allow for more detailed feature extraction. Instead, the models based on our approach increased in accuracy by +6% for the ResNet-152 and by +10% for the Resnet-50 network, reaching 100% accuracy.

Citation

@misc{tassi2022sociable,
  title={Sociable and Ergonomic Human-Robot Collaboration through Action Recognition and Augmented Hierarchical Quadratic Programming},
  author={Tassi et al. (2022)},
  year={2022},
  note={arXiv:2207.03435}
}

• arXiv: 2207.03435