Duet Dyadic Har Eval

duet-dyadic-har-eval

Decoding Psychological States Through Movement: Inferring Human Kinesic Functions with Application to Built Environments — Lin et al. (2026) (arXiv:2601.17194, 2026)

What this evaluates

Evaluates the ability of human activity recognition models to classify dyadic kinesic functions and interactions across different subjects and physical locations. It probes robustness to viewpoint changes, background variations, occlusion, and modality-specific limitations (RGB vs. depth vs. 3D skeletons).

Datasets

• DUET — total ?; splits: train (-1), test (-1)

Metrics

• Cross-location accuracy (%) (primary) — range: percent
  • Percentage of correctly classified dyadic interaction samples when the model is trained on two physical locations and tested on a held-out third location.
• Cross-subject accuracy (%) (primary) — range: percent
  • Percentage of correctly classified dyadic interaction samples when the model is trained on 80% of subjects and tested on the remaining 20%.

Input / output format

Input: RGB video frames, depth maps, or 3D skeleton joint coordinates representing dyadic human interactions.

Output: Class label corresponding to a specific dyadic kinesic function or interaction type.

Scoring recipe

def calculate_accuracy(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    return (correct / len(gold_labels)) * 100

Common pitfalls

• RGB and depth models often learn background or view-dependent cues rather than motion patterns, causing severe performance drops in cross-location evaluation.
• 3D skeleton representations are sparse and may fail to capture fine-grained gesture distinctions (e.g., thumbs up vs. palms out).
• Occlusion significantly degrades RGB-based recognition accuracy, requiring careful handling or exclusion of occluded samples for fair comparison.

Evidence (verbatim from paper)

Overall, the cross-subject evaluation outperforms the cross-location evaluation across all modalities in the state-of-the-art algorithms, which can be explained by two key factors.

Citation

@misc{lin2026decoding,
  title={Decoding Psychological States Through Movement: Inferring Human Kinesic Functions with Application to Built Environments},
  author={Lin et al. (2026)},
  year={2026},
  note={arXiv:2601.17194}
}

• arXiv: 2601.17194