Ear Challenge Eval

ear-challenge-eval

Elderly Activity Recognition in the Wild: Results from the EAR Challenge — Duong (2025) (arXiv:2503.07821, 2025)

What this evaluates

Evaluates video action recognition models on classifying untrimmed real-world videos of elderly individuals into six daily activity categories. It probes robustness and generalization in wild, uncontrolled settings using a held-out test set.

Datasets

• EAR Challenge Test Set — total 4616; splits: test (4616)

Metrics

• accuracy (primary) — range: [0, 1]
  • Average accuracy of predictions on the test set, calculated as the number of correctly classified videos divided by the total number of videos.

Input / output format

Input: Untrimmed video files representing daily activities of elderly individuals in real-world settings.

Output: A single class label from six predefined categories: locomotion, manipulation, hygiene, eating, communication, and leisure.

Scoring recipe

def calculate_accuracy(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    return correct / len(gold_labels)

Common pitfalls

• The test set is untrimmed and unlabeled during the challenge, requiring models to handle variable-length videos without explicit temporal boundaries.
• Leaderboard rankings are determined solely by the private split (50% of test set), so public split performance may not reflect final ranking.
• Performance heavily depends on training duration; the authors note significant gains from 10 to 100 epochs, which may not be feasible under competition time limits.

Evidence (verbatim from paper)

The evaluation metric used for ranking submissions was the average accuracy of predictions on the test set.

Citation

@misc{duong2025ear,
  title={Elderly Activity Recognition in the Wild: Results from the EAR Challenge},
  author={Duong (2025)},
  year={2025},
  note={arXiv:2503.07821}
}

• arXiv: 2503.07821