Human Behavior Atlas Eval

human-behavior-atlas-eval

Human Behavior Atlas: Benchmarking Unified Psychological and Social Behavior Understanding — Ong et al. (2025) (arXiv:2510.04899, 2025)

What this evaluates

Evaluates multimodal models' ability to understand and classify diverse psychological and social behaviors (e.g., emotion, sarcasm, depression, intent) across text, audio, and video inputs. It also tests transfer learning capabilities to held-out datasets and the impact of adding behavioral descriptors.

Datasets

• Human Behavior Atlas — total 101964; splits: train (-1), val (-1), test (-1)

Metrics

• Unified behavioral metrics (primary) — range: [0, 1]
  • Task-dependent: binary weighted F1 for SEN; mean per-class weighted accuracy for EMO; weighted F1 for HUM, SAR, ANX, DEP, PTSD; and LLM-Judge accuracy for open-ended tasks SOC, INT, NVC.

Input / output format

Input: Multimodal inputs (text, audio, video) with optional behavioral descriptors (MediaPipe facial keypoints, OpenSMILE audio features). For MMPSY (A), text-only input is used.

Output: Class labels for structured tasks (EMO, HUM, PTSD, ANX, DEP, SEN, SAR); free-form text responses for open-ended tasks (SOC, INT, NVC), evaluated via LLM-Judge.

Scoring recipe

def compute_metric(task, preds, gold):
    if task in ['SEN', 'HUM', 'SAR', 'ANX', 'DEP', 'PTSD']:
        return weighted_f1_score(gold, preds, average='weighted')
    elif task in ['EMO']:
        return accuracy_score(gold, preds, average='macro')
    elif task in ['SOC', 'INT', 'NVC']:
        return llm_judge_accuracy(gold, preds)
    else:
        raise ValueError('Unknown task')

Common pitfalls

• Different tasks require different metrics (F1 vs accuracy vs LLM-Judge); applying a single metric across all tasks will yield incorrect results.
• LLM-Judge is used for open-ended tasks (SOC, INT, NVC), introducing potential non-determinism and prompt-sensitivity.
• MMPSY (A) subset uses text-only input, which breaks the standard multimodal evaluation pipeline if not handled separately.
• Transfer learning experiments use a fixed minimal epoch budget (1 epoch, 2 for DAIC-WOZ); varying this budget changes comparability.

Evidence (verbatim from paper)

Following the unified metrics (Sec. [3.3]), we use binary weighted F1 for SEN; mean per-class weighted accuracy for EMO; weighted F1 for HUM, SAR, ANX, DEP, PTSD; and LLM-Judge accuracy for SOC, INT, NVC.

Citation

@misc{ong2025humanbehavioratlas,
  title={Human Behavior Atlas: Benchmarking Unified Psychological and Social Behavior Understanding},
  author={Ong et al. (2025)},
  year={2025},
  note={arXiv:2510.04899}
}

• arXiv: 2510.04899