Blmore Challenge Eval

blmore-challenge-eval

Foundation Model Embeddings Meet Blended Emotions: A Multimodal Fusion Approach for the BLEMORE Challenge — Chapariniya et al. (2026) (arXiv:2603.23650, 2026)

What this evaluates

Evaluates multimodal emotion recognition systems on detecting blended emotions (presence and salience) across unseen actors. It probes the model's ability to generalize actor-invariant emotional semantics from audio, visual, and combined modalities under a strict threshold-based discretization protocol.

Datasets

• BLEMORE — total 3050; splits: train (2456), test (594)

Metrics

• ACC_P — range: [0, 1]
  • Presence accuracy: fraction of clips where the discretized presence prediction (score ≥ α) matches the ground truth presence label.
• ACC_S — range: [0, 1]
  • Salience accuracy: fraction of clips where the discretized salience prediction (score ≥ β) matches the ground truth salience label.
• Score (primary) — range: [0, 1]
  • Official BLEMORE challenge metric. Continuous outputs are thresholded using α (presence) and β (salience) via grid search on validation data. Score is the official challenge metric (typically the average of ACC_P and ACC_S).

Input / output format

Input: Video clips containing audio and visual modalities (face, body, or full-body sequences).

Output: Continuous presence and salience scores per emotion class, discretized via thresholds α (presence) and β (salience).

Scoring recipe

def compute_blemore_metrics(predictions, ground_truth, alpha, beta):
    pred_presence = (predictions.presence >= alpha).astype(int)
    pred_salience = (predictions.salience >= beta).astype(int)
    acc_p = (pred_presence == ground_truth.presence).mean()
    acc_s = (pred_salience == ground_truth.salience).mean()
    score = (acc_p + acc_s) / 2
    return acc_p, acc_s, score

Common pitfalls

• Threshold β instability: Optimal salience threshold varies drastically (0.05–0.43) across folds due to actor-specific expression diversity, causing significant validation-to-test performance gaps.
• Actor-disjoint evaluation: Test set uses completely unseen actors, making models prone to overfitting to actor-specific visual/audio cues rather than general emotional semantics.

Evidence (verbatim from paper)

Following the official BLEMORE protocol, continuous outputs are discretized via presence threshold α and salience threshold β, selected by grid search on validation data. ... Our 12-encoder ensemble with Gemini achieves ACC_P = 0.391, substantially outperforming the best baseline presence (VideoMAEv2 + HuBERT, 0.332).

Citation

@misc{chapariniya2026blended,
  title={Foundation Model Embeddings Meet Blended Emotions: A Multimodal Fusion Approach for the BLEMORE Challenge},
  author={Chapariniya et al. (2026)},
  year={2026},
  note={arXiv:2603.23650}
}

• arXiv: 2603.23650