Momenta Misinformation Eval

momenta-misinformation-eval

MOMENTA: Mixture-of-Experts Over Multimodal Embeddings with Neural Temporal Aggregation for Misinformation Detection — Abdollahinejad et al. (2026) (arXiv:2604.16172, 2026)

What this evaluates

Evaluates a multimodal misinformation detection model's ability to classify fake vs. real news across heterogeneous datasets, measuring classification accuracy, ranking quality, and class-balanced performance under calibrated decision thresholds.

Datasets

• Fakeddit — total ?; splits: test (-1)
• MMCoVaR — total ?; splits: test (-1)
• Weibo — total ?; splits: test (-1)
• XFacta — total ?; splits: test (-1)

Metrics

• Accuracy — range: [0, 1]
  • Ratio of correctly classified instances to total instances.
• F1 (primary) — range: [0, 1]
  • Harmonic mean of Precision and Recall: 2 * (Prec * Rec) / (Prec + Rec).
• Macro-F1 — range: [0, 1]
  • Unweighted mean of F1 scores computed per class, providing class-balanced performance.
• AUC — range: [0, 1]
  • Area under the Receiver Operating Characteristic curve, measuring ranking quality across thresholds.
• MCC — range: other
  • Matthews Correlation Coefficient, a balanced measure for binary classification especially on imbalanced datasets.

Input / output format

Input: Multimodal social media posts containing paired text and image content.

Output: Binary classification label (fake vs. real) or continuous confidence score.

Scoring recipe

def compute_metrics(y_true, y_pred):
    tp = sum((y_true == 1) & (y_pred == 1))
    tn = sum((y_true == 0) & (y_pred == 0))
    fp = sum((y_true == 0) & (y_pred == 1))
    fn = sum((y_true == 1) & (y_pred == 0))
    acc = (tp + tn) / (tp + tn + fp + fn)
    prec = tp / (tp + fp) if (tp + fp) > 0 else 0
    rec = tp / (tp + fn) if (tp + fn) > 0 else 0
    f1 = 2 * prec * rec / (prec + rec) if (prec + rec) > 0 else 0
    return acc, prec, rec, f1

Common pitfalls

• Failing to calibrate decision thresholds per dataset before test evaluation, which can lead to suboptimal performance on imbalanced splits.
• Relying solely on Accuracy without reporting Macro-F1 or MCC, which masks poor performance on minority classes in imbalanced misinformation datasets.
• Comparing results across datasets without accounting for different score distributions and evaluation protocols used by prior baselines.

Evidence (verbatim from paper)

Following standard practice in binary misinformation detection, we report Accuracy, Precision, Recall, F1, Macro-F1, AUC, and MCC. We calibrate decision thresholds on each validation split before final test evaluation to avoid fixing an arbitrary global threshold across datasets with different score distributions

Citation

@misc{abdollahinejad2026momenta,
  title={MOMENTA: Mixture-of-Experts Over Multimodal Embeddings with Neural Temporal Aggregation for Misinformation Detection},
  author={Abdollahinejad et al. (2026)},
  year={2026},
  note={arXiv:2604.16172}
}

• arXiv: 2604.16172