# Mfmdqwen Eval

> Evaluates large language models on multilingual financial misinformation detection across nine tasks in English, Chinese, Greek, and Bengali. It probes the model's ability to identify false or misleading financial claims, handle numerical sensitivity and reversed causality, and generalize across diverse linguistic settings. Use when the user wants to benchmark on MFMDBench, or asks about evaluating this task. Reports Macro-F1.

- Skill: `qhjqhj00/mfmdqwen-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/mfmdqwen-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/mfmdqwen-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/mfmdqwen-eval

---


# mfmdqwen-eval

> MFMDQwen: Multilingual Financial Misinformation Detection Based on Large Language Model — Liu et al. (2026) (arXiv:2604.18272, 2026)

## What this evaluates

Evaluates large language models on multilingual financial misinformation detection across nine tasks in English, Chinese, Greek, and Bengali. It probes the model's ability to identify false or misleading financial claims, handle numerical sensitivity and reversed causality, and generalize across diverse linguistic settings.

## Datasets

- **MFMDBench** — total ?; splits: test (-1); repo https://github.com/lzw108/FMD

## Metrics

- `Accuracy` — range: [0, 1]
  - Standard classification accuracy, calculated as the number of correct predictions divided by the total number of predictions.
- `Macro-F1` **(primary)** — range: [0, 1]
  - Macro-averaged F1 score, computed as the unweighted mean of the F1 scores for each class, treating all classes equally regardless of support.

## Input / output format

**Input**: Multilingual textual financial claims or news snippets (English, Chinese, Greek, or Bengali), often accompanied by evidence or context.

**Output**: Binary classification label indicating whether the input contains financial misinformation.

## Scoring recipe

```python
def compute_metrics(predictions, gold_labels):
    accuracy = sum(p == g for p, g in zip(predictions, gold_labels)) / len(gold_labels)
    macro_f1 = 0.0
    for label in unique_labels:
        tp = sum(1 for p, g in zip(predictions, gold_labels) if p == label and g == label)
        fp = sum(1 for p, g in zip(predictions, gold_labels) if p == label and g != label)
        fn = sum(1 for p, g in zip(predictions, gold_labels) if p != label and g == label)
        precision = tp / (tp + fp) if (tp + fp) > 0 else 0
        recall = tp / (tp + fn) if (tp + fn) > 0 else 0
        f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0
        macro_f1 += f1
    macro_f1 /= len(unique_labels)
    return accuracy, macro_f1
```

## Common pitfalls

- Models with strict safety mechanisms may refuse to answer or output irrelevant responses, artificially lowering F1 scores.
- Reasoning-capable models may underperform non-reasoning counterparts of the same size due to overthinking or generating incorrect intermediate steps.
- Performance on low-resource languages (e.g., Bengali) may appear near-saturated due to limited dataset size and reduced task complexity rather than true model capability.

## Evidence (verbatim from paper)

> We uses metrics such as Accuracy, Macro-F1 for misinformation detection evaluation. In the following analysis, we primarily focus on the F1 score.

## Citation

```bibtex
@misc{liu2026mfmdqwen,
  title={MFMDQwen: Multilingual Financial Misinformation Detection Based on Large Language Model},
  author={Liu et al. (2026)},
  year={2026},
  note={arXiv:2604.18272}
}
```

- arXiv: 2604.18272