Fraud R1 Eval

fraud-r1-eval

Fraud-R1 : A Multi-Round Benchmark for Assessing the Robustness of LLM Against Augmented Fraud and Phishing Inducements — Yang et al. (2025) (arXiv:2502.12904, 2025)

What this evaluates

This benchmark evaluates large language models' robustness against multi-round fraud and phishing inducements. It probes whether models can successfully identify and defend against deceptive prompts across five fraud categories under both standard helpful-assistant and role-play settings, while also measuring cross-lingual performance gaps.

Datasets

• Fraud-R1 — total 8564; splits: test (8564); repo https://github.com/kaustpradalab/Fraud-R1

Metrics

• Defense Success Rate (DSR) (primary) — range: percent
  • The percentage of fraud cases where the model successfully identifies the inducement and refuses or defends against it across multiple conversation rounds. Calculated as (number of successfully defended cases / total cases) × 100.

Input / output format

Input: Multi-turn conversational prompts simulating fraud/phishing scenarios (fake job postings, impersonation, phishing scams, online relationship, fraudulent service) in English and Chinese, delivered under either a standard Helpful-Assistant or a Role-play instruction.

Output: The model's generated response at each turn of the multi-round conversation.

Scoring recipe

successful_count = 0
for case in dataset:
    response = model.generate(case.prompt)
    if judge_gpt4o_mini.evaluate(response, case.gold_label) == "success":
        successful_count += 1
dsr = (successful_count / len(dataset)) * 100

Common pitfalls

• Role-play prompts significantly degrade fraud detection performance compared to standard assistant prompts.
• Models show a pronounced performance gap between English and Chinese inputs, with English generally yielding higher DSR.
• Multi-round escalation increases the number of turns needed for detection, raising real-world financial risk.

Evidence (verbatim from paper)

Table[2] presents the comprehensive Defense Success Rate (DSR) of different LLMs, including overall and category-specific scores across Helpful Assistant and Role-play settings. Our evaluation employs GPT-4o-mini as an automated judge to assess the responses of the model over multiple rounds.

Citation

@misc{yang2025fraudr1,
  title={Fraud-R1 : A Multi-Round Benchmark for Assessing the Robustness of LLM Against Augmented Fraud and Phishing Inducements},
  author={Yang et al. (2025)},
  year={2025},
  note={arXiv:2502.12904}
}

• arXiv: 2502.12904