Simpletoolhallubench Eval

simpletoolhallubench-eval

The Reasoning Trap: How Enhancing LLM Reasoning Amplifies Tool Hallucination — Chenlong Yin et al. (2025) (arXiv:2510.22977, 2025)

What this evaluates

Probes an agent's ability to abstain from tool use when no appropriate tools are available, specifically measuring its tendency to hallucinate tool invocations in two controlled scenarios: when no tools are provided and when irrelevant distractor tools are present.

Datasets

• SimpleToolHalluBench — total 296; splits: test (296)

Metrics

• R_NTA (primary) — range: [0, 1]
  • Hallucination rate for the No-Tool-Available task, calculated as the fraction of responses flagged by an LLM-as-judge as hallucinating tool invocations: R_NTA = H_NTA / N_NTA.
• R_DT — range: [0, 1]
  • Hallucination rate for the Distractor-Tool task, calculated as the fraction of responses flagged by an LLM-as-judge as hallucinating tool invocations: R_DT = H_DT / N_DT.

Input / output format

Input: User query paired with a system prompt that either provides no tools (NTA) or provides an irrelevant distractor tool (DT).

Output: Model's generated response, which may include a tool invocation, a refusal/abstention, or a hallucinated tool call.

Scoring recipe

# For each task (NTA or DT):
N = total number of samples in the task
H = 0
for response in model_responses:
    if llm_as_judge_flags_hallucination(response):
        H += 1
rate = H / N
return rate

Common pitfalls

• Queries are explicitly designed to be impossible without the specific required tool, so models should not rely on internal knowledge to answer them.
• Scoring relies on an LLM-as-judge to flag hallucinations, which may introduce variability or bias compared to deterministic tool-existence checks.
• The benchmark separates 'NTA' (no tools provided) and 'DT' (irrelevant tool provided) scenarios, so results should not be averaged without noting the distinct failure modes.

Evidence (verbatim from paper)

We calculate the hallucination rate for each task independently. These rates are defined as the fraction of responses flagged by the LLM-as-judge: R_NTA = H_NTA / N_NTA and R_DT = H_DT / N_DT, where H_NTA and H_DT are the counts of hallucinated responses in their respective tasks, and N_NTA and N_DT are the total number of samples in each task set.

Citation

@misc{yin2025reasoningtrap,
  title={The Reasoning Trap: How Enhancing LLM Reasoning Amplifies Tool Hallucination},
  author={Chenlong Yin et al. (2025)},
  year={2025},
  note={arXiv:2510.22977}
}

• arXiv: 2510.22977