# Ragsearch Eval

> Evaluates dense RAG and GraphRAG retrieval backends when integrated into agentic search systems. It probes the agent's ability to dynamically retrieve, reason, and answer general and multi-hop QA queries under both training-free prompting and reinforcement learning paradigms. Use when the user wants to benchmark on NQ, PopQA, TriviaQA, HotpotQA, 2Wiki, Musique, or asks about evaluating this task. Reports Exact Match (EM).

- Skill: `qhjqhj00/ragsearch-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/ragsearch-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/ragsearch-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/ragsearch-eval

---


# ragsearch-eval

> Do We Still Need GraphRAG? Benchmarking RAG and GraphRAG for Agentic Search Systems — Fan et al. (2026) (arXiv:2604.09666, 2026)

## What this evaluates

Evaluates dense RAG and GraphRAG retrieval backends when integrated into agentic search systems. It probes the agent's ability to dynamically retrieve, reason, and answer general and multi-hop QA queries under both training-free prompting and reinforcement learning paradigms.

## Datasets

- **NQ** — total ?; splits: test (-1)
- **PopQA** — total ?; splits: test (-1)
- **TriviaQA** — total ?; splits: test (-1)
- **HotpotQA** — total ?; splits: test (-1)
- **2Wiki** — total ?; splits: test (-1)
- **Musique** — total ?; splits: test (-1)

## Metrics

- `Exact Match (EM)` **(primary)** — range: [0, 1]
  - 1 if the predicted answer exactly matches the gold answer string, 0 otherwise. Reported as the overall percentage across the dataset.

## Input / output format

**Input**: A natural language query q and a retrieval backend B (dense RAG or GraphRAG). The agent interacts iteratively, receiving retrieved text chunks or subgraphs wrapped in <information> tags.

**Output**: A final answer enclosed within <answer> and </answer> tags, following an interleaved sequence of <think> and <search> steps.

## Scoring recipe

```python
def compute_em(predictions, golds):
    correct = 0
    for pred, gold in zip(predictions, golds):
        if pred.strip() == gold.strip():
            correct += 1
    return correct / len(golds)
```

## Common pitfalls

- Confusing the agentic control policy with the retrieval backend; the benchmark keeps the agent protocol identical across Dense RAG and GraphRAG to isolate retrieval infrastructure effects.
- Failing to account for the multi-step agentic loop; single-shot baselines are evaluated separately from training-free and RL-based multi-round agents.
- GraphRAG backends require offline graph construction, but the benchmark metric only measures runtime answer correctness, not preprocessing time or cost.

## Evidence (verbatim from paper)

> In RAGSearch, rewards are defined at the trajectory level and focus on task correctness and output validity. Specifically, we combine (i) an outcome-based reward that measures answer correctness (e.g., exact match or task-specific accuracy), and (ii) a format-based reward that encourages the agent to follow the expected interaction and answer format.

## Citation

```bibtex
@misc{fan2026ragsearch,
  title={Do We Still Need GraphRAG? Benchmarking RAG and GraphRAG for Agentic Search Systems},
  author={Fan et al. (2026)},
  year={2026},
  note={arXiv:2604.09666}
}
```

- arXiv: 2604.09666