# Hypothesis Composition Eval

> Assesses an LLM's capability to synthesize novel research hypotheses by combining a given research background with retrieved inspiration papers. It probes the model's ability to mutate and recombine scientific concepts into coherent, groundtruth-aligned proposals. Use when the user wants to benchmark on ResearchBench Hypothesis Composition, or asks about evaluating this task. Reports Normalized Composition Score.

- Skill: `qhjqhj00/hypothesis-composition-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/hypothesis-composition-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/hypothesis-composition-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/hypothesis-composition-eval

---


# hypothesis-composition-eval

> ResearchBench: Benchmarking LLMs in Scientific Discovery via Inspiration-Based Task Decomposition — Liu et al. (2025) (arXiv:2503.21248, 2025)

## What this evaluates

Assesses an LLM's capability to synthesize novel research hypotheses by combining a given research background with retrieved inspiration papers. It probes the model's ability to mutate and recombine scientific concepts into coherent, groundtruth-aligned proposals.

## Datasets

- **ResearchBench Hypothesis Composition** — total 1386; splits: test (1386)

## Metrics

- `Normalized Composition Score` **(primary)** — range: [0, 1]
  - Average score on a 0-5 Likert scale measuring coverage of key points in the groundtruth hypothesis, normalized by dividing by the maximum possible score (5).

## Input / output format

**Input**: Research background, groundtruth inspiration papers (2-3), and prompts for mutate/refine/recombine steps.

**Output**: A composed research hypothesis text.

## Scoring recipe

```python
def normalized_score(predictions, gold):
    raw = llm_judge_score(predictions, gold) # 0 to 5
    return raw / 5.0
```

## Common pitfalls

- Scoring relies on a 6-point Likert scale judged by LLM/experts, which may vary across runs or judges.
- Task uses groundtruth inspirations for evaluation, not retrieved ones, isolating composition ability from retrieval errors.

## Evidence (verbatim from paper)

> we use a 6-point Likert scale (from 0 to 5) to measure whether the generated hypothesis has covered the key points in the groundtruth hypothesis. To compute the generation accuracy, we normalize the average score by dividing it by the maximum possible score (5).

## Citation

```bibtex
@misc{liu2025researchbench,
  title={ResearchBench: Benchmarking LLMs in Scientific Discovery via Inspiration-Based Task Decomposition},
  author={Liu et al. (2025)},
  year={2025},
  note={arXiv:2503.21248}
}
```

- arXiv: 2503.21248