# Biobert Re Eval

> Tests a model's capability to extract biomedical relations (gene-disease, gene-chemical) from text using minimal task-specific modifications. It probes whether domain-specific pre-training improves relation classification on small-scale biomedical corpora. Use when the user wants to benchmark on GAD, EU-ADR, CHEMPROT, or asks about evaluating this task. Reports entity-level F1.

- Skill: `qhjqhj00/biobert-re-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/biobert-re-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/biobert-re-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/biobert-re-eval

---


# biobert-re-eval

> BioBERT: a pre-trained biomedical language representation model for biomedical text mining — Lee et al. (2019) (arXiv:1901.08746, 2019)

## What this evaluates

Tests a model's capability to extract biomedical relations (gene-disease, gene-chemical) from text using minimal task-specific modifications. It probes whether domain-specific pre-training improves relation classification on small-scale biomedical corpora.

## Datasets

- **GAD** — total ?; splits: test (-1)
- **EU-ADR** — total ?; splits: test (-1)
- **CHEMPROT** — total ?; splits: test (-1)

## Metrics

- `entity-level F1` **(primary)** — range: percent
  - Harmonic mean of precision and recall calculated at the entity pair level. P = correct relations / predicted relations, R = correct relations / gold relations, F1 = 2PR/(P+R).

## Input / output format

**Input**: Biomedical text containing entity pairs.

**Output**: Predicted relation labels between entity pairs.

## Scoring recipe

```python
def compute_entity_f1(preds, gold):
  pred_set = set(preds)
  gold_set = set(gold)
  tp = len(pred_set & gold_set)
  fp = len(pred_set - gold_set)
  fn = len(gold_set - pred_set)
  p = tp / (tp + fp) if (tp + fp) > 0 else 0
  r = tp / (tp + fn) if (tp + fn) > 0 else 0
  f1 = 2 * p * r / (p + r) if (p + r) > 0 else 0
  return p, r, f1
```

## Common pitfalls

- Using complex linguistic features or multi-task learning architectures that differ from the single-architecture setup used for BERT/BioBERT.
- Not reporting entity-level metrics as specified in the protocol.
- Averaging results across datasets without weighting or reporting per-dataset scores separately.

## Evidence (verbatim from paper)

> The RE results are shown in Table 5. Unlike the NER results, BERT achieves better performance than that of the state-of-the-art models, which demonstrates its effectiveness in RE. On average, BioBERT (+ PubMed + PMC) outperformed the state-of-the-art models by 3.49 in terms of F1 score. On 2 out of 3 biomedical RE datasets, BioBERT achieved state-of-the-art performance in terms of F1 score. Considering the complexity of the state-of-the-art models which use numerous linguistic features (Bhasuran and Natarajan, 2018), the results are very promising.

## Citation

```bibtex
@misc{lee2019biobert,
  title={BioBERT: a pre-trained biomedical language representation model for biomedical text mining},
  author={Lee et al. (2019)},
  year={2019},
  note={arXiv:1901.08746}
}
```

- arXiv: 1901.08746