qhjqhj00 evidencenet-extraction-eval
Building evidence-based knowledge graphs from full-text literature for disease-specific biomedical reasoning — Chang Zong et al. (2026) (arXiv:2603.28325, 2026)
What this evaluates
This evaluation probes the fidelity of an LLM-assisted pipeline in extracting structured, PICO-style evidence nodes from unstructured full-text biomedical literature, and the precision of subsequent entity normalization against a reference resource.
Datasets
- HCC and CRC PubMed corpus — total ?; splits: (unstated); repo https://github.com/ZUST-BIT/EvidenceNet-code
Metrics
field-level extraction accuracy(primary) — range: [0, 1]- Proportion of extracted evidence fields that correctly match ground-truth annotations.
entity-link accuracy— range: [0, 1]- Proportion of normalized biomedical entities that correctly align with the TarKG reference resource.
Input / output format
Input: Full-text biomedical articles segmented into 3,000-character chunks with 300-character overlap, preserving section labels.
Output: Structured evidence objects containing PICO fields (study object, intervention, comparison, outcomes, mechanism, phenotype, design, stage) and quantitative attributes (p-values, sample size, fold change).
Scoring recipe
Not explicitly defined in the provided text. The paper reports 98.3% field-level extraction accuracy and 100% high-confidence entity-link accuracy, implying exact or fuzzy string matching against curated gold standards. A representative implementation would be:
def compute_accuracy(preds, gold):
correct = sum(1 for p, g in zip(preds, gold) if p == g)
return correct / len(gold)
Common pitfalls
- Chunks shorter than 500 characters are discarded during preprocessing, which may bias evaluation results toward longer, more methodologically detailed studies.
- Document-level aggregation removes exact duplicates across sections, potentially undercounting repeated findings and affecting the granularity of extraction accuracy calculations.
Evidence (verbatim from paper)
Validation shows 98.3% field-level extraction accuracy, 100% high-confidence entity-link accuracy, and significant improvements in retrieval-augmented QA and link prediction, enabling more reliable, context-aware biomedical reasoning for precision medicine applications.
Citation
@misc{zong2026evidencenet,
title={Building evidence-based knowledge graphs from full-text literature for disease-specific biomedical reasoning},
author={Chang Zong et al. (2026)},
year={2026},
note={arXiv:2603.28325}
}
- arXiv: 2603.28325