qhjqhj00 dud-e-virtual-screening-eval
Protein-ligand binding representation learning from fine-grained interactions — Feng et al. (2023) (arXiv:2311.16160, 2023)
What this evaluates
Ranks active compounds against decoys for a given protein target. It probes the model's ability to prioritize true binders in a large pool of inactive decoys and resist dataset biases.
Datasets
- DUD-E — total ?; splits: 3-fold CV (-1)
- AD — total ?; splits: test (-1)
Metrics
AUC(primary) — range: [0, 1]- Area Under the ROC Curve (AUROC). Higher values indicate better ranking of actives over decoys.
ROC Enrichment (RE)— range: other- Number of active compounds in the top X% of the ranked list divided by (total actives * X%). Evaluated at 0.5%, 1.0%, 2.0%, 5.0%.
Input / output format
Input: CLS token embeddings of pocket and ligand concatenated, passed through an MLP.
Output: Binary classification score (active vs inactive).
Scoring recipe
auc = compute_auroc(scores, labels)
for pct in [0.5, 1.0, 2.0, 5.0]:
top_k = ceil(pct/100 * len(scores))
re[pct] = sum(labels[top_k]) / (total_actives * pct/100)
Common pitfalls
- DUD-E contains decoy bias (topological vs physical properties), which artificially inflates supervised methods' performance.
- Uses 3-fold cross-validation; similar targets are kept within the same fold to ensure fair comparison.
Evidence (verbatim from paper)
We employ a 3-fold cross-validation for training and evaluation, and our dataset split setting is consistent with AttentionDTI and DrugVQA, ensuring that similar targets are kept within the same fold to facilitate a fair comparison. Several widely used measures on DUD-E are employed in our evaluation, including AUROC and the ROC Enrichment metric (denoted as RE).
Citation
@misc{feng2023proteinligand,
title={Protein-ligand binding representation learning from fine-grained interactions},
author={Feng et al. (2023)},
year={2023},
note={arXiv:2311.16160}
}
- arXiv: 2311.16160