Davis Complete Eval

davis-complete-eval

Towards Precision Protein-Ligand Affinity Prediction Benchmark: A Complete and Modification-Aware DAVIS Dataset — Ming-Hsiu Wu et al. (2025) (arXiv:2512.00708, 2025)

What this evaluates

Evaluates protein-ligand binding affinity prediction models on a modification-aware dataset, testing their ability to generalize across different train-test splits (new ligands, new proteins, modifications) and assessing robustness to wild-type overfitting and few-shot fine-tuning.

Datasets

• DAVIS-complete — total 4032; splits: train (-1), test (-1), wild-type subset (-1), modification subset (-1); repo https://github.com/ZhiGroup/DAVIS-complete

Metrics

• Rp (primary) — range: other
  • Pearson correlation coefficient between predicted and true $pK_d$ values.
• MSE — range: other
  • Mean Squared Error between predicted and true $pK_d$ values.
• C-index — range: other
  • Concordance index measuring the probability that predicted and true affinity rankings are consistent.

Input / output format

Input: Protein sequence/structure and ligand structure (SMILES or 3D coordinates depending on model type).

Output: Predicted binding affinity as a continuous $pK_d$ value.

Scoring recipe

def compute_metrics(pred, true):
    mse = np.mean((pred - true) ** 2)
    rp = np.corrcoef(pred, true)[0, 1]
    c_index = concordance_index(true, pred)
    return mse, rp, c_index

Common pitfalls

• Models may overfit to wild-type predictions, yielding high correlation on modified pairs by simply echoing seen wild-type values rather than learning modification effects.
• Performance heavily depends on whether proteins or ligands are seen during training; new-ligand splits are generally harder than new-protein splits.
• Capped vs uncapped affinity values in test subsets significantly impact results, with capped values masking poor generalization.

Evidence (verbatim from paper)

MSE, $R_p$, and C-index are computed between predicted and true $pK_d$ values.

Citation

@misc{wu2025daviscomplete,
  title={Towards Precision Protein-Ligand Affinity Prediction Benchmark: A Complete and Modification-Aware DAVIS Dataset},
  author={Ming-Hsiu Wu et al. (2025)},
  year={2025},
  note={arXiv:2512.00708}
}

• arXiv: 2512.00708