Dti Binding Affinity Eval

dti-binding-affinity-eval

Learning to Align Molecules and Proteins: A Geometry-Aware Approach to Binding Affinity — Refahi et al. (2025) (arXiv:2509.20693, 2025)

What this evaluates

Evaluates a model's ability to predict continuous drug-target binding affinity and classify binary drug-target interactions. It probes geometry-aware representation learning, metric consistency, and generalization across diverse chemical-proteomic domains.

Datasets

• DTI-DG — total ?; splits: test (-1)
• BIOSNAP — total ?; splits: test (-1)
• BindingDB — total ?; splits: test (-1)
• DAVIS — total ?; splits: test (-1)

Metrics

• PCC (primary) — range: [-1, 1]
  • Pearson correlation coefficient between predicted and true binding affinity values. Higher values indicate better regression alignment.
• AUPR — range: [0, 1]
  • Area under the Precision-Recall Curve for binary drug-target interaction classification. Measures performance on imbalanced interaction data.
• AUROC — range: [0, 1]
  • Area under the Receiver Operating Characteristic Curve for binary drug-target interaction classification. Measures ranking quality of interaction probabilities.

Input / output format

Input: Paired drug (e.g., SMILES/graph) and protein (e.g., sequence/embedding) representations.

Output: Continuous binding affinity score (regression) or binary interaction logit/probability (classification).

Scoring recipe

def compute_metrics(preds, golds, task='regression'):
    if task == 'regression':
        return pearsonr(preds, golds).statistic  # PCC
    else:
        preds_prob = sigmoid(preds)
        auprc = average_precision_score(golds, preds_prob)
        auroc = roc_auc_score(golds, preds_prob)
        return auprc, auroc
# Report mean ± s.e.m. over 5 random seeds.

Common pitfalls

• Metrics are reported as mean ± standard error of the mean over 5 random seeds, not single-run values.
• DTI-DG is an out-of-domain generalization benchmark; standard in-distribution train/test splits do not apply.
• Task-specific metrics must be matched correctly: PCC for affinity regression, AUPR/AUROC for binary interaction classification.

Evidence (verbatim from paper)

Table[1] reports the mean PCC over the test domains when removing the FiLM conditioning layer or the triplet loss. ... Table[2] compares FIRM-DTI with strong baselines on three standard DTI datasets. Across BIOSNAP and BindingDB our model consistently achieves the highest or comparable AUPR and AUROC scores, showing that the FiLM-conditioned, geometry-aware representation generalizes well beyond affinity regression.

Citation

@misc{refahi2025learning,
  title={Learning to Align Molecules and Proteins: A Geometry-Aware Approach to Binding Affinity},
  author={Refahi et al. (2025)},
  year={2025},
  note={arXiv:2509.20693}
}

• arXiv: 2509.20693