Scope Dti Eval

scope-dti-eval

SCOPE-DTI: Semi-Inductive Dataset Construction and Framework Optimization for Practical Usability Enhancement in Deep Learning-Based Drug Target Interaction Prediction — Chen et al. (2025) (arXiv:2503.09251, 2025)

What this evaluates

This evaluation protocol assesses the ability of deep learning models to predict drug-target interactions (DTI) by learning from molecular graphs and protein sequences. It probes the model's capacity to capture cross-domain interaction patterns between small molecules and proteins, particularly in semi-inductive settings where novel compounds are paired with known protein families.

Datasets

• BindingDB — total ?; splits: train (-1), val (-1), test (-1)
• KIBA — total ?; splits: train (-1), val (-1), test (-1)
• Human — total ?; splits: train (-1), val (-1), test (-1)
• SCOPE — total ?; splits: train (-1), val (-1), test (-1)

Metrics

• AUROC (primary) — range: [0, 1]
  • Area under the Receiver Operating Characteristic curve, measuring the trade-off between true positive rate and false positive rate across all classification thresholds.

Input / output format

Input: Pairs of protein sequences (up to 2000 amino acids) and compound molecular structures (represented as graphs with atom/edge features or fingerprints).

Output: A continuous interaction score or probability indicating the likelihood of binding between the drug and target protein.

Scoring recipe

from sklearn.metrics import roc_auc_score
def compute_metric(y_true, y_pred):
    return roc_auc_score(y_true, y_pred)

Common pitfalls

• Failing to distinguish between the original and debiased versions of the datasets, which significantly impacts performance due to label bias.
• Tuning hyperparameters or selecting the best model checkpoint using the test set instead of strictly using the validation set for early stopping/model selection.
• Ignoring the specific molecular graph construction parameters (e.g., edge cutoff distances for proteins and compounds) which are critical for reproducing the input features.

Evidence (verbatim from paper)

The best performing model is selected at the epoch giving the best AUROC score on the validation set, which is then used to evaluate the final performance on the test set.

Citation

@misc{chen2025scopedti,
  title={SCOPE-DTI: Semi-Inductive Dataset Construction and Framework Optimization for Practical Usability Enhancement in Deep Learning-Based Drug Target Interaction Prediction},
  author={Chen et al. (2025)},
  year={2025},
  note={arXiv:2503.09251}
}

• arXiv: 2503.09251