Dti Inductive Prediction Eval

dti-inductive-prediction-eval

Drug-Target Interaction Prediction via an Ensemble of Weighted Nearest Neighbors with Interaction Recovery — Bin Liu et al. (2020) (arXiv:2012.12325, 2020)

What this evaluates

Evaluates the ability of machine learning models to predict drug-target interactions in inductive settings where test drugs, targets, or both are unseen during training. It probes cold-start prediction capabilities and robustness to local class imbalance in sparse biological networks.

Datasets

• NR — total 90; splits: S2 (drug-wise 10-fold CV) (-1), S3 (target-wise 10-fold CV) (-1), S4 (block-wise 3-fold CV) (-1)
• GPCR — total 635; splits: S2 (drug-wise 10-fold CV) (-1), S3 (target-wise 10-fold CV) (-1), S4 (block-wise 3-fold CV) (-1)
• IC — total 1476; splits: S2 (drug-wise 10-fold CV) (-1), S3 (target-wise 10-fold CV) (-1), S4 (block-wise 3-fold CV) (-1)
• E — total 2926; splits: S2 (drug-wise 10-fold CV) (-1), S3 (target-wise 10-fold CV) (-1), S4 (block-wise 3-fold CV) (-1)
• DB — total 3681; splits: S2 (drug-wise 10-fold CV) (-1), S3 (target-wise 10-fold CV) (-1), S4 (block-wise 3-fold CV) (-1)

Metrics

• AUPR (primary) — range: [0, 1]
  • Area Under the Precision-Recall curve. Computed by integrating precision over recall thresholds across all predicted scores. Heavily penalizes highly ranked false positives.

Input / output format

Input: Drug and target feature vectors or similarity matrices, along with the known drug-target interaction matrix (Y) for training. Test instances consist of unseen drug-target pairs requiring prediction.

Output: Continuous interaction scores or probabilities for each queried drug-target pair.

Scoring recipe

def compute_aupr(y_true, y_pred):
    from sklearn.metrics import precision_recall_curve, auc
    precision, recall, _ = precision_recall_curve(y_true, y_pred)
    return auc(recall, precision)

Common pitfalls

• Failing to exclude test drug/target similarities from the training phase breaks the inductive assumption, leading to data leakage.
• Gold standard DTI datasets contain many missing interactions; evaluating on them without acknowledging missingness can underestimate true model performance.
• Local imbalance metrics (LI^d, LI^t) vary significantly across datasets and directly impact the difficulty of S2 vs S3 settings, requiring careful interpretation of cross-dataset results.

Evidence (verbatim from paper)

The Area Under the Precision-Recall curve (AUPR) which heavily punishes highly ranked false positive predictions [60] is used to evaluate the performance of inductive DTI prediction approaches in our experiments.

Citation

@misc{liu2020drugtargetinteraction,
  title={Drug-Target Interaction Prediction via an Ensemble of Weighted Nearest Neighbors with Interaction Recovery},
  author={Bin Liu et al. (2020)},
  year={2020},
  note={arXiv:2012.12325}
}

• arXiv: 2012.12325