Bibldr Drug Repositioning Eval

bibldr-drug-repositioning-eval

BiBLDR: Bidirectional Behavior Learning for Drug Repositioning — Zhang et al. (2025) (arXiv:2505.23861, 2025)

What this evaluates

Evaluates a model's ability to predict novel drug-disease associations by modeling them as a recommendation task using bidirectional behavioral sequences and prototype spaces. It probes cold-start generalization and robustness to highly sparse interaction data.

Datasets

• Gdataset — total 1933; splits: 10-fold CV (-1)
• Cdataset — total 2352; splits: 10-fold CV (-1)
• LRSSL — total 3051; splits: 10-fold CV (-1)

Metrics

• AUPRC (primary) — range: [0, 1]
  • Area under the precision-recall curve, computed by integrating precision over recall thresholds.
• AUROC — range: [0, 1]
  • Area under the receiver operating characteristic curve, computed by integrating the true positive rate over the false positive rate.

Input / output format

Input: Drug-disease pairs represented as bidirectional behavioral sequences, augmented with precomputed drug similarity matrices (2D chemical fingerprints) and disease similarity matrices (phenotypic features).

Output: Continuous prediction scores for each drug-disease pair, ranked to identify top candidate drugs.

Scoring recipe

def evaluate(y_true, y_pred):
    auroc = roc_auc_score(y_true, y_pred)
    auprc = average_precision_score(y_true, y_pred)
    return {'AUROC': auroc, 'AUPRC': auprc}

Common pitfalls

• Negative samples are randomly sampled to exactly match the number of positive samples, creating an artificially balanced test set that differs from real-world sparsity.
• Cold-start evaluation masks all known associations for a target drug, forcing reliance solely on disease-side sequences rather than partial cold-start conditions.
• Text and Table 1 report conflicting entity/association counts for Cdataset and LRSSL, requiring careful data reconciliation before replication.

Evidence (verbatim from paper)

We select Area Under the Precision-Recall Curve (AUPRC) and Area Under the Receiver Operating Characteristic Curve (AUROC) as evaluation metrics to assess model performance. These metrics are widely adopted in drug repositioning tasks due to their robustness and relevance. We first divide all positive samples in the drug-disease association matrix A into 10 equal folds. An equal number of negative samples are randomly selected from A and split into 10 corresponding folds. These folds are paired to create 10 cross-validation subsets. During evaluation, we rotate one fold as the test set and use the remaining nine for training, repeating this until all folds have served as the test set.

Citation

@misc{zhang2025bibldr,
  title={BiBLDR: Bidirectional Behavior Learning for Drug Repositioning},
  author={Zhang et al. (2025)},
  year={2025},
  note={arXiv:2505.23861}
}

• arXiv: 2505.23861