Gram Dti Eval

gram-dti-eval

GRAM-DTI: adaptive multimodal representation learning for drug target interaction prediction — Feng Jiang et al. (2025) (arXiv:2509.21971, 2025)

What this evaluates

Evaluates multimodal drug-target interaction prediction and zero-shot retrieval capabilities across multiple benchmark datasets and cold-start scenarios.

Datasets

• Activation — total 1913; splits: train (-1), test (-1)
• Yamanishi_08 — total 5127; splits: train (-1), test (-1)
• Hetionet — total 49942; splits: train (-1), test (-1)
• Inhibition — total 21055; splits: train (-1), test (-1)

Metrics

• AUPR (primary) — range: [0, 1]
  • Area Under the Precision-Recall Curve. Computed by integrating precision over recall thresholds across all predicted probabilities.
• AUROC — range: [0, 1]
  • Area Under the Receiver Operating Characteristic Curve. Computed by integrating the true positive rate over the false positive rate across all classification thresholds.
• Recall@K — range: [0, 1]
  • Proportion of relevant items retrieved within the top-K results. Calculated as the number of true positives in the top-K ranked candidates divided by the total number of true positives.

Input / output format

Input: Multimodal representations of drug-target pairs: SMILES strings, text descriptions, HTA annotations, and protein sequences. For retrieval, single drug or protein queries.

Output: Binary interaction probability for DTI/MoA tasks; ranked candidate lists for retrieval tasks.

Scoring recipe

# DTI/MoA Prediction
preds = model.predict(pair)
auprc = average_precision_score(y_true, preds)
auroc = roc_auc_score(y_true, preds)

# Zero-shot Retrieval
scores = cosine_similarity(query_emb, candidate_embs)
top_k_indices = argsort(scores, descending=True)[:K]
recall_at_k = sum(1 for idx in top_k_indices if y_true[idx]) / len(y_true)

Common pitfalls

• Negative samples are generated at a 1:10 ratio with positives, which differs from standard balanced or 1:1 sampling protocols.
• Cold-start splits enforce strict entity-level separation (drug or protein) to prevent data leakage, requiring careful fold construction.
• Cross-validation fold counts differ by task: 10-fold for DTI datasets and 5-fold for MoA datasets.

Evidence (verbatim from paper)

The performance is measured using standard metrics, including Recall@K (R@1, R@10, R@100), which indicate the proportion of relevant items retrieved within the top-K results.

Citation

@misc{jiang2025gramdti,
  title={GRAM-DTI: adaptive multimodal representation learning for drug target interaction prediction},
  author={Feng Jiang et al. (2025)},
  year={2025},
  note={arXiv:2509.21971}
}

• arXiv: 2509.21971