Molecule Net Adc Eval

molecule-net-adc-eval

Dumpling GNN: Hybrid GNN Enables Better ADC Payload Activity Prediction Based on Chemical Structure — Xu et al. (2024) (arXiv:2410.05278, 2024)

What this evaluates

Evaluates molecular property prediction and ADC payload activity classification using hybrid graph neural networks. Probes the model's ability to capture 2D topological and 3D structural features for binary classification across diverse chemical and biological tasks.

Datasets

• MoleculeNet — total ?; splits: train (-1), val (-1), test (-1)
• ADC Payload Dataset — total ?; splits: test (-1)

Metrics

• ROC-AUC (primary) — range: [0, 1]
  • Area under the Receiver Operating Characteristic curve. Computed as the integral of the true positive rate versus false positive rate across all classification thresholds. Reported as a percentage in the paper.
• Accuracy — range: [0, 1]
  • Ratio of correctly classified instances to total instances.
• Sensitivity — range: [0, 1]
  • True positive rate: TP / (TP + FN).
• Specificity — range: [0, 1]
  • True negative rate: TN / (TN + FP).
• MCC — range: [-1, 1]
  • Matthews Correlation Coefficient: (TPTN - FPFN) / sqrt((TP+FP)(TP+FN)(TN+FP)(TN+FN)).
• F1 Score — range: [0, 1]
  • Harmonic mean of precision and recall: 2 * (Precision * Recall) / (Precision + Recall).

Input / output format

Input: Molecular graphs where nodes represent atoms and edges represent bonds, derived from SMILES strings. Optionally augmented with 3D conformational coordinates generated via docking simulations.

Output: Binary classification label (active/inactive) or predicted probability score for thresholding.

Scoring recipe

def compute_roc_auc(y_true, y_scores):
    # y_true: ground truth binary labels (0/1)
    # y_scores: predicted probabilities for positive class
    from sklearn.metrics import roc_auc_score
    return roc_auc_score(y_true, y_scores)

# For multi-task datasets (e.g., ToxCast, PCBA), compute ROC-AUC per task and average across tasks.

Common pitfalls

• 3D structural information was only incorporated for the BBBP dataset due to data limitations, not applied uniformly across all benchmarks.
• Baseline model comparisons use values aggregated from Papers with Code leaderboards rather than re-running baselines under identical experimental splits.
• Data splitting strategy differs by dataset: 8:1:1 train/val/test for those without predefined splits, but 8:2 train/val when a predefined test set is provided.

Evidence (verbatim from paper)

The performance metric used is the area under the receiver operating characteristic curve (ROC-AUC), a standard measure in binary classification tasks that is particularly suitable for potentially imbalanced molecular datasets. ... For datasets without predefined splits, we employed an 8:1:1 ratio for training, validation, and test sets, respectively. When a predefined test set was available, we adhered to an 8:2 split for training and validation.

Citation

@misc{xu2024dumplinggnn,
  title={Dumpling GNN: Hybrid GNN Enables Better ADC Payload Activity Prediction Based on Chemical Structure},
  author={Xu et al. (2024)},
  year={2024},
  note={arXiv:2410.05278}
}

• arXiv: 2410.05278