Molclr Molecular Property Eval

molclr-molecular-property-eval

Molecular Contrastive Learning of Representations via Graph Neural Networks — Wang et al. (2021) (arXiv:2102.10056, 2021)

What this evaluates

Evaluates the ability of graph neural networks to learn robust molecular representations via self-supervised contrastive learning, and their transferability to downstream molecular property prediction tasks (classification and regression).

Datasets

• BBBP — total 2039; splits: test (-1)
• Tox21 — total 7831; splits: test (-1)
• ClinTox — total 1478; splits: test (-1)
• HIV — total 41127; splits: test (-1)
• BACE — total 1513; splits: test (-1)
• SIDER — total 1478; splits: test (-1)
• MUV — total 93087; splits: test (-1)
• FreeSolv — total 642; splits: test (-1)
• ESOL — total 1128; splits: test (-1)
• Lipo — total 4200; splits: test (-1)
• QM7 — total 6830; splits: test (-1)
• QM8 — total 21786; splits: test (-1)

Metrics

• ROC-AUC (primary) — range: [0, 1]
  • Area under the Receiver Operating Characteristic curve. Measures the trade-off between true positive rate and false positive rate across classification thresholds.
• MAE — range: [0, ∞)
  • Mean Absolute Error. Computes the average of the absolute differences between predicted and true continuous values: MAE = (1/N) * Σ|y_true - y_pred|.

Input / output format

Input: Molecule graph constructed from SMILES string, where nodes represent atoms and edges represent chemical bonds.

Output: Predicted molecular property value (continuous for regression, binary/multi-class label for classification).

Scoring recipe

def compute_roc_auc(y_true, y_pred_proba):
    return roc_auc_score(y_true, y_pred_proba)

def compute_mae(y_true, y_pred):
    return mean_absolute_error(y_true, y_pred)

# For multi-task datasets, compute metric per task and average across tasks.

Common pitfalls

• Averaging ROC-AUC across tasks with different numbers of labels (e.g., Tox21 has 12 tasks, SIDER has 27) without specifying task-weighting vs dataset-weighting.
• Using non-standard train/validation/test splits, as these benchmarks have widely adopted fixed splits in the GNN literature.
• Confusing the pre-training contrastive loss objective with the downstream fine-tuning evaluation metric.

Evidence (verbatim from paper)

Table 1 demonstrates the test ROC-AUC performance of our MolCLR model on classification tasks in comparison to baseline models. The average and standard deviation of three individual runs are reported. Table 2 demonstrates the test MAE performance of our MolCLR model on regression benchmarks in comparison to baseline models.

Citation

@misc{wang2021molclr,
  title={Molecular Contrastive Learning of Representations via Graph Neural Networks},
  author={Wang et al. (2021)},
  year={2021},
  note={arXiv:2102.10056}
}

• arXiv: 2102.10056