Molecular Property Prediction Eval

molecular-property-prediction-eval

BiScale-GTR: Fragment-Aware Graph Transformers for Multi-Scale Molecular Representation Learning — Yi Yang et al. (2026) (arXiv:2604.06336, 2026)

What this evaluates

Evaluates a model's ability to learn molecular representations for property prediction across classification and regression tasks. It probes the model's capacity to capture local atomic environments, multi-scale fragment structures, and long-range graph dependencies.

Datasets

• MoleculeNet — total ?; splits: train (-1), val (-1), test (-1)
• PharmaBench — total ?; splits: train (-1), test (-1)
• LRGB — total 15535; splits: train (-1), val (-1), test (-1)

Metrics

• ROC-AUC (primary) — range: [0, 1]
  • Area under the receiver operating characteristic curve. Used for classification tasks on MoleculeNet biological datasets to measure ranking performance between positive and negative classes.
• RMSE (primary) — range: [0, inf)
  • Root mean squared error. Used for regression tasks on PharmaBench ADMET properties and LRGB structural predictions to measure the average magnitude of prediction errors.

Input / output format

Input: Molecular graphs represented as fragment tokens and atom-level structural features.

Output: Predicted class labels (for classification) or continuous property values (for regression).

Scoring recipe

scores = []
for seed in range(10):
    best_val, best_model = -float('inf'), None
    model = train(dataset, seed, early_stop=True if dataset in [MoleculeNet, PharmaBench] else False, epochs=200 if dataset == LRGB else None)
    val_metric = evaluate(model, val_set)
    if val_metric > best_val: best_val, best_model = val_metric, model
    test_metric = evaluate(best_model, test_set)
    scores.append(test_metric)
return mean(scores), std(scores)

Common pitfalls

• Failing to use scaffold splits, which are explicitly required to prevent data leakage between training and test sets.
• Neglecting to apply positive class weighting for imbalanced classification tasks.
• Reporting single-run results instead of the required mean and standard deviation across 10 random seeds.

Evidence (verbatim from paper)

We evaluate our model on several widely used molecular property prediction benchmarks... We adopt scaffold splitting... For classification tasks with significant class imbalance, positive class weighting is applied during training... Each experiment is repeated with 10 different random seeds, and we report the mean and standard deviation of the evaluation metrics across runs.

Citation

@misc{yang2026biscalegtr,
  title={BiScale-GTR: Fragment-Aware Graph Transformers for Multi-Scale Molecular Representation Learning},
  author={Yi Yang et al. (2026)},
  year={2026},
  note={arXiv:2604.06336}
}

• arXiv: 2604.06336