Pointgat Molecule C10 Eval

pointgat-molecule-c10-eval

PointGAT: A quantum chemical property prediction model integrating graph attention and 3D geometry — Zhang et al. (2023) (arXiv:2310.05217, 2023)

What this evaluates

Evaluates a hybrid graph attention and 3D point cloud neural network's ability to predict quantum chemical properties and molecular physicochemical traits. It probes the model's capacity to integrate 2D topological graph features with 3D spatial geometry for accurate regression and classification of molecular energies and properties.

Datasets

• MoleculeNet — total ?; splits: test (-1)
• C10 — total 11841; splits: test (-1)

Metrics

• MAE (primary) — range: other
  • Mean Absolute Error: average of absolute differences between predicted and actual values. Calculated as (1/n) * Σ|y_true - y_pred|.
• RMSE — range: other
  • Root Mean Square Error: square root of the average of squared differences between predicted and actual values. Calculated as sqrt((1/n) * Σ(y_true - y_pred)²).
• R² (primary) — range: [0, 1]
  • Coefficient of Determination: proportion of variance in the dependent variable predictable from the independent variables. Calculated as 1 - (SS_res / SS_tot).
• ROAUC — range: [0, 1]
  • Receiver Operating Characteristic Area Under Curve: measures binary classification performance across all threshold settings by plotting TPR vs FPR.

Input / output format

Input: 2D molecular graph with atom-level features (charge, radius, ring information) and 3D point cloud coordinates (x, y, z) for each atom.

Output: Continuous scalar value representing the predicted quantum chemical property (e.g., relative energy in kcal/mol) or binary classification label.

Scoring recipe

import numpy as np
def compute_metrics(preds, gold):
    preds, gold = np.array(preds), np.array(gold)
    mae = np.mean(np.abs(preds - gold))
    rmse = np.sqrt(np.mean((preds - gold) ** 2))
    ss_res = np.sum((gold - preds) ** 2)
    ss_tot = np.sum((gold - np.mean(gold)) ** 2)
    r2 = 1 - (ss_res / ss_tot)
    return {'MAE': mae, 'RMSE': rmse, 'R2': r2}

Common pitfalls

• Geometry optimization level drastically affects performance; using MMFF94 yields MAE=1.616 kcal/mol while B3LYP yields MAE=1.228 kcal/mol on C10.
• The graph attention module is critical for accuracy; ablating it increases MAE to 6.727 kcal/mol, whereas ablating the 3D point cloud module only increases it to 1.802 kcal/mol.
• Dataset train/val/test splits are not explicitly detailed in the text, relying on standard MoleculeNet and C10 conventions rather than a defined protocol.

Evidence (verbatim from paper)

PointGAT achieved an MAE of 1.616 kcal/mol and an R² of 0.950 in the C10 test set (Table 3), with around 77% of the test samples exhibiting a prediction MAE within 2 kcal/mol (Fig. 4a).

Citation

@misc{zhang2023pointgat,
  title={PointGAT: A quantum chemical property prediction model integrating graph attention and 3D geometry},
  author={Zhang et al. (2023)},
  year={2023},
  note={arXiv:2310.05217}
}

• arXiv: 2310.05217