Qm9 Mol Structtok Eval

qm9-mol-structtok-eval

Tokenizing 3D Molecule Structure with Quantized Spherical Coordinates — Gao et al. (2024) (arXiv:2412.01564, 2024)

What this evaluates

Evaluates the ability of a tokenization framework to generate valid 3D molecular structures and predict quantum mechanical properties. It probes structural validity, geometric plausibility, conditional controllability, and property prediction accuracy on organic molecules.

Datasets

• QM9 — total 134000; splits: train (110000), val (10000), test (10831)

Metrics

• Mean Absolute Error (MAE) (primary) — range: other
  • MAE = (1/N) * Σ|y_true - y_pred|, where y_true is the target quantum property and y_pred is the predicted property of the generated molecule. Lower values indicate better controllability and prediction accuracy.
• Validity Rate — range: percent
  • Percentage of generated molecules that successfully pass bond assignment via xyz2mol or OpenBabel without charge imbalances, or convert to valid SMILES strings.
• Uniqueness Rate — range: percent
  • Proportion of generated molecules that have unique SMILES representations, calculated as 1 - (number of duplicate structures / total generated molecules).
• PoseBusters Geometric Checks — range: percent
  • Pass rate (%) across seven geometric and energetic sanity checks: All Atoms Connected, Reasonable Bond Angles, Reasonable Bond Lengths, Aromatic Ring Flatness, Double Bond Flatness, Reasonable Internal Energy, and No Internal Steric Clash.

Input / output format

Input: 3D atomic coordinates and atom types; SELFIES string representations for autoregressive generation; scalar property values (e.g., polarizability, HOMO/LUMO energies) for conditional generation.

Output: Discrete structural tokens (VQ-VAE latent codes), 3D molecular coordinates (xyz), or predicted scalar property values.

Scoring recipe

def compute_metrics(generated_mols, y_true, y_pred):
    # Validity
    valid_count = sum(1 for m in generated_mols if xyz2mol_or_openbabel(m) and not has_charge_imbalance(m))
    validity_rate = valid_count / len(generated_mols)
    # Uniqueness
    smiles_list = [mol_to_smiles(m) for m in generated_mols]
    uniqueness_rate = 1 - (len(smiles_list) - len(set(smiles_list))) / len(smiles_list)
    # PoseBusters
    posebusters_pass = all([
        check_atoms_connected(m), check_bond_angles(m), check_bond_lengths(m),
        check_aromatic_flatness(m), check_double_bond_flatness(m),
        check_internal_energy(m), check_steric_clash(m)
    ] for m in generated_mols)
    posebusters_rate = sum(posebusters_pass) / len(generated_mols)
    # MAE
    mae = mean(abs(y_true - y_pred))
    return validity_rate, uniqueness_rate, posebusters_rate, mae

Common pitfalls

• Validity is defined by bond assignment tools (xyz2mol/OpenBabel) rather than strict chemical rules, so results may vary depending on the tool version or lookup table used.
• Conditional generation MAE is evaluated on generated molecules using a fixed classifier, not on ground-truth molecules, which can inflate or deflate scores compared to standard property prediction benchmarks.
• PoseBusters was originally designed for protein-ligand complexes; applying its small-molecule-specific metrics may yield conservative or slightly mismatched geometric thresholds.

Evidence (verbatim from paper)

The primary evaluation metric was the Mean Absolute Error (MAE) between the given property ξ and the predicted property ξ̂ of the generated molecule. A lower MAE reflects the model's ability to accurately generate molecules with the desired quantum properties.

Citation

@misc{gao2024tokenizing,
  title={Tokenizing 3D Molecule Structure with Quantized Spherical Coordinates},
  author={Gao et al. (2024)},
  year={2024},
  note={arXiv:2412.01564}
}

• arXiv: 2412.01564