Guacamol Molecule Generation Eval

guacamol-molecule-generation-eval

A collaborative constrained graph diffusion model for the generation of realistic synthetic molecules — Ruiz-Botella et al. (2025) (arXiv:2505.16365, 2025)

What this evaluates

This evaluation probes a molecular generative model's ability to produce chemically valid, diverse, and structurally realistic molecules. It measures how well the generated molecules match the physicochemical property distributions of real compounds while maintaining high novelty and uniqueness rates.

Datasets

• GuacaMol benchmark suite — total ?; splits: train (-1), test (-1)

Metrics

• KL divergence (primary) — range: percent
  • Measures the divergence between the distribution of ten physicochemical properties of generated molecules and real molecules. The paper reports it as a percentage score where higher values indicate better distribution matching.
• Validity — range: percent
  • Percentage of generated molecules that satisfy basic chemical valence rules.
• Uniqueness — range: percent
  • Percentage of generated molecules that are unique (no duplicates in the generated set).
• Novelty — range: percent
  • Percentage of generated molecules that do not appear in the known chemical universe or training set.
• Jensen-Shannon distance — range: other
  • Symmetric measure of difference between probability distributions of individual molecular properties, used for detailed property-level analysis.

Input / output format

Input: Molecular graph representation (atoms, bonds, and optionally molecular fingerprints) along with a diffusion time step t.

Output: A chemically valid molecular graph preserving the input degree sequence (valence constraints), generated via iterative edge swap reversals.

Scoring recipe

def evaluate(predictions, gold):
    valid = sum(1 for m in predictions if is_valid(m)) / len(predictions)
    unique = len(set(predictions)) / len(predictions)
    novel = sum(1 for m in predictions if m not in gold_set) / len(predictions)
    kl_scores = []
    for prop in physicochemical_properties:
        dist_gen = get_distribution(predictions, prop)
        dist_real = get_distribution(gold, prop)
        kl = compute_kl_divergence(dist_gen, dist_real)
        kl_scores.append(1 - kl)
    kl_pct = np.mean(kl_scores) * 100
    return {'validity': valid*100, 'uniqueness': unique*100, 'novelty': novel*100, 'kl_divergence_pct': kl_pct}

Common pitfalls

• The reported 'KL divergence' is a transformed percentage score where higher is better, not the raw KL divergence (which ranges from 0 to ∞).
• 100% validity is enforced by construction via constrained diffusion, not learned by the model, so it does not reflect the model's chemical reasoning capability.
• The 36-property evaluation was conducted post-training to prevent benchmark overfitting and should not be conflated with the primary GuacaMol benchmark results.

Evidence (verbatim from paper)

The GuacaMol benchmark quantifies this by measuring the Kullback-Liebler (KL) divergence between the distributions of ten physicochemical properties of generated molecules and those of real molecules. As demonstrated by the KL divergence scores (Table 1), CoCoGraph generates molecules whose property distributions more closely match those of real molecules, achieving scores of 96.0% and 96.7% for the BASE and FPS versions respectively, compared to 92.6% for DiGress and 47.3% for JTVAE.

Citation

@misc{ruizbotella2025cocograph,
  title={A collaborative constrained graph diffusion model for the generation of realistic synthetic molecules},
  author={Ruiz-Botella et al. (2025)},
  year={2025},
  note={arXiv:2505.16365}
}

• arXiv: 2505.16365