Molecules Moses Eval

molecules-moses-eval

FragFM: Hierarchical Framework for Efficient Molecule Generation via Fragment-Level Discrete Flow Matching — Joongwon Lee et al. (2025) (arXiv:2502.15805, 2025)

What this evaluates

Evaluates the quality, validity, and chemical relevance of generated small drug-like molecules using standard molecular graph generative benchmarks. It probes a model's ability to produce chemically valid structures that match the distribution of real drug-like molecules while maintaining property fidelity and scaffold diversity.

Datasets

• MOSES — total ?; splits: train (-1), val (-1), test (-1)

Metrics

• FCD (primary) — range: other (lower is better)
  • Fréchet ChemNet Distance measures the distributional similarity between generated and reference molecules using features from a pretrained ChemNet model. Lower values indicate better distribution matching.
• Validity — range: [0, 1]
  • Fraction of generated molecular graphs that are chemically valid according to standard valence rules.
• Filters — range: [0, 1]
  • Fraction of generated molecules that pass a set of 147 medicinal chemistry filters (e.g., PAINS, structural alerts).

Input / output format

Input: Molecular graph structures or latent fragment representations for generation.

Output: Generated molecular graphs (25,000 molecules per run) evaluated for validity, uniqueness, novelty, and property metrics.

Scoring recipe

def compute_moses_metrics(generated_mols, ref_mols, n=25000):
    valid = sum(1 for m in generated_mols[:n] if is_valid(m)) / n
    unique = len(set(smiles(m) for m in generated_mols[:n])) / n
    novel = len(set(smiles(m) for m in generated_mols[:n]) - set(smiles(m) for m in ref_mols)) / n
    filters = sum(1 for m in generated_mols[:n] if passes_filters(m)) / n
    fcd = frechet_distance(chemnet_features(generated_mols[:n]), chemnet_features(ref_mols))
    snn = mean_max_sim(generated_mols[:n], ref_mols)
    scaf = scaffold_match_rate(generated_mols[:n], ref_mols)
    return {'valid': valid, 'unique': unique, 'novel': novel, 'filters': filters, 'fcd': fcd, 'snn': snn, 'scaf': scaf}

Common pitfalls

• Scaffold-split evaluation can artificially lower the Scaf score if test scaffolds are absent from the training fragment library, penalizing generalization unfairly.
• FCD and validity often show an inverse relationship for diffusion/flow models compared to autoregressive baselines, requiring careful trade-off analysis rather than relying on a single metric.

Evidence (verbatim from paper)

We evaluate FragFM on the MOSES benchmark, which focus on small drug-like molecule generation... Note that we report the result with the scaffold-splitted test set... FragFM achieves nearly 100% validity—on par with the best autoregressive models—and attains an FCD of 0.58, outperforming all one-shot models by a large margin.

Citation

@misc{lee2025fragfm,
  title={FragFM: Hierarchical Framework for Efficient Molecule Generation via Fragment-Level Discrete Flow Matching},
  author={Joongwon Lee et al. (2025)},
  year={2025},
  note={arXiv:2502.15805}
}

• arXiv: 2502.15805