Hit Molecule Eval

hit-molecule-eval

From In Silico to In Vitro: Evaluating Molecule Generative Models for Hit Generation — Osman et al. (2025) (arXiv:2512.22031, 2025)

What this evaluates

This benchmark evaluates graph-based generative models for their ability to produce biologically relevant, drug-like hit molecules rather than just chemically valid structures. It probes the models' capacity to satisfy strict medicinal chemistry constraints, maintain distributional similarity to known bioactive compounds, and achieve strong predicted binding affinity to specific protein targets.

Datasets

• REINVENT Dataset — total 1086248; splits: train (-1)
• Hit-like Dataset — total 58837; splits: train (-1)
• Target-Specific Ligand Sets — total ?; splits: train (-1)

Metrics

• VUN (Validity, Uniqueness, Novelty) (primary) — range: [0, 1]
  • Validity is the percentage of generated molecules obeying valence rules. Uniqueness is the proportion of valid molecules that are structurally distinct. Novelty is the fraction of valid and unique molecules absent from the training dataset.
• Fréchet ChemNet Distance (FCD) — range: other
  • A distance metric measuring distributional similarity between generated and reference molecules based on chemical and biological features extracted via a ChemNet model.
• Hit-like Filter Pass Rate — range: [0, 1]
  • The proportion of generated molecules that satisfy a multi-stage filtering pipeline including severity score ≤10, MW 150–350 Da, logP 1–3, SAS ≤5, ring constraints, and pChEMBL ≥5.
• Docking Score — range: other
  • Predicted binding affinity to seven protein targets computed via molecular docking simulations. Lower values indicate stronger predicted interactions.

Input / output format

Input: Empty molecular graph or noise vector for unconditional generation, or target-specific conditioning vector for conditional generation.

Output: Molecular graph representation (atoms, bonds, and associated features) representing a candidate compound.

Scoring recipe

def evaluate(generated_mols, train_mols):
    valid = [m for m in generated_mols if check_valence(m)]
    unique = list({mol_to_smiles(m) for m in valid})
    novel = [m for m in unique if mol_to_smiles(m) not in train_smiles]
    v, u, n = len(valid)/len(generated_mols), len(unique)/len(valid), len(novel)/len(unique)
    
    hit_pass = [m for m in unique if 
                150 <= mw(m) <= 350 and 1 <= logp(m) <= 3 and 
                sas(m) <= 5 and sev_score(m) <= 10 and 
                pchembl(m) >= 5 and ring_constraints(m)]
                
    dock_scores = [dock(m, target) for m in hit_pass for target in targets]
    return {"VUN": (v, u, n), "Hit_Pass_Rate": len(hit_pass)/len(unique), "Docking": dock_scores}

Common pitfalls

• Docking scores are only computed on molecules that pass both VUN and hit-like filters, not on the raw generated set, so reporting raw docking metrics without filtering context is misleading.
• The evaluation uses a strict multi-stage filtering pipeline rather than a single aggregate score, so reporting only chemical validity misses the biological relevance assessment.
• Target-specific datasets are constructed from ChEMBL with pChEMBL ≥ 5 and confidence ≥ 9, which may bias models toward known chemical space rather than true novelty.

Evidence (verbatim from paper)

Validity measures the percentage of generated molecules that obey valence rules and contain no structural errors. Uniqueness assesses the proportion of valid molecules that are structurally distinct within the generated set. Novelty quantifies the fraction of valid and unique molecules that are not present in the training dataset. These three metrics (VUN) serve as the minimal baseline for generative model performance and are standard across the field.

Citation

@misc{osman2025in,
  title={From In Silico to In Vitro: Evaluating Molecule Generative Models for Hit Generation},
  author={Osman et al. (2025)},
  year={2025},
  note={arXiv:2512.22031}
}

• arXiv: 2512.22031