Softmol Molecular Generation Eval

softmol-molecular-generation-eval

From Tokens to Blocks: A Block-Diffusion Perspective on Molecular Generation — Qianwei Yang et al. (2026) (arXiv:2601.21964, 2026)

What this evaluates

Evaluates a diffusion-based molecular language model's ability to generate chemically valid, drug-like molecules and optimize them for specific protein targets. It probes distribution matching, structural diversity, and target-aware binding affinity prediction.

Datasets

• ZINC-Curated — total 427000000; splits: train (-1)
• SMILES — total 324000000; splits: train (-1)
• SAFE — total 322000000; splits: train (-1)

Metrics

• Novel Top-hit 5% Score (primary) — range: kcal/mol
  • Mean docking score (DS) of the top 5% unique and novel generated hits. Hits must satisfy DS < median active DS, QED > 0.5, and SA < 5.0.
• Quality — range: percent
  • Proportion of unique, valid molecules meeting strict drug-likeness criteria: QED ≥ 0.6 and SA ≤ 4.
• Docking-Filter — range: percent
  • Proportion of generated molecules passing a pre-screening proxy for docking viability: QED > 0.5 and SA < 5.
• Hit Ratio — range: percent
  • Proportion of unique generated molecules that qualify as novel hits (DS < median active, QED > 0.5, SA < 5.0).
• #Circles — range: other
  • Number of distinct structural clusters among novel hits, measuring diversity and coverage of the explored chemical space (threshold set to 0.75).

Input / output format

Input: Protein target identifier (e.g., parp1, fa7, 5ht1b, braf, jak2) and structural constraints for target-aware generation; or unconstrained prompts for de novo generation.

Output: Generated SMILES strings representing molecular structures.

Scoring recipe

def evaluate(generated_smiles, target=None):
    valid = [s for s in generated_smiles if is_valid(s)]
    unique = list(dict.fromkeys(valid))
    hits = [s for s in unique if qed(s) > 0.5 and sa(s) < 5.0]
    if target:
        ds_scores = [dock_score(s, target) for s in hits]
        top5 = sorted(ds_scores)[:max(1, len(ds_scores)//20)]
        return sum(top5)/len(top5) if top5 else float('inf')
    return {
        'uniqueness': len(unique)/len(generated_smiles),
        'hit_ratio': len(hits)/len(generated_smiles),
        'circles': count_clusters(hits, threshold=0.75)
    }

Common pitfalls

• Failing to filter generated molecules for novelty (excluding training set members) before computing docking scores.
• Using incorrect QED/SA thresholds (paper specifies QED > 0.5, SA < 5.0 for hits, but QED ≥ 0.6, SA ≤ 4 for the Quality metric).
• Not averaging results over three independent random seeds as mandated by the protocol.
• Confusing the 'Unconstrained' SoftMol variant results with the main constrained model.

Evidence (verbatim from paper)

The primary evaluation metric is the Novel Top-hit 5% Score, defined as the mean docking score (DS) of the top 5% unique and novel generated hits.

Citation

@misc{yang2026fromtokens,
  title={From Tokens to Blocks: A Block-Diffusion Perspective on Molecular Generation},
  author={Qianwei Yang et al. (2026)},
  year={2026},
  note={arXiv:2601.21964}
}

• arXiv: 2601.21964