Mol Exp Eval

mol-exp-eval

Test-Time Training Scaling Laws for Chemical Exploration in Drug Design — Thomas et al. (2025) (arXiv:2501.19153, 2025)

What this evaluates

Evaluates a model's ability to explore chemical space and rediscover structurally diverse molecules with similar bioactivity against a specific target, rather than just optimizing for a single molecule.

Datasets

• MolExp — total ?; splits: test (-1)

Metrics

• MolExpL (primary) — range: [0, 1]
  • Product of the maximum Levenshtein-based string-edit similarity between each target molecule and the set of generated molecules: $\prod_{t \in T} \max_{m \in M} \text{sim}(m, t)$.
• SEDiv — range: [0, 1]
  • Sphere Exclusion Diversity measured at a sample size of 1,000, counting non-redundant molecules within a Tanimoto threshold of 0.65, normalized by sample size.

Input / output format

Input: The model generates molecular SMILES strings. During evaluation, generation is guided by an RL oracle that provides a reward signal based on the maximum similarity to a set of target molecules.

Output: A set of generated molecular SMILES strings (up to a specified budget, e.g., 10,000 molecules per agent).

Scoring recipe

def compute_mol_exp_score(generated_smiles, target_smiles):
    score = 1.0
    for t in target_smiles:
        max_sim = max(levenshtein_similarity(m, t) for m in generated_smiles)
        score *= max_sim
    return score

Common pitfalls

• The benchmark uses string-edit (Levenshtein) similarity for the primary MolExpL task, not fingerprint similarity, which can lead to different performance rankings.
• The metric is a product of similarities across all targets; failing to rediscover even a single target molecule drastically penalizes the score.
• Scaling the molecule budget for a single agent does not improve multi-target rediscovery; performance gains require scaling the number of independent RL agents.

Evidence (verbatim from paper)

Metric Performance is measured by calculating the product of the maximum similarity achieved to each target molecule $\prod_{t\in T}max(sim(m_i,m_t))$ . This metric focuses on the ability to generate similar molecules to all objectives. Molecular diversity is also measured by sphere exclusion diversity (SEDiv) at a sample size of 1,000 which provides a more accurate representation of chemical space coverage than internal diversity. $^{31,45}$ Note this is equivalent to the later published #Circles $^{45}$ metric at a Tanimoto threshold of 0.65 and normalized by the sample size.

Citation

@misc{thomas2025testtime,
  title={Test-Time Training Scaling Laws for Chemical Exploration in Drug Design},
  author={Thomas et al. (2025)},
  year={2025},
  note={arXiv:2501.19153}
}

• arXiv: 2501.19153