Toximol Eval

toximol-eval

Breaking Bad Molecules: Are MLLMs Ready for Structure-Level Molecular Detoxification? — Lin et al. (2025) (arXiv:2506.10912, 2025)

What this evaluates

Evaluates whether Multimodal Large Language Models (MLLMs) can generate structurally valid, low-toxicity alternative molecules from toxic inputs while adhering to drug-likeness, synthetic feasibility, and structural similarity constraints. It probes the model's ability to perform structure-aware molecular editing and cross-modal scientific reasoning.

Datasets

• ToxiMol — total 560; splits: test (560); repo https://github.com/DeepYoke/ToxiMol

Metrics

• Toxicity Repair Success Rate (primary) — range: percent
  • Percentage of test samples where at least one of the generated candidate molecules passes the ToxiEval multi-dimensional evaluation chain. The chain requires the molecule to satisfy safety thresholds (S_safe), drug-likeness (QED), VEGA drug-likeness (V_ro5), synthetic accessibility (S_sas), and structural similarity (S_sim).

Input / output format

Input: A toxic molecule represented by its SMILES string and 2D molecular image, accompanied by a natural language repair objective specifying the toxicity target to mitigate.

Output: Exactly 3 candidate molecule structures (SMILES) per input sample.

Scoring recipe

success_count = 0
for sample in test_set:
    candidates = model.generate(sample, k=3)
    sample_success = False
    for mol in candidates:
        if passes_toxieval(mol):  # checks S_safe, Q, V_ro5, S_sas, S_sim
            sample_success = True
            break
    if sample_success:
        success_count += 1
return (success_count / len(test_set)) * 100

Common pitfalls

• High structural validity does not guarantee repair success; models often produce valid molecules that remain toxic or fail drug-likeness constraints.
• Increasing the number of generated candidates (k) yields diminishing returns, particularly for complex regression tasks like LD50 and SkinRxn.
• The QED (drug-likeness) metric acts as the strictest filter, frequently causing failures even when toxicity is successfully mitigated.

Evidence (verbatim from paper)

Table 1: Toxicity Repair Success Rates (%). ... A unified evaluation criterion is applied: a sample is successfully repaired if at least one candidate molecule passes the ToxiEval multi-dimensional evaluation chain. ... We report the per-task success rate and the overall success rate as the primary performance metrics.

Citation

@misc{lin2025toximol,
  title={Breaking Bad Molecules: Are MLLMs Ready for Structure-Level Molecular Detoxification?},
  author={Lin et al. (2025)},
  year={2025},
  note={arXiv:2506.10912}
}

• arXiv: 2506.10912