Ssrs Retrosynthesis Eval

ssrs-retrosynthesis-eval

MMAI Gym for Science: Training Liquid Foundation Models for Drug Discovery — Kuznetsov et al. (2026) (arXiv:2603.03517, 2026)

What this evaluates

Evaluates single-step retrosynthesis capability by predicting reactant molecules from a given target product, testing both in-distribution chemical knowledge and out-of-distribution generalization.

Datasets

• USPTO-50K-test — total ?; splits: test (-1)
• URSA-expert-2026 — total ?; splits: test (-1)

Metrics

• Unique (primary) — range: percent
  • Fraction of unique valid reactant sets among generated samples.
• Max ChemCensor (CC) — range: other
  • Per-target maximum ChemCensor score averaged over targets, measuring synthetic precedent plausibility.
• Av. PT-Top-K CC — range: other
  • Per-target average ChemCensor score over the top-K unique predictions.

Input / output format

Input: Target molecule structure (SMILES) with a prompt requesting single-step retrosynthetic disconnection.

Output: Chain-of-thought reasoning in tags, followed by an block containing predicted reactant SMILES.

Scoring recipe

preds = []
for rep in range(num_reps):
    out = model.generate(prompt(), target_mol)
    preds.append(parse_answer(out))
unique_preds = list(set(preds))
unique_frac = len(unique_preds) / len(preds)
max_cc = max(chemcensor_score(m) for m in unique_preds)
avg_cc_topk = mean(chemcensor_score(m) for m in unique_preds[:K])

Common pitfalls

• ChemCensor scores require an external database of synthetic precedents; scores are not intrinsic to the model output.
• Top-K metrics depend on the value of K (e.g., @3, @5, @10) specified in the prompt.
• Reactant sets must be canonicalized and validated for chemical plausibility before scoring.

Evidence (verbatim from paper)

Columns report the following metrics. Unique: fraction of unique valid reactant sets among samples. Max: per-target maximum ChemCensor score averaged over targets. Av. PT-Top-K CC: per-target average ChemCensor score over top-K unique predictions.

Citation

@misc{kuznetsov2026mmai,
  title={MMAI Gym for Science: Training Liquid Foundation Models for Drug Discovery},
  author={Kuznetsov et al. (2026)},
  year={2026},
  note={arXiv:2603.03517}
}

• arXiv: 2603.03517