Mumo Instruct Eval

mumo-instruct-eval

MMAI Gym for Science: Training Liquid Foundation Models for Drug Discovery — Kuznetsov et al. (2026) (arXiv:2603.03517, 2026)

What this evaluates

Evaluates a model's ability to perform multi-objective molecular lead optimization by modifying a starting molecule to simultaneously improve multiple conflicting pharmacological properties while retaining structural similarity.

Datasets

• MuMO-Instruct — total ?; splits: test (-1)

Metrics

• Success Rate (SR) (primary) — range: percent
  • Percentage of generated molecules that satisfy all property optimization constraints.
• Similarity (Sim) — range: [0, 1]
  • Tanimoto similarity coefficient between the input and generated optimized molecule.
• Relative Improvement (RI) — range: other
  • Quantifies the magnitude of property enhancement achieved by the generated molecule relative to the baseline.

Input / output format

Input: Molecular input (SMILES/structure) combined with a prompt assembled from randomly sampled paraphrased instruction templates.

Output: Chain-of-thought reasoning enclosed in tags, followed by an block containing the generated molecule SMILES.

Scoring recipe

predictions = []
for rep in range(num_reps):
    prompt = sample_paraphrase()
    mol = augment_input(mol)
    out = model.generate(prompt, mol, stochastic=True)
    predictions.append(parse_answer_block(out))
final_pred = mode(predictions)
sr = sum(1 for m in final_pred if satisfies_constraints(m)) / len(final_pred)
sim = tanimoto_similarity(input_mol, final_pred)
ri = property_enhancement(input_mol, final_pred)

Common pitfalls

• Models may output invalid SMILES strings that fail to parse or violate valence rules.
• Aggregation across stochastic decoding runs and prompt paraphrases is required; reporting a single greedy decode underestimates performance.
• Tanimoto similarity and property constraints must be computed on canonicalized molecular graphs.

Evidence (verbatim from paper)

Performance is measured using three metrics: Success Rate (SR), the percentage of generated molecules that satisfy all property optimization constraints; Similarity (Sim), the Tanimoto similarity between the input and generated optimized molecule; and Relative Improvement (RI), quantifying the magnitude of property enhancement.

Citation

@misc{kuznetsov2026mmai,
  title={MMAI Gym for Science: Training Liquid Foundation Models for Drug Discovery},
  author={Kuznetsov et al. (2026)},
  year={2026},
  note={arXiv:2603.03517}
}

• arXiv: 2603.03517