Tdc Adme Pk Eval

tdc-adme-pk-eval

MMAI Gym for Science: Training Liquid Foundation Models for Drug Discovery — Kuznetsov et al. (2026) (arXiv:2603.03517, 2026)

What this evaluates

Assesses pharmacological property prediction across ADME, PK, and toxicity tasks, including regression, classification, and correlation-based evaluation.

Datasets

• TDC Benchmark — total ?; splits: test (-1)

Metrics

• MAE — range: other
  • Mean Absolute Error for regression tasks (e.g., Caco2 permeability, HIA).
• Spearman's Correlation — range: [-1, 1]
  • Spearman rank correlation coefficient for pharmacokinetics tasks.
• AUROC / AUPRC (primary) — range: [0, 1]
  • Area Under the Receiver Operating Characteristic / Precision-Recall Curve for classification tasks (e.g., BBB penetration, hERG inhibition).

Input / output format

Input: Molecular input (SMILES/structure) with task-specific instruction prompt.

Output: Chain-of-thought reasoning in tags, followed by an block containing a numeric value or class label.

Scoring recipe

preds = []
for rep in range(num_reps):
    out = model.generate(prompt(), mol)
    preds.append(parse_answer(out))
final = median(preds) if regression else mode(preds)
if task_type == 'regression':
    mae = mean(abs(p - g) for p, g in zip(final, gold))
elif task_type == 'pk':
    spearman = rank_correlation(final, gold)
else:
    auroc = roc_auc_score(gold, class_probs)
    auprc = pr_auc_score(gold, class_probs)

Common pitfalls

• TDC tasks have strict data splits and preprocessing pipelines; deviating from the standard protocol invalidates comparisons.
• Class probabilities must be extracted from first-token logprobs via softmax, not from the final discrete prediction.
• Spearman correlation is used for PK tasks instead of Pearson or RMSE, which is a common point of confusion.

Evidence (verbatim from paper)

Following the standard TDC protocol, we utilize task-specific metrics including Mean Absolute Error (MAE) for regression tasks, Spearman’s Correlation for pharmacokinetics, and AUROC or AUPRC for classification tasks

Citation

@misc{kuznetsov2026mmai,
  title={MMAI Gym for Science: Training Liquid Foundation Models for Drug Discovery},
  author={Kuznetsov et al. (2026)},
  year={2026},
  note={arXiv:2603.03517}
}

• arXiv: 2603.03517