Dual Target Drug Design Eval

dual-target-drug-design-eval

Reprogramming Pretrained Target-Specific Diffusion Models for Dual-Target Drug Design — Zhou et al. (2024) (arXiv:2410.20688, 2024)

What this evaluates

Evaluates the ability of generative models to design dual-target ligands that simultaneously bind to two protein pockets with high affinity while maintaining favorable drug-like properties. It measures both binding strength and molecular quality across a large set of target pairs.

Datasets

• Dual-target drug design dataset — total 12917; splits: test (12917); repo https://github.com/zhouxiangxin1998/DualDiff

Metrics

• P-1 Vina Dock — range: kcal/mol
  • AutoDock Vina binding affinity score for target P1, estimated via re-docking. Lower (more negative) values indicate stronger binding.
• P-2 Vina Dock — range: kcal/mol
  • AutoDock Vina binding affinity score for target P2, estimated via re-docking. Lower values indicate stronger binding.
• Max Vina Dock — range: kcal/mol
  • The maximum Vina Dock score between the two targets for a given molecule. Lower values indicate better simultaneous binding capability.
• Dual High Affinity (primary) — range: percent
  • Proportion of generated molecules that exhibit binding affinity exceeding that of the reference molecules on both respective targets. Higher is better.
• QED — range: [0, 1]
  • Quantitative Estimate of Drug-likeness, a composite score of molecular properties. Higher is better.
• SA — range: other
  • Synthetic Accessibility score estimating ease of synthesis. Higher is better.
• Diversity — range: other
  • Molecular diversity of the generated set, typically measured via pairwise structural similarity or fingerprint distance. Higher is better.
• RMSD — range: Å
  • Root Mean Square Deviation between docked poses of the generated molecule towards the dual targets. Lower indicates better structural consistency.

Input / output format

Input: 3D coordinates of two protein binding pockets (P1 and P2), and optionally a reference molecule or molecular fragments for baseline methods.

Output: 3D coordinates and atom types for a single generated ligand molecule.

Scoring recipe

def evaluate(predictions, gold):
    results = []
    for mol in predictions:
        v1 = autodock_vina(mol, gold['P1'])
        v2 = autodock_vina(mol, gold['P2'])
        results.append({'v1': v1, 'v2': v2, 'max_v': max(v1, v2),
                        'ref_v1': gold['ref_v1'], 'ref_v2': gold['ref_v2']})
    dual_high = sum(1 for r in results if r['v1'] < r['ref_v1'] and r['v2'] < r['ref_v2']) / len(results)
    return {
        'P-1 Vina Dock': mean([r['v1'] for r in results]),
        'P-2 Vina Dock': mean([r['v2'] for r in results]),
        'Max Vina Dock': mean([r['max_v'] for r in results]),
        'Dual High Affinity': dual_high,
        'QED': mean([qed_score(r['mol']) for r in results]),
        'SA': mean([synth_access(r['mol']) for r in results]),
        'Diversity': compute_diversity([r['mol'] for r in results]),
        'RMSD': mean([rmsd_pose(r['mol'], gold['P1']) for r in results])
    }

Common pitfalls

• Vina Dock scores are negative; lower (more negative) values indicate better binding, which is counterintuitive to readers accustomed to positive accuracy metrics.
• Dual High Affinity requires simultaneous improvement over the reference on both targets, making it a strict success rate metric rather than an average affinity measure.
• Baseline methods (DiffLinker, LinkerNet) are repurposed and provided with reference molecules/fragments, while the proposed methods are zero-shot without references, creating an asymmetric comparison if not explicitly noted.

Evidence (verbatim from paper)

We employ AutoDock Vina [12] to estimate the target binding affinity... we report Dual High Affinity (abbreviated as Dual High Aff.) which represents the proportion of generated molecules that exhibit binding affinity that exceeds that of the reference molecules on both the respective targets.

Citation

@misc{zhou2024reprogramming,
  title={Reprogramming Pretrained Target-Specific Diffusion Models for Dual-Target Drug Design},
  author={Zhou et al. (2024)},
  year={2024},
  note={arXiv:2410.20688}
}

• arXiv: 2410.20688