# Casp 2016 Dti Eval

> casp-2016-dti-eval

- Skill: `qhjqhj00/casp-2016-dti-eval` (Agent Skill)
- Install (CLI): `npx skillmds@latest add qhjqhj00/casp-2016-dti-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/casp-2016-dti-eval/raw
- Safety review: pending (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-21
- Page: https://skillmd.com/skills/qhjqhj00/casp-2016-dti-eval

---


# casp-2016-dti-eval

> PIGNet: A physics-informed deep learning model toward generalized drug-target interaction predictions — Moon et al. (2020) (arXiv:2008.12249, 2020)

## What this evaluates

Evaluates deep learning models for drug-target interaction prediction across four tasks: scoring (affinity correlation), ranking (pose affinity ordering), docking (native pose retrieval from decoys), and screening (true binder identification among random molecules). Probes both regression accuracy and virtual screening generalization.

## Datasets

- **CASF-2016** — total ?; splits: test (-1); repo https://github.com/jaechanglim/DTI_PDBbind.git
- **CSAR NRC-HiQ** — total ?; splits: test (-1)

## Metrics

- `Scoring power (Pearson's R)` — range: [0, 1]
  - Pearson correlation coefficient between predicted and experimental binding affinities.
- `Ranking power (Spearman's ρ)` — range: [0, 1]
  - Spearman's rank-correlation coefficient measuring the ability to correctly rank binding affinities of true binders.
- `Docking power (Top-1 Success Rate)` **(primary)** — range: [0, 1]
  - Percentage of complexes where the native binding pose is ranked within the top 1 candidates among computationally generated decoys.
- `Screening power (Top-1% Success Rate & EF)` **(primary)** — range: [0, 1]
  - Success rate: percentage of targets where the true binder is in the top 1% of candidates. Enhancement Factor (EF): ratio of true binders in the top 1% to the expected ratio if randomly selected.

## Input / output format

**Input**: 3D atomic coordinates and connectivity for protein-ligand complexes (native or computationally generated decoy poses), ligand molecular graph/SMILES, and protein sequence/structure.

**Output**: Predicted binding affinity score, ranked list of ligand poses, or binary/continuous screening score for each candidate molecule.

## Scoring recipe

```python
R = pearsonr(predicted_affinities, experimental_affinities)
rho = spearmanr(predicted_affinities, experimental_affinities)

docking_success = sum(1 for c in complexes if native_pose_rank[c] == 1) / len(complexes)

top_k = max(1, int(0.01 * len(candidates)))
screening_success = sum(1 for t in targets if true_binder_rank[t] <= top_k) / len(targets)
EF = screening_success / 0.01
```

## Common pitfalls

- High scoring power does not guarantee high docking or screening power, as models often overfit to true-binding complexes and fail on decoys.
- Scoring power alone is insufficient for evaluation; models must be tested on computationally generated decoys to assess generalization.
- Enhancement factor and success rate thresholds (e.g., top 1%) must be explicitly stated, as they vary across studies and significantly impact reported performance.

## Evidence (verbatim from paper)

> The scoring power measures a linear correlation of predicted binding affinities and experimental values, calculated by a Pearson's correlation coefficient R. The ranking power measures an ability of a model to correctly rank the binding affinities of true binders of the actual binding pose, calculated by a Spearman's rank-correlation coefficient ρ. These two metrics are designed to assess the model's ability upon the stable-and-precise binding structures. On the other hand, the docking power and the screening power deal with the unnatural structures which are generated computationally. The docking power measures an ability of a model to find out the native binding pose of a ligand among computer-generated decoys, quantified as a success rate within the top N candidates. The screening power measures the ability of a model to identify the specific binding ligand for a given target protein among a set of random molecules, quantified as a success rate and an enhancement factor (EF) within the top α percent of candidates.

## Citation

```bibtex
@misc{moon2020pignet,
  title={PIGNet: A physics-informed deep learning model toward generalized drug-target interaction predictions},
  author={Moon et al. (2020)},
  year={2020},
  note={arXiv:2008.12249}
}
```

- arXiv: 2008.12249