# Ppb Affinity Eval

> Evaluates protein language model architectures for predicting binding affinity in multi-chain protein-protein complexes. It probes how well different architectural designs capture inter-chain interactions compared to simple sequence or embedding concatenation. Use when the user wants to benchmark on PPB-Affinity, or asks about evaluating this task. Reports Spearman ρ.

- Skill: `qhjqhj00/ppb-affinity-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/ppb-affinity-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/ppb-affinity-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/ppb-affinity-eval

---


# ppb-affinity-eval

> Beyond Simple Concatenation: Fairly Assessing PLM Architectures for Multi-Chain Protein-Protein Interactions Prediction — Alsamkary et al. (2025) (arXiv:2505.20036, 2025)

## What this evaluates

Evaluates protein language model architectures for predicting binding affinity in multi-chain protein-protein complexes. It probes how well different architectural designs capture inter-chain interactions compared to simple sequence or embedding concatenation.

## Datasets

- **PPB-Affinity** — total ?; splits: val (-1), test (-1); repo https://github.com/Proteinea/ppiseq

## Metrics

- `Spearman ρ` **(primary)** — range: [-1, 1]
  - Spearman rank correlation coefficient between predicted and true binding affinities.
- `Pearson r` — range: [-1, 1]
  - Pearson linear correlation coefficient between predicted and true binding affinities.
- `RMSE (pKd)` — range: [0, ∞)
  - Root mean squared error between predicted and true pKd values, calculated as sqrt(mean((y_pred - y_true)^2)).

## Input / output format

**Input**: Paired protein sequences representing ligand and receptor chains for multi-chain complexes.

**Output**: A single continuous scalar value representing the predicted binding affinity (pKd).

## Scoring recipe

```python
import numpy as np
from scipy.stats import spearmanr, pearsonr

def compute_metrics(y_true, y_pred):
    spearman_rho, _ = spearmanr(y_true, y_pred)
    pearson_r, _ = pearsonr(y_true, y_pred)
    rmse = np.sqrt(np.mean((y_true - y_pred) ** 2))
    return spearman_rho, pearson_r, rmse
```

## Common pitfalls

- Data leakage can occur if sequence identity between splits exceeds 30%; the protocol enforces a strict ≤30% identity split.
- Simple concatenation (SC) of all chains into one sequence with a single EOS token obscures ligand-receptor boundaries, diluting inter-chain interaction signals.
- Full fine-tuning of certain PLMs (e.g., ESM3-SM-Open) can cause training instability with exploding hidden states, requiring careful learning rate and normalization adjustments.

## Evidence (verbatim from paper)

> Metrics (mean ± standard deviation, 3 seeds): Spearman  $
ho$ , Pearson r, and RMSE  $(pK_d)$ . PAD: Pooled attention addition; HP: Hierarchical pooling

## Citation

```bibtex
@misc{alsamkary2025beyond,
  title={Beyond Simple Concatenation: Fairly Assessing PLM Architectures for Multi-Chain Protein-Protein Interactions Prediction},
  author={Alsamkary et al. (2025)},
  year={2025},
  note={arXiv:2505.20036}
}
```

- arXiv: 2505.20036