Peptide Protein Interaction Eval

peptide-protein-interaction-eval

An Integrated Deep-Learning Framework for Peptide-Protein Interaction Prediction and Target-Conditioned Peptide Generation with ConGA-PePPI and TC-PepGen — Tang et al. (2026) (arXiv:2604.18467, 2026)

What this evaluates

Evaluates a model's ability to predict whether a given peptide-protein pair interacts (binary classification) and to localize binding residues on both the peptide and protein sequences. It also assesses the model's capacity to generate target-specific peptide sequences that improve structural binding affinity over native templates.

Datasets

• Test167 — total ?; splits: test (-1)
• LEADS-PEP — total ?; splits: test (-1)
• Test251 — total ?; splits: test (-1)

Metrics

• AUROC (primary) — range: [0, 1]
  • Area under the Receiver Operating Characteristic curve, measuring the trade-off between true positive rate and false positive rate across classification thresholds.
• AUPR — range: [0, 1]
  • Area under the Precision-Recall curve, summarizing performance across thresholds, particularly sensitive to class imbalance.
• MCC — range: [-1, 1]
  • Matthews Correlation Coefficient, a balanced measure for binary classification that accounts for true and false positives and negatives.
• F1 — range: [0, 1]
  • Harmonic mean of precision and recall.
• ipTM hit rate — range: percent
  • Percentage of generated peptide sequences whose predicted interface TM-score (ipTM) from Chai-1, ESMFold, or AlphaFold 3 exceeds that of the native test peptide.

Input / output format

Input: Paired peptide and protein amino acid sequences.

Output: Binary interaction label (interacting vs. non-interacting) or interaction probability score; for generation, a peptide amino acid sequence.

Scoring recipe

def compute_metrics(y_true, y_pred_proba):
    y_pred = (y_pred_proba >= 0.5).astype(int)
    return {
        'AUROC': roc_auc_score(y_true, y_pred_proba),
        'AUPR': average_precision_score(y_true, y_pred_proba),
        'F1': f1_score(y_true, y_pred),
        'MCC': matthews_corrcoef(y_true, y_pred),
        'ACC': accuracy_score(y_true, y_pred),
        'Precision': precision_score(y_true, y_pred),
        'Recall': recall_score(y_true, y_pred)
    }

Common pitfalls

• The evaluation uses temporally separated and low-homology test sets (e.g., Test167 removes pairs with >80% sequence similarity to training data) to prevent data leakage, which must be strictly enforced.
• Binding-site prediction suffers from severe class imbalance due to sparse interface residues, making AUPR a more reliable metric than accuracy or AUROC for the protein side.
• Generation evaluation relies on predicted structural scores (ipTM) rather than experimental validation, so high ipTM hit rates indicate structural plausibility but not necessarily biological activity.

Evidence (verbatim from paper)

We used Test167, a temporally separated low-homology set constructed after removing pairs with sequence similarity above 80% to the training or validation data, and we further evaluated the model on LEADS-PEP and Test251. Across these three benchmarks, ConGA-PepPI remains competitive relative to representative sequence-based and structure-based baselines, and it achieves an AUROC of 0.936 on LEADS-PEP.

Citation

@misc{tang2026congapippi,
  title={An Integrated Deep-Learning Framework for Peptide-Protein Interaction Prediction and Target-Conditioned Peptide Generation with ConGA-PePPI and TC-PepGen},
  author={Tang et al. (2026)},
  year={2026},
  note={arXiv:2604.18467}
}

• arXiv: 2604.18467