qhjqhj00 synthetic-drug-data-eval
Domain Knowledge Infused Conditional Generative Models for Accelerating Drug Discovery — Bing Hu et al. (arXiv:2510.09837, 2025)
What this evaluates
Evaluates the distributional fidelity of generated pharmacokinetic and drug-target interaction properties against real data, and measures the utility of the synthetic data for downstream regression tasks.
Datasets
- TDCommons/BindingDB PK & DTI Collection — total ?; splits: train (-1), test (-1); repo https://github.com/GenerativeDrugDiscovery/xImagand-DKI
Metrics
Hellinger Distance (HD)(primary) — range: [0, 1]- Quantifies similarity between two discrete probability distributions P and Q. Formula: HD^2(p,q) = 1/2 * sum((sqrt(p_i) - sqrt(q_i))^2). Lower values indicate closer distribution match.
Differential Pairwise Correlations (DPC)— range: [0, 2]- Measures bivariate correlation similarity between real and synthetic data. Formula: |rho_XY_r - rho_XY_s|, where rho is the Pearson correlation coefficient. Values near 0 indicate high similarity.
Mean Squared Error (mse)— range: [0, inf)- Standard regression loss measuring average squared difference between predicted and actual values.
R-Squared (R2)— range: (-inf, 1]- Proportion of variance in the dependent variable predictable from the independent variables.
Pearson Correlation Coefficient (pcc)— range: [-1, 1]- Linear correlation between predicted and actual values.
Input / output format
Input: SMILES strings and protein embeddings (T5 and ProtBERT) conditioned on domain knowledge (GO-derived protein embeddings and molecular fingerprints).
Output: Continuous pharmacokinetic (PK) and drug-target interaction (DTI) property values (regression targets).
Scoring recipe
def compute_hd(real_dist, synth_dist):
return 0.5 * sum((sqrt(p) - sqrt(q))**2 for p, q in zip(real_dist, synth_dist))
def compute_dpc(real_data, synth_data, x, y):
rho_r = pearsonr(real_data[x], real_data[y])
rho_s = pearsonr(synth_data[x], synth_data[y])
return abs(rho_r - rho_s)
def compute_mle(real_train, real_test, synth_train, synth_test):
model_r = LinearRegression().fit(real_train, real_train_y)
model_s = LinearRegression().fit(synth_train, synth_train_y)
pred_r = model_r.predict(real_test)
pred_s = model_s.predict(synth_test)
return mse(real_test_y, pred_r), r2(real_test_y, pred_r), pearsonr(real_test_y, pred_r)
Common pitfalls
- HD requires discretization/binning of continuous PK/DTI properties before distribution comparison.
- DPC omits pairwise combinations with fewer than 10 examples, potentially biasing correlation estimates.
- MLE evaluation uses a specific 50/50 real and 90/10 synthetic split with 1.5 IQR outlier removal, which may not generalize to other splits.
Evidence (verbatim from paper)
Hellinger distance (HD) quantifies the similarity between two probability distributions and can be used as a summary statistic of differences for each PK target property between real and synthetic datasets. Given two discrete probability distributions $P={p_{1},p_{2},...,p_{n}}$ and $Q={q_{1},q_{2},...,q_{n}}$, the HD between $P$ and $Q$ is expressed in Equation [5].
Citation
@misc{hu2025domainknowledge,
title={Domain Knowledge Infused Conditional Generative Models for Accelerating Drug Discovery},
author={Bing Hu et al.},
year={2025},
note={arXiv:2510.09837}
}
- arXiv: 2510.09837