Pfm Gene Reg Eval

pfm-gene-reg-eval

Learning biophysical models of gene regulation with probability flow matching — Maddu et al. (2026) (arXiv:2604.25062, 2026)

What this evaluates

Evaluates the ability of probability flow matching to infer stochastic gene regulatory dynamics and cell differentiation trajectories from time-resolved single-cell omics data. It probes interpolation accuracy, generalization to unseen initial conditions, and recovery of biologically validated gene-gene regulatory interactions.

Datasets

• 2D Ornstein-Uhlenbeck process — total ?; splits: test (-1)
• Multistable Waddington-like landscape — total ?; splits: test (-1)
• Ex vivo Hematopoiesis scRNA-seq — total ?; splits: test (-1)

Metrics

• leave-one-out energy distance (primary) — range: [0, inf)
  • Statistical distance measuring the discrepancy between the predicted and empirical marginal distributions at each time point. Computed by leaving out one time point, training on the rest, and evaluating the energy distance on the held-out distribution.
• RMSE — range: [0, inf)
  • Root Mean Squared Error between the inferred drift/force matrix and the ground-truth analytical matrix. Calculated as the square root of the mean squared differences across all matrix elements.
• cosine similarity — range: [-1, 1]
  • Cosine of the angle between the inferred signed gene-regulatory interaction vector and the experimentally validated network vector. Measures directional alignment of activation/inhibition patterns.
• AUPR — range: [0, 1]
  • Area Under the Precision-Recall Curve comparing inferred gene-regulatory networks against ChIP-seq validation datasets. Summarizes the trade-off between precision and recall across classification thresholds.
• Pearson correlation — range: [-1, 1]
  • Linear correlation coefficient between predicted and experimentally measured terminal gene-expression profiles after optimal transport alignment. Quantifies generalization to unseen initial conditions and knockout populations.

Input / output format

Input: Time-resolved single-cell transcriptomic snapshots (mRNA counts or PCA projections of 24 key transcription factors) at discrete time points, or synthetic multi-marginal distributions from known stochastic processes (e.g., OU, Waddington landscape).

Output: Inferred velocity field u_t(x), force field f(x), diffusion matrix D(x), and predicted terminal gene-expression distributions or regulatory interaction matrices.

Scoring recipe

def compute_metrics(predictions, gold):
    # RMSE for drift matrix
    rmse = np.sqrt(np.mean((predictions['drift'] - gold['drift'])**2))
    
    # Leave-one-out energy distance
    e_distances = []
    for t in time_points:
        train_data = remove_time(gold, t)
        model = fit_pfm(train_data)
        pred_dist = model.predict(t)
        e_distances.append(energy_distance(pred_dist, gold[t]))
    
    # Cosine similarity for regulatory interactions
    cos_sim = np.dot(predictions['reg_matrix'], gold['reg_matrix']) / \
              (np.linalg.norm(predictions['reg_matrix']) * np.linalg.norm(gold['reg_matrix']))
    
    # AUPR for GRN vs ChIP-seq
    aupr = average_precision_score(gold['chip_seq_labels'], predictions['grn_scores'])
    
    # Pearson correlation for generalization
    pearson_r = np.corrcoef(predictions['terminal_pred'], gold['terminal_actual'])[0, 1]
    
    return {'RMSE': rmse, 'LOO_Energy': np.mean(e_distances), 'Cosine_Sim': cos_sim, 'AUPR': aupr, 'Pearson_R': pearson_r}

Common pitfalls

• Identifiability issue: multiple combinations of force and diffusion can interpolate the data equally well, making it difficult to recover true biophysical parameters without strong constraints.
• Joint training of score and force fields can lead to identifiability problems where score estimation errors are absorbed by drift and diffusion terms, corrupting regulatory inferences.
• Using linear or cubic-spline interpolation for conditional paths introduces discontinuities in the velocity field, significantly degrading accuracy in high-dimensional, low-sample regimes compared to Chebyshev interpolation.

Evidence (verbatim from paper)

Interpolation accuracy, quantified by leave-one-out energy distance, for different dynamical models fitted to the marginal data using PFM. (Inset) In-silico perturbation of the inferred force field f(x) to estimate the regulatory response matrix Δ_ij. (e,f) Cosine similarity between the inferred signed gene-regulatory interactions, indicating activation or inhibition, and experimentally validated networks for erythroid and megakaryocytic lineages, respectively. (g) Area under the precision–recall curve (AUPR) comparing cell-type-specific (titles) inferred gene-regulatory networks against corresponding ChIP-seq datasets across different methods.

Citation

@misc{maddu2026pfm,
  title={Learning biophysical models of gene regulation with probability flow matching},
  author={Maddu et al. (2026)},
  year={2026},
  note={arXiv:2604.25062}
}

• arXiv: 2604.25062