Single Cell Clustering Eval

single-cell-clustering-eval

A Bayesian approach to model uncertainty in single-cell genomic data — Ren et al. (2025) (arXiv:2508.02061, 2025)

What this evaluates

Evaluates clustering algorithms on single-cell RNA sequencing data to assess their ability to separate distinct cell types and capture transitional states. It combines quantitative clustering metrics with visual validation using marker gene expression profiles.

Datasets

• Breast cancer dataset — total ?; splits: test (-1)
• Embryo neurone dataset — total ?; splits: test (-1)

Metrics

• misclustering rate (primary) — range: [0, 1]
  • A novel metric introduced to evaluate clustering performance without ground truth labels. It quantifies the proportion of cells assigned to clusters that are statistically indistinguishable or contain noise, validated via differential expression and SigClust tests.
• NMI — range: [0, 1]
  • Normalized Mutual Information, a standard clustering metric measuring the agreement between predicted clusters and reference labels, normalized to [0, 1].
• ARI — range: [0, 1]
  • Adjusted Rand Index, measuring the similarity between two clusterings while accounting for chance, ranging typically from -1 to 1 but reported here as positive values.
• AUC — range: [0, 1]
  • Area Under the ROC Curve used to measure the correspondence between clustering results and individual marker gene categories. Higher values indicate better discriminatory capacity for each marker.

Input / output format

Input: Single-cell gene expression profiles (scRNA-seq data) reduced via UMAP-LE projection.

Output: Cluster assignments for each cell, with optional posterior probabilities of cluster membership (for VB-GMM).

Scoring recipe

def evaluate_clustering(predictions, labels=None, marker_genes=None):
    m_rate = compute_misclustering_rate(predictions)  # Novel unsupervised metric
    nmi = normalized_mutual_info(labels, predictions)
    ari = adjusted_rand_index(labels, predictions)
    auc_scores = [roc_auc_score(marker, cluster_probs) for marker in marker_genes]
    return {"misclustering_rate": m_rate, "NMI": nmi, "ARI": ari, "AUC": auc_scores}

Common pitfalls

• Relying exclusively on quantitative metrics like NMI/ARI, which may appear low due to continuous biological states or imperfect ground truth, while ignoring visual marker gene validation.
• Assuming the misclustering rate follows standard accuracy conventions; it is specifically designed for unsupervised evaluation without ground truth labels.
• Ignoring the discrepancy between the K value optimal for quantitative metrics versus the K value optimal for visual marker separation.

Evidence (verbatim from paper)

Based on the results of the three metrics (misclustering rate, NMI and ARI), it indicates that the optimal numbers of clusters are $K = 7$ , $K = 11$ , and $K = 4$ for GMM and $K = 5$ , $K = 11$ for VB-GMM (based on misclustering rate and ARI).

Citation

@misc{ren2025bayesian,
  title={A Bayesian approach to model uncertainty in single-cell genomic data},
  author={Ren et al. (2025)},
  year={2025},
  note={arXiv:2508.02061}
}

• arXiv: 2508.02061