Elasticc3 Clustering Eval

elasticc3-clustering-eval

Elastic Coupled Co-clustering for Single-Cell Genomic Data — Zeng et al. (2020) (arXiv:2003.12970, 2020)

What this evaluates

Evaluates an unsupervised transfer learning method's ability to jointly cluster cells and genomic features across two datasets with differing distributions. It probes the model's capacity to elastically transfer clustering knowledge from an auxiliary dataset to a target dataset based on data similarity, without requiring labeled data or matching cluster counts.

Datasets

• Simulated scATAC-seq & scRNA-seq — total 200; splits: test (200)
• Real data 1 (Human scRNA & scATAC) — total 420; splits: test (420)
• Real data 2 (Human & Mouse scRNA) — total 656; splits: test (656)

Metrics

• NMI (primary) — range: [0, 1]
  • Normalized Mutual Information measures the mutual dependence between true and predicted cluster labels, normalized by their entropies. Values range from 0 (independent) to 1 (perfect match).
• ARI — range: [-1, 1]
  • Adjusted Rand Index corrects the Rand Index for chance, measuring the similarity between two data clusterings. Values range from -1 (no agreement) to 1 (perfect match).
• RI — range: [0, 1]
  • Rand Index measures the proportion of pairs of samples that are either in the same or different clusters in both true and predicted labelings.
• purity — range: [0, 1]
  • Purity calculates the fraction of the most frequent true class in each predicted cluster, averaged over all clusters. Higher values indicate better clustering.

Input / output format

Input: Binarized data matrices (non-zero entries set to 1) for auxiliary and target datasets. For real data, the top 100 most variable features are selected prior to binarization.

Output: Cluster assignments for cells in both auxiliary and target datasets, along with feature cluster assignments.

Scoring recipe

def evaluate_clustering(true_labels, pred_labels):
    nmi = normalized_mutual_information(true_labels, pred_labels)
    ari = adjusted_rand_index(true_labels, pred_labels)
    ri = rand_index(true_labels, pred_labels)
    cluster_to_true = defaultdict(Counter)
    for t, p in zip(true_labels, pred_labels):
        cluster_to_true[p][t] += 1
    purity = sum(max(counts.values()) for counts in cluster_to_true.values()) / len(true_labels)
    return {'NMI': nmi, 'ARI': ari, 'RI': ri, 'Purity': purity}

Common pitfalls

• STC assumes identical feature distributions between auxiliary and target data, causing performance degradation when domains differ (e.g., human vs. mouse or scRNA vs. scATAC).
• Co-clustering is mathematically equivalent to elasticC3 with α=0 and β=0, meaning it performs standard co-clustering without any cross-domain knowledge transfer.
• Hyperparameters α, β, and K require grid search tuning; using fixed defaults or naive initialization may yield suboptimal clustering results.

Evidence (verbatim from paper)

We use four criteria to evaluate the clustering results, including normalized mutual information (NMI), adjusted Rand index (ARI), Rand index (RI) and purity.

Citation

@misc{zeng2020elasticc3,
  title={Elastic Coupled Co-clustering for Single-Cell Genomic Data},
  author={Zeng et al. (2020)},
  year={2020},
  note={arXiv:2003.12970}
}

• arXiv: 2003.12970