Tabular Deep Embed Clustering Eval

tabular-deep-embed-clustering-eval

Effectiveness of Deep Image Embedding Clustering Methods on Tabular Data — Sakib Abrar, Ali Sekmen, Manar D. Samad (2022) (arXiv:2212.14111, 2022)

What this evaluates

Evaluates the effectiveness of deep image embedding clustering methods compared to traditional clustering algorithms on heterogeneous tabular datasets. It probes whether architectures designed for spatial image data can effectively learn representations for low-dimensional, non-spatial tabular data.

Datasets

• malware — total ?; splits: full (-1)
• mice — total ?; splits: full (-1)
• vehicle — total ?; splits: full (-1)
• olive — total ?; splits: full (-1)
• dermatology — total ?; splits: full (-1)
• breast cancer — total ?; splits: full (-1)
• Ecoli — total ?; splits: full (-1)

Metrics

• clustering accuracy (primary) — range: percent
  • Percentage of instances correctly assigned to their ground-truth cluster after optimal label permutation matching. The paper also reports average rank across datasets to summarize relative performance.

Input / output format

Input: Tabular feature vectors representing heterogeneous, low-dimensional, non-spatial data.

Output: Discrete cluster labels (integers) assigned to each instance.

Scoring recipe

def clustering_accuracy(predictions, gold):
    # predictions and gold are 1D arrays of cluster/true labels
    # Standard evaluation uses optimal label matching (e.g., Hungarian algorithm)
    # to account for arbitrary cluster label permutations.
    best_perm = find_best_permutation(predictions, gold)
    correct = sum(p == g for p, g in zip(best_perm, gold))
    return (correct / len(gold)) * 100

Common pitfalls

• Clustering accuracy is not invariant to label permutations; direct element-wise comparison without optimal label matching will yield artificially low scores.
• The paper evaluates only seven small-to-medium tabular datasets, which may not generalize to high-dimensional or large-scale tabular benchmarks.
• Deep embedding methods require unsupervised representation learning before clustering, but the paper does not detail hyperparameter tuning or convergence criteria, making exact reproduction difficult.

Evidence (verbatim from paper)

Table 3 compares the clustering accuracy of eight clustering and deep embedding clustering methods. The traditional clustering Gaussian mixture model (GMM) ranks the second best for the malware, mice, and vehicle data sets. The k-means clustering of tabular data ranks the second best for the olive data set.

Citation

@misc{abrar2022effectiveness,
  title={Effectiveness of Deep Image Embedding Clustering Methods on Tabular Data},
  author={Sakib Abrar, Ali Sekmen, Manar D. Samad (2022)},
  year={2022},
  note={arXiv:2212.14111}
}

• arXiv: 2212.14111