# Embedding Clustering Eval

> This evaluation probes how well different embedding models capture underlying number-theoretic structures in numeric sequences. It measures the quality of the latent representation space by comparing clustering performance against ground-truth mathematical group labels versus unsupervised KMeans assignments. Use when the user wants to benchmark on number-theoretic-sequences, or asks about evaluating this task. Reports Silhouette Coefficient.

- Skill: `qhjqhj00/embedding-clustering-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/embedding-clustering-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/embedding-clustering-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/embedding-clustering-eval

---


# embedding-clustering-eval

> Evaluating Embedding Generalization: How LLMs, LoRA, and SLERP Shape Representational Geometry — Kabane (2025) (arXiv:2511.21703, 2025)

## What this evaluates

This evaluation probes how well different embedding models capture underlying number-theoretic structures in numeric sequences. It measures the quality of the latent representation space by comparing clustering performance against ground-truth mathematical group labels versus unsupervised KMeans assignments.

## Datasets

- **number-theoretic-sequences** — total ?; splits: test (-1)

## Metrics

- `Silhouette Coefficient` **(primary)** — range: [-1, 1]
  - Quantifies how similar an object is to its own cluster compared to other clusters. Higher values indicate better-defined, more separated clusters.
- `Davies–Bouldin Index` — range: other
  - Computes the average similarity ratio of each cluster with its most similar one, based on scatter and separation measures. Lower values indicate better clustering quality.

## Input / output format

**Input**: Numeric sequences with associated number-theoretic group labels (for supervised evaluation) or unlabeled numeric sequences (for unsupervised KMeans evaluation).

**Output**: Embedding vectors for each sequence, followed by clustering assignments or direct metric computation against true labels.

## Scoring recipe

```python
embeddings = model.encode(sequences)
sil_sup = silhouette_score(embeddings, true_labels)
db_sup = davies_bouldin_score(embeddings, true_labels)
kmeans_labels = KMeans(n_clusters=k).fit_predict(embeddings)
sil_unsup = silhouette_score(embeddings, kmeans_labels)
db_unsup = davies_bouldin_score(embeddings, kmeans_labels)
return sil_sup, db_sup, sil_unsup, db_unsup
```

## Common pitfalls

- Negative Silhouette scores indicate poor global structure, but local regularities may still exist and be captured by KMeans.
- A higher KMeans Silhouette score than the true-label score suggests the embedding space clusters according to learned geometric priors rather than mathematical definitions.

## Evidence (verbatim from paper)

> Two standard metrics are employed: the Silhouette Coefficient (higher is better) and the Davies–Bouldin Index (lower is better).

## Citation

```bibtex
@misc{kabane2025evaluatingembedding,
  title={Evaluating Embedding Generalization: How LLMs, LoRA, and SLERP Shape Representational Geometry},
  author={Kabane (2025)},
  year={2025},
  note={arXiv:2511.21703}
}
```

- arXiv: 2511.21703