Eulearn Genus Classification Eval

eulearn-genus-classification-eval

EuLearn: A 3D database for learning Euler characteristics — Fritz et al. (2025) (arXiv:2505.13539, 2025)

What this evaluates

This benchmark evaluates whether deep learning models can accurately classify 3D surfaces by their topological genus (Euler characteristic) using point clouds and mesh connectivity graphs. It specifically probes the models' ability to leverage structural and adjacency information rather than relying solely on raw geometric coordinates.

Datasets

• EuLearn — total 2750; splits: train (1925), test (825); repo https://github.com/appliedgeometry/EuLearn_db

Metrics

• F1 score (primary) — range: [0, 1]
  • Macro-averaged F1 score computed across all 11 genus classes (0 to 10). Calculated as the harmonic mean of precision and recall per class, then averaged. Macro and weighted averages are identical due to the perfectly balanced test set.

Input / output format

Input: A point cloud of exactly 3000 sampled points per surface, accompanied by an adjacency matrix (or graph structure) encoding the triangulation connectivity of the original mesh.

Output: A single integer class label representing the surface's genus (0 through 10).

Scoring recipe

def compute_macro_f1(y_true, y_pred, num_classes=11):
    f1s = []
    for c in range(num_classes):
        tp = sum(1 for t, p in zip(y_true, y_pred) if t == c and p == c)
        fp = sum(1 for t, p in zip(y_true, y_pred) if t != c and p == c)
        fn = sum(1 for t, p in zip(y_true, y_pred) if t == c and p != c)
        prec = tp / (tp + fp) if (tp + fp) > 0 else 0
        rec = tp / (tp + fn) if (tp + fn) > 0 else 0
        f1 = 2 * prec * rec / (prec + rec) if (prec + rec) > 0 else 0
        f1s.append(f1)
    return sum(f1s) / len(f1s)

Common pitfalls

• Vanilla point-cloud-only models (e.g., standard PointNet) perform poorly (~0.49 accuracy) not because of weak feature extractors, but because they lack adjacency/topological information required for genus classification.
• The dataset is strictly balanced (75 examples per class in the test set), so macro and weighted average metrics are identical; reporting only one is sufficient but unbalanced splits would change the evaluation dynamics.
• The graph sampling algorithm explicitly reconnects edges after node removal, increasing the average vertex degree from 6 to 37, which fundamentally alters the graph structure compared to standard mesh downsampling techniques.

Evidence (verbatim from paper)

We evaluated the models using precision, recall, F1 score, and accuracy metrics. The average over the classes is the precision, recall, and F1 scores reported. The macro and weighted averages are identical since the test dataset is balanced (75 examples per class).

Citation

@misc{fritz2025eulearn,
  title={EuLearn: A 3D database for learning Euler characteristics},
  author={Fritz et al. (2025)},
  year={2025},
  note={arXiv:2505.13539}
}

• arXiv: 2505.13539