Graph Classification Tfgw Eval

graph-classification-tfgw-eval

Template based Graph Neural Network with Optimal Transport Distances — Cédric Vincent-Cuaz et al. (arXiv:2205.15733, 2022)

What this evaluates

Evaluates the ability of graph neural networks and optimal transport-based methods to classify graphs by learning discriminative representations that capture both structural and feature dissimilarities. It probes expressiveness beyond the Weisfeiler-Lehman test and generalization on heterogeneous real-world graph structures.

Datasets

• 4-CYCLES — total ?; splits: test (-1)
• SKIP-CIRCLES — total ?; splits: test (-1)
• MUTAG — total ?; splits: test (-1)
• PTC — total ?; splits: test (-1)
• ENZYMES — total ?; splits: test (-1)
• PROTEIN — total ?; splits: test (-1)
• NCI1 — total ?; splits: test (-1)
• IMDB-B — total ?; splits: test (-1)
• IMDB-M — total ?; splits: test (-1)
• COLLAB — total ?; splits: test (-1)

Metrics

• accuracy (primary) — range: [0, 1]
  • Classification accuracy, defined as the proportion of correctly classified graphs out of the total number of graphs in the test set. Reported as mean and standard deviation over 10 runs or folds.

Input / output format

Input: Attributed graphs represented either as adjacency matrices or shortest-path matrices, optionally pre-processed by a GIN architecture (0, 1, or 2 layers). Social network graphs are augmented with node degree features.

Output: Discrete class label corresponding to the graph's category.

Scoring recipe

correct = 0
for graph, true_label in test_set:
    pred_label = model(graph)
    if pred_label == true_label:
        correct += 1
accuracy = correct / len(test_set)
# Model selection: retain parameters maximizing average validation accuracy across 10 folds

Common pitfalls

• Using standard 10-fold cross-validation without a holdout test set, which overestimates generalization compared to the paper's recommended holdout protocol.
• Failing to augment unattributed social network graphs with node degree features, causing methods requiring node features to fail or perform poorly.
• Comparing against baseline results reported in their original papers rather than re-evaluating them under the same 10-fold CV with holdout setting.

Evidence (verbatim from paper)

We suggest here to quantify the generalization capacities of GNN based models by performing a 10-fold cross validation with a holdout test set never seen during training. For each split, we track the accuracy on the validation fold every 5 epochs, then the model whose parameters maximize that accuracy is retained. Finally, the model used to predict on the holdout test set is the one with maximal validation accuracy averaged across all folds.

Citation

@misc{vincentcuaz2022template,
  title={Template based Graph Neural Network with Optimal Transport Distances},
  author={Cédric Vincent-Cuaz et al.},
  year={2022},
  note={arXiv:2205.15733}
}

• arXiv: 2205.15733