Gnn Architecture Eval

gnn-architecture-eval

On the choice of graph neural network architectures — Vignac et al. (2019) (arXiv:1911.05384, 2019)

What this evaluates

This evaluation probes how graph neural network (GNN) performance depends on the ratio of observed training nodes to feature dimensionality (N_obs/D). It tests whether standard benchmarks are biased toward low-dimensional regimes and evaluates the effectiveness of decoupling feature extraction from graph propagation.

Datasets

• Cora, Citeseer, Pubmed — total ?; splits: train (-1), test (-1)

Metrics

• Accuracy (primary) — range: [0, 1]
  • Accuracy is the fraction of correctly classified nodes in the test set. Results are reported as the mean accuracy over 100 random train-test splits, along with the 95% confidence interval.

Input / output format

Input: Graph adjacency matrix and node feature matrix X (optionally transformed via random sketching to target dimension D').

Output: Predicted class label for each node in the test set.

Scoring recipe

correct = sum(1 for pred, gold in zip(predictions, gold_labels) if pred == gold)
accuracy = correct / len(gold_labels)
# Reported as mean(accuracies) ± 95% CI over 100 random 50% train splits

Common pitfalls

• Assuming standard benchmark configurations (low N_obs/D) generalize to high-data regimes where complex GNNs are needed.
• Ignoring that feature dimensions are artificially manipulated via random sketching, which changes the effective input distribution.
• Confusing the number of propagation steps (K=2) with the number of learnable layers in the model.

Evidence (verbatim from paper)

We repeat experiments with different train-test splits (selected uniformly at random) for each configuration of parameters to avoid overfitting to a particular training scenario. ... Table 2: Accuracy and 95% confidence interval over 100 splits with 300 features and 50% of the nodes in the training set.

Citation

@misc{vignac2019choice,
  title={On the choice of graph neural network architectures},
  author={Vignac et al. (2019)},
  year={2019},
  note={arXiv:1911.05384}
}

• arXiv: 1911.05384