Gcn Node Classification Eval

gcn-node-classification-eval

Semi-Supervised Classification with Graph Convolutional Networks — Kipf et al. (2016) (arXiv:1609.02907, 2016)

What this evaluates

Semi-supervised node classification on citation and knowledge graphs. It probes the model's ability to learn graph-structured representations and classify nodes using only a small fraction of labeled examples.

Datasets

• Citeseer — total 3327; splits: train (120), val (500), test (1000)
• Cora — total 2708; splits: train (140), val (500), test (1000)
• Pubmed — total 19717; splits: train (60), val (500), test (1000)
• NELL — total 65755; splits: train (210), val (500), test (1000)

Metrics

• prediction accuracy (primary) — range: [0, 1]
  • Standard classification accuracy: the fraction of correctly predicted class labels among all nodes in the test set.

Input / output format

Input: Sparse bag-of-words feature vectors for each node, plus a binary symmetric adjacency matrix representing citation links or knowledge graph relations.

Output: Class label prediction for each node in the test set.

Scoring recipe

correct = sum(1 for pred, gold in zip(predictions, gold_labels) if pred == gold)
accuracy = correct / len(gold_labels)
return accuracy

Common pitfalls

• The training set uses a fixed number of labeled nodes per class (20 for citation networks, 1 for NELL), not a fixed percentage of the total nodes.
• The validation set is strictly for hyperparameter tuning and early stopping; its labels are never used for training.
• The adjacency matrix is constructed as binary and symmetric, ignoring edge direction and multiplicity.

Evidence (verbatim from paper)

We train a two-layer GCN as described in Section 3.1 and evaluate prediction accuracy on a test set of 1,000 labeled examples. We choose the same dataset splits as in Yang et al. (2016) with an additional validation set of 500 labeled examples for hyperparameter optimization (dropout rate for all layers, L2 regularization factor for the first GCN layer and number of hidden units). We do not use the validation set labels for training. For training, we only use 20 labels per class, but all feature vectors.

Citation

@misc{kipf2016semi,
  title={Semi-Supervised Classification with Graph Convolutional Networks},
  author={Kipf et al. (2016)},
  year={2016},
  note={arXiv:1609.02907}
}

• arXiv: 1609.02907