Gnn4eeg Eval

gnn4eeg-eval

GNN4EEG: A Benchmark and Toolkit for Electroencephalography Classification with Graph Neural Network — Zhang et al. (2023) (arXiv:2309.15515, 2023)

What this evaluates

Evaluates Graph Neural Networks for classifying emotional states from 32-channel EEG signals. It probes spatial-temporal feature extraction, cross-subject generalization, and robustness to hyperparameter settings in neuroscience signal processing.

Datasets

• FACED — total ?; splits: intra-2 (-1), intra-9 (-1), cross-2 (-1), cross-9 (-1); repo https://github.com/Miracle-2001/GNN4EEG

Metrics

• Accuracy (primary) — range: percent
  • Standard classification accuracy: the proportion of correctly predicted emotional class labels out of the total number of predictions, reported as a percentage.

Input / output format

Input: 30-second multi-channel EEG signals (32 channels) recorded during emotion-inducing video clips, labeled with 2-class or 9-class emotional states.

Output: Predicted emotional class label for each EEG segment.

Scoring recipe

def compute_accuracy(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    return (correct / len(gold_labels)) * 100

Common pitfalls

• Reporting validation set accuracy (CV/FCV) instead of true test set accuracy (NCV), which overestimates generalization due to data leakage.
• Ignoring subject-level distribution shifts in cross-subject tasks, where EEG signals vary significantly between individuals, leading to poor real-world transferability.
• Using fixed epoch counts across all folds rather than early stopping or nested cross-validation, which biases hyperparameter tuning and performance reporting.

Evidence (verbatim from paper)

Table 1 shows the overall accuracy of each task. By extracting features from both spatial-spectral and spatial-temporal domains, HetEmotionNet outperforms other models in intra-subject tasks, especially in the intra-2 task. However, RGNN achieves the best accuracy in cross-subject tasks due to the application of the NodeDAT regularizer. SparseDGCNN has similar performance compared to DGCNN under this experimental setting. For validation protocols, CV performs slightly better than FCV because of the variety of the chosen epoch numbers on different folds. Besides, to address data leakage, NCV reports the performances on the test set, leading to a poorer result most of the time compared to CV, whose performances are reported on the validation set. Although NCV performs worse than FCV in certain tasks, the rationality in measuring generalization ability still makes it a practical protocol.

Citation

@misc{zhang2023gnn4eeg,
  title={GNN4EEG: A Benchmark and Toolkit for Electroencephalography Classification with Graph Neural Network},
  author={Zhang et al. (2023)},
  year={2023},
  note={arXiv:2309.15515}
}

• arXiv: 2309.15515