Deap Eval

deap-eval

ScalingNet: extracting features from raw EEG data for emotion recognition — Hu et al. (2021) (arXiv:2105.13987, 2021)

What this evaluates

Evaluates the capability of neural architectures to perform binary emotion recognition (valence, arousal, dominance) directly from raw, multi-channel EEG time-series data without hand-crafted features. It measures how well a model generalizes across subjects using a standard cross-validation protocol.

Datasets

• DEAP — total 1280; splits: 5-fold cross-validation (-1)

Metrics

• Accuracy (primary) — range: [0, 1]
  • Standard classification accuracy: the proportion of correctly predicted binary labels out of the total number of samples in the evaluation fold.

Input / output format

Input: Raw EEG time-series signals (downsampled to 128Hz, filtered 4–45Hz) for 40 videos per subject across 32 subjects.

Output: Binary classification labels for three emotion dimensions: valence, arousal, and dominance (positive/negative based on a self-rating threshold of 5).

Scoring recipe

def compute_accuracy(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    return correct / len(gold_labels)

Common pitfalls

• Using physiological signals (e.g., EDA, ECG) instead of strictly EEG-only signals as specified in the protocol.
• Applying a different rating threshold than 5 for binary classification, which changes the class distribution.
• Not using the standard 5-fold cross-validation strategy, leading to non-comparable results with reported baselines.

Evidence (verbatim from paper)

The five-fold cross-validation strategy is employed to objectively evaluate the raw EEG data based emotion recognition performance of the proposed ScalingNet architecture. ... Where evaluation criteria are the emotion recognition accuracies of arousal, valance, dominance in closely following previous studies.

Citation

@misc{hu2021scalingnet,
  title={ScalingNet: extracting features from raw EEG data for emotion recognition},
  author={Hu et al. (2021)},
  year={2021},
  note={arXiv:2105.13987}
}

• arXiv: 2105.13987