Eeg Ssl Emotion Eval

eeg-ssl-emotion-eval

PARSE: Pairwise Alignment of Representations in Semi-Supervised EEG Learning for Emotion Recognition — Zhang et al. (2022) (arXiv:2202.05400, 2022)

What this evaluates

Evaluates semi-supervised EEG-based emotion recognition under extreme label scarcity. It probes the model's ability to leverage unlabeled data via representation alignment while maintaining classification performance across subject-dependent and subject-independent protocols.

Datasets

• SEED — total ?; splits: train (9), test (6)
• SEED-IV — total ?; splits: train (16), test (8)
• SEED-V — total ?; splits: train (5), val (5), test (5)
• AMIGOS — total ?; splits: train (-1), test (-1)

Metrics

• accuracy (primary) — range: [0, 1]
  • Ratio of correctly predicted instances to the total number of instances. Computed per subject-dependent fold.
• F1-score — range: [0, 1]
  • Macro-averaged F1-score across all classes. Harmonic mean of precision and recall per class, then averaged. Used for the imbalanced AMIGOS dataset.

Input / output format

Input: 1-D feature vectors extracted from EEG segments. For SEED-series: Differential Entropy across 5 bands (delta, theta, alpha, beta, gamma) from 62 channels, reshaped and min-max normalized to [0,1]. For AMIGOS: Log-PSD across 5 bands + asymmetry from 14 channels, reshaped and normalized to [0,1].

Output: Discrete class predictions (3 classes for SEED/IV, 5 classes for SEED-V, 2 classes for AMIGOS valence/arousal).

Scoring recipe

def compute_metric(predictions, gold, metric_type):
    if metric_type == 'accuracy':
        return sum(p == g for p, g in zip(predictions, gold)) / len(gold)
    elif metric_type == 'f1':
        classes = sorted(set(gold))
        f1s = []
        for c in classes:
            tp = sum(p == c and g == c for p, g in zip(predictions, gold))
            fp = sum(p == c and g != c for p, g in zip(predictions, gold))
            fn = sum(p != c and g == c for p, g in zip(predictions, gold))
            prec = tp / (tp + fp) if (tp + fp) > 0 else 0
            rec = tp / (tp + fn) if (tp + fn) > 0 else 0
            f1 = 2 * prec * rec / (prec + rec) if (prec + rec) > 0 else 0
            f1s.append(f1)
        return sum(f1s) / len(f1s)

Common pitfalls

• Applying accuracy instead of macro F1-score for the highly imbalanced AMIGOS dataset, which violates the paper's explicit protocol.
• Mixing subject-dependent and subject-independent evaluation splits (SEED-series use fixed trial splits per participant, AMIGOS uses leave-one-participant-out).
• Failing to vary the number of labeled samples per class (m ∈ {1,3,5,7,10,25}) when reporting semi-supervised results, as the protocol requires evaluation across multiple label scarcity levels.

Evidence (verbatim from paper)

Since the class distributions are almost balanced in the SEED-series datasets, accuracy is selected as the evaluation metric. In AMIGOS, we adopt the same leave-one-participant-out protocol for training and testing data splits... As the class distribution is very imbalanced, we use the F1-score (mean F1-score for both classes) as the evaluation metric as suggested in [13].

Citation

@misc{zhang2022parse,
  title={PARSE: Pairwise Alignment of Representations in Semi-Supervised EEG Learning for Emotion Recognition},
  author={Zhang et al. (2022)},
  year={2022},
  note={arXiv:2202.05400}
}

• arXiv: 2202.05400