Ssvep Riemannian Eval

ssvep-riemannian-eval

Using Riemannian geometry for SSVEP-based Brain Computer Interface — Kalunga et al. (2015) (arXiv:1501.03227, 2015)

What this evaluates

Evaluates the classification performance of SSVEP-based Brain-Computer Interface algorithms using Riemannian geometry on EEG covariance matrices. It probes the robustness of different covariance estimators and online/offline classification pipelines under varying trial lengths, latency delays, and outlier conditions.

Datasets

• SSVEP BCI dataset (12 subjects) — total ?; splits: train (-1), test (32)

Metrics

• classification accuracy (primary) — range: percent
  • Percentage of correctly classified trials out of the total number of trials in the test set.
• Information Transfer Rate (ITR) — range: bits/min
  • Computed in bits/min based on classification accuracy and the number of classes (M=4), reflecting the communication speed of the BCI system.
• Integrated Discrimination Improvement (IDI) — range: other
  • Measures the improvement in discrimination ability of a covariance estimator relative to a baseline (SCM). Negative values indicate deterioration in classification performance.

Input / output format

Input: EEG signals recorded on 8 channels (Oz, O1, O2, POz, PO3, PO4, PO7, PO8) at 256 Hz. Subjects view 3 visual targets blinking at 13, 17, and 21 Hz, plus a resting class. Trials are 6 seconds long (1536 samples), rearranged into matrices X ∈ R^{24×N}.

Output: Predicted class label (13Hz, 17Hz, 21Hz, or resting) per trial or epoch, with optional confidence scores for the online asynchronous algorithm.

Scoring recipe

def compute_accuracy(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    return (correct / len(gold_labels)) * 100

def compute_itr(accuracy_pct, num_classes=4, trial_len_sec=6.0):
    P = accuracy_pct / 100.0
    if P <= 0 or P >= 1:
        return 0.0
    itr = (math.log(num_classes, 2) + P * math.log(P, 2) + (1 - P) * math.log((1 - P) / (num_classes - 1), 2)) / trial_len_sec
    return itr * 60.0  # bits/min

Common pitfalls

• Failing to account for the 2-second latency delay between cue onset and SSVEP synchronization, which can drop accuracy by nearly 10% if trials are analyzed from the start.
• Using unregularized sample covariance matrices (SCM) for short trial lengths (<2 seconds), which results in ill-conditioned or singular matrices and degrades classifier performance.
• Applying the 'Riemannian potato' outlier filter indiscriminately, which may discard legitimate trials that are close to the global mean but far from their specific class mean.

Evidence (verbatim from paper)

The evaluation is done in terms of classification accuracy, information transfer rate (ITR), and integrated discrimination improvement (IDI), obtained by each estimator (see Section 3.2) while using the offline MDRM classifier.

Citation

@misc{kalunga2015riemannian,
  title={Using Riemannian geometry for SSVEP-based Brain Computer Interface},
  author={Kalunga et al. (2015)},
  year={2015},
  note={arXiv:1501.03227}
}

• arXiv: 1501.03227