qhjqhj00 pmemo-emotion-recognition-eval
User independent Emotion Recognition with Residual Signal-Image Network — Yin et al. (2019) (arXiv:1908.03692, 2019)
What this evaluates
Binary classification of user-independent emotional states (valence and arousal) from electrodermal activity (EDA) signals. It probes the model's ability to generalize across subjects by using subject-specific thresholds and fusing physiological signals with external music benchmarks.
Datasets
- PMEmo — total 7962; splits: test (-1)
Metrics
accuracy(primary) — range: percent- Percentage of correctly classified instances out of the total number of instances. Computed as (TP+TN) / (TP+TN+FP+FN).
F1-score— range: percent- Harmonic mean of precision and recall: 2 * (precision * recall) / (precision + recall).
precision— range: percent- Ratio of true positive predictions to the total number of positive predictions: TP / (TP + FP).
recall— range: percent- Ratio of true positive predictions to the total number of actual positives: TP / (TP + FN).
Input / output format
Input: Continuous EDA signals (phasic, tonic, or mixed) transformed into image-like representations via cvxEDA decomposition, optionally fused with external music features.
Output: Binary classification label (0 or 1) representing low or high emotion for valence or arousal dimensions.
Scoring recipe
def compute_metrics(y_true, y_pred):
tp = np.sum((y_true == 1) & (y_pred == 1))
tn = np.sum((y_true == 0) & (y_pred == 0))
fp = np.sum((y_true == 0) & (y_pred == 1))
fn = np.sum((y_true == 1) & (y_pred == 0))
accuracy = (tp + tn) / (tp + tn + fp + fn)
precision = tp / (tp + fp) if (tp + fp) > 0 else 0.0
recall = tp / (tp + fn) if (tp + fn) > 0 else 0.0
f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0.0
return accuracy, f1, precision, recall
Common pitfalls
- The evaluation converts continuous V/A regression to binary classification using subject-specific k-means thresholds; applying a fixed global threshold will misalign with the protocol.
- Results are averaged over 10-fold cross-validation; reporting single-split or subject-dependent splits will not match the paper's reported metrics.
- Music features are explicitly noted as subjective and unreliable for user-independent tasks; fusing them without accounting for subject specificity may artificially inflate accuracy.
Evidence (verbatim from paper)
As we convert emotion recognition to a binary classification, average classification accuracy, F1-score, precision and recall are adopted as the classifying evaluation criteria. In the pre-experiment of correlation analysing, Root Mean Square Error (RMSE) and Pearson Correlation Coefficient (r) are adopted according to the baseline in [[13]]. We applied 10-fold cross-validation method. After tests of 10 flods were finished, the evaluation indexes were averaged at the end.
Citation
@misc{yin2019userindependent,
title={User independent Emotion Recognition with Residual Signal-Image Network},
author={Yin et al. (2019)},
year={2019},
note={arXiv:1908.03692}
}
- arXiv: 1908.03692