Confer Eval

confer-eval

Continual Facial Expression Recognition: A Benchmark — Churamani et al. (2023) (arXiv:2305.06448, 2023)

What this evaluates

Evaluates continual learning methods for facial expression recognition under incremental, non-i.i.d. data settings. It probes a model's ability to learn new expressions sequentially while preserving prior knowledge, measuring both forward adaptation and backward forgetting.

Datasets

• CK+ (Extended Cohn-Kanade) — total 1300; splits: test (-1)

Metrics

• Average Accuracy Score (primary) — range: [0, 1]
  • Mean of per-task classification accuracies evaluated sequentially after each new task is introduced. Higher is better.
• Catastrophic Forgetting — range: other
  • Difference between accuracy on previous tasks immediately after learning them and accuracy after learning subsequent tasks. Lower values indicate less forgetting; negative values indicate positive transfer.

Input / output format

Input: RGB facial expression images.

Output: Predicted facial expression class label.

Scoring recipe

def compute_avg_acc(preds, golds, task_ids):
    accs = []
    for t in sorted(set(task_ids)):
        mask = task_ids == t
        accs.append(np.mean(preds[mask] == golds[mask]))
    return np.mean(accs)

def compute_cf(preds, golds, task_ids, prev_accs):
    # CF at task t = avg_acc_on_prev_tasks_after_t - prev_accs[t-1]
    # Lower is better.
    pass

Common pitfalls

• Task-IL assumes the model knows which task it is currently processing, while Class-IL requires predicting from all previously seen classes without task IDs.
• CF scores can be negative, indicating positive transfer or regularization effects rather than forgetting.
• Task ordering significantly impacts rehearsal-based methods; results should be averaged over multiple class orderings.

Evidence (verbatim from paper)

Table 3 presents model performances, in terms of the Acc (Average Accuracy Score) under the Task-IL settings with CK+, both without and with data augmentation. Table 4 shows CF (Catastrophic Forgetting) scores across all the methods.

Citation

@misc{churamani2023confer,
  title={Continual Facial Expression Recognition: A Benchmark},
  author={Churamani et al. (2023)},
  year={2023},
  note={arXiv:2305.06448}
}

• arXiv: 2305.06448