Clip Continual Learning Eval

clip-continual-learning-eval

Harnessing Textual Semantic Priors for Knowledge Transfer and Refinement in CLIP-Driven Continual Learning — Lingfeng He et al. (arXiv:2508.01579, 2025)

What this evaluates

This evaluation protocol probes a model's ability to perform continual learning (CIL) using vision-language models (CLIP) without catastrophic forgetting. It measures how well the model retains knowledge from previous tasks while adapting to new ones, specifically testing stability-plasticity trade-offs under varying task splits and replay settings.

Datasets

• ImageNetR — total 30000; splits: 10-split (-1), 20-split (-1)
• ImageNetA — total 7500; splits: 10-split (-1), 20-split (-1)
• CIFAR-100 — total 60000; splits: 10-split (-1), 20-split (-1)

Metrics

• Last (primary) — range: percent
  • The final classification accuracy over all classes after completing the last incremental task.
• Avg — range: percent
  • The arithmetic mean of the classification accuracies across all incremental tasks evaluated sequentially.

Input / output format

Input: RGB images (32×32 for CIFAR-100, standard resolution for ImageNet variants) paired with class labels during training. The model leverages CLIP's text encoder for semantic priors and uses a ViT-B/16 visual backbone.

Output: Class predictions (logits or probabilities) over the full set of classes (200 for ImageNetR/A, 100 for CIFAR-100).

Scoring recipe

def compute_metrics(predictions, labels, num_tasks):
    accuracies = []
    for pred, label in zip(predictions, labels):
        acc = (pred == label).mean() * 100
        accuracies.append(acc)
    last_acc = accuracies[-1]
    avg_acc = sum(accuracies) / num_tasks
    return last_acc, avg_acc

Common pitfalls

• Failing to average results over the three specified random seeds, which the paper explicitly requires for reporting mean performances.
• Confusing the replay-free SECA variant with the replay-based SECA++ variant, as they use different baselines and have distinct performance characteristics.
• Misinterpreting the 10-split vs 20-split settings, which change the number of incremental steps and disproportionately affect the Last and Avg metrics.

Evidence (verbatim from paper)

Following prior PEFT-based CIL works*(Tan et al. [2024]; Gao, Cen, and Chang [2024]; Huang et al. [2024])*, we adopt two standard evaluation metrics: (1) The last session accuracy (Last): the final accuracy over all classes after completing the last task, and (2) Average accuracy (Avg), the average accuracy across all incremental tasks. All experiments are conducted using three random seeds, and we report the mean performances.

Citation

@misc{he2025harnessing,
  title={Harnessing Textual Semantic Priors for Knowledge Transfer and Refinement in CLIP-Driven Continual Learning},
  author={Lingfeng He et al.},
  year={2025},
  note={arXiv:2508.01579}
}

• arXiv: 2508.01579