qhjqhj00 cifar-robustness-eval
Exploring Model Robustness with Adaptive Networks and Improved Adversarial Training — Xu et al. (2020) (arXiv:2006.00387, 2020)
What this evaluates
Evaluates the robustness of adversarially trained neural networks against Projected Gradient Descent (PGD) attacks on CIFAR-10 and CIFAR-100. It measures both clean (natural) classification accuracy and robust accuracy under varying attack strengths (PGD-20 and PGD-100).
Datasets
- CIFAR-10 — total ?; splits: test (-1); HF
cifar10 - CIFAR-100 — total ?; splits: test (-1); HF
cifar100
Metrics
natural accuracy— range: percent- Percentage of correctly classified clean validation images.
PGD-20 accuracy(primary) — range: percent- Percentage of correctly classified images after applying a 20-step Projected Gradient Descent attack with epsilon=8.
PGD-100 accuracy— range: percent- Percentage of correctly classified images after applying a 100-step Projected Gradient Descent attack with epsilon=8.
Input / output format
Input: RGB image from CIFAR-10 or CIFAR-100 dataset.
Output: Predicted class label (0-9 for CIFAR-10, 0-99 for CIFAR-100).
Scoring recipe
def compute_accuracy(predictions, gold_labels):
correct = sum(p == g for p, g in zip(predictions, gold_labels))
return (correct / len(gold_labels)) * 100
Common pitfalls
- Attack evaluation parameters (epsilon, step size, iterations) must be strictly controlled; the paper uses epsilon=8 for evaluation but varies training step sizes (epsilon_s=2 or 6).
- Comparisons must account for model capacity differences, as adaptive networks have slightly more parameters than non-adaptive baselines.
Evidence (verbatim from paper)
We report validation accuracy on natural images and adversarial images generated using PGD attacks with K=20 iterations and K=100 iterations.
Citation
@misc{xu2020exploring,
title={Exploring Model Robustness with Adaptive Networks and Improved Adversarial Training},
author={Xu et al. (2020)},
year={2020},
note={arXiv:2006.00387}
}
- arXiv: 2006.00387