qhjqhj00 grasp-pruning-eval
Picking Winning Tickets Before Training by Preserving Gradient Flow — Wang et al. (2020) (arXiv:2002.07376, 2020)
What this evaluates
Evaluates the test accuracy of single-shot pruning methods at initialization on image classification tasks. It measures how well a pruned sub-network can be trained and generalizes compared to baselines like SNIP and random pruning.
Datasets
- CIFAR-10 — total ?; splits: train (-1), test (-1)
- CIFAR-100 — total ?; splits: train (-1), test (-1)
- Tiny-ImageNet — total ?; splits: train (-1), test (-1)
- ImageNet — total ?; splits: train (-1), val (-1)
Metrics
test accuracy(primary) — range: percent- Percentage of correctly classified images on the held-out test set. For ImageNet, both top-1 and top-5 accuracy are reported.
Input / output format
Input: Image classification dataset (images and labels) with a specified neural network architecture (e.g., VGG, ResNet) and a target pruning ratio.
Output: A binary mask indicating which weights to keep/prune, followed by the trained pruned network's test accuracy on the evaluation split.
Scoring recipe
def compute_test_accuracy(predictions, gold_labels):
correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
return (correct / len(gold_labels)) * 100
Common pitfalls
- Pruning is performed at initialization, so gradients must be computed on the untrained model before any weight updates.
- Results are averaged over multiple trials (3 for CIFAR/Tiny-ImageNet, 10 for gradient norm analysis) to account for initialization variance.
- High sparsity levels (e.g., 90%+) often lead to underfitting, making convergence speed and gradient norm preservation critical factors.
Evidence (verbatim from paper)
To evaluate the effectiveness of GraSP on real world tasks, we test GraSP on four image classification datasets, CIFAR-10/100, Tiny-ImageNet and ImageNet, with two modern network architectures, VGGNet and ResNet... The test accuracy is reported in Table 1... We run each experiment for 3 trials for obtaining more stable results.
Citation
@misc{wang2020grasp,
title={Picking Winning Tickets Before Training by Preserving Gradient Flow},
author={Wang et al. (2020)},
year={2020},
note={arXiv:2002.07376}
}
- arXiv: 2002.07376