Snip Pruning Eval

snip-pruning-eval

SNIP: Single-shot Network Pruning based on Connection Sensitivity — Lee et al. (2018) (arXiv:1810.02340, 2018)

What this evaluates

Evaluates a single-shot pruning method's ability to identify and remove unimportant network connections at initialization, preserving classification accuracy across varying sparsity levels on standard vision and sequence datasets.

Datasets

• MNIST — total ?; splits: train (-1), val (-1)
• CIFAR-10 — total ?; splits: train (-1), val (-1)
• Tiny-ImageNet — total ?; splits: train (-1)

Metrics

• accuracy (primary) — range: percent
  • Percentage of correctly classified samples out of the total test set. Calculated as (number of correct predictions / total number of samples) * 100%.

Input / output format

Input: Image tensors (28x28 grayscale for MNIST, 32x32 RGB for CIFAR-10) or sequential row vectors for RNN tasks.

Output: Class label predictions (0-9 for MNIST/CIFAR-10) or probability distributions over classes.

Scoring recipe

def compute_accuracy(predictions, gold_labels):
    total = len(gold_labels)
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    return (correct / total) * 100

Common pitfalls

• Confusing the sparsity level $\bar{\kappa}$ (percentage of pruned parameters) with the remaining parameter count $\kappa$.
• Assuming the method requires pretraining or iterative pruning schedules; SNIP computes sensitivity once at random initialization.
• Using incorrect mini-batch sizes for sensitivity computation (100 for MNIST, 128 for CIFAR), which directly affects which connections are pruned.

Evidence (verbatim from paper)

Overall, our approach prunes a substantial amount of parameters in a variety of network models with minimal or no loss in accuracy (< 1%).

Citation

@misc{lee2018snip,
  title={SNIP: Single-shot Network Pruning based on Connection Sensitivity},
  author={Lee et al. (2018)},
  year={2018},
  note={arXiv:1810.02340}
}

• arXiv: 1810.02340