qhjqhj00 imagenet32-eval
A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets — Chrabaszcz et al. (2017) (arXiv:1707.08819, 2017)
What this evaluates
Evaluates image classification performance on downsampled variants of ImageNet to test whether lower-resolution datasets can serve as reliable proxies for full-resolution ImageNet in hyperparameter tuning and architecture search. It probes the stability of optimal hyperparameters and model performance across different spatial resolutions while maintaining the original dataset's class structure and image count.
Datasets
- ImageNet32x32 — total ?; splits: train (-1), val (-1); repo https://github.com/PatrykChrabaszcz/Imagenet32_Scripts
- ImageNet64x64 — total ?; splits: train (-1), val (-1); repo https://github.com/PatrykChrabaszcz/Imagenet32_Scripts
- ImageNet16x16 — total ?; splits: train (-1), val (-1); repo https://github.com/PatrykChrabaszcz/Imagenet32_Scripts
Metrics
validation error rate(primary) — range: percent- Fraction of misclassified images in the validation set. Computed as (number of incorrect predictions) / (total validation samples). Reported after 31 epochs of training.
Input / output format
Input: RGB images resized to 32×32, 64×64, or 16×16 pixels. During training, images are augmented with horizontal flips and random shifts of up to 4 pixels horizontally and vertically.
Output: Predicted class label from the 1000 ImageNet categories.
Scoring recipe
def compute_validation_error_rate(predictions, gold_labels):
total = len(gold_labels)
errors = sum(1 for p, g in zip(predictions, gold_labels) if p != g)
return errors / total
Common pitfalls
- Network architecture is explicitly modified per resolution (adding or removing residual stacks), so performance differences may reflect architectural adaptations rather than pure dataset difficulty.
- The learning rate schedule is fixed (dropped by factor 5.0 every 10 epochs) regardless of resolution, which may not be optimal for all variants and can skew error rate comparisons.
- Data augmentation is limited to horizontal flips and small random shifts, unlike standard ImageNet's random crops and flips, potentially underestimating true classification performance.
Evidence (verbatim from paper)
Throughout, we show validation error rates obtained after training for 31 epochs (right after the last drop of the learning rate).
Citation
@misc{chrabaszcz2017imagenet32,
title={A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets},
author={Chrabaszcz et al. (2017)},
year={2017},
note={arXiv:1707.08819}
}
- arXiv: 1707.08819