Snntop1 Accuracy Eval

snntop1-accuracy-eval

Going Deeper With Directly-Trained Larger Spiking Neural Networks — Hanle Zheng et al. (2020) (arXiv:2011.05280, 2020)

What this evaluates

Evaluates the classification accuracy of directly-trained spiking neural networks (SNNs) on both static image recognition and neuromorphic event-based vision tasks. It probes the model's ability to maintain gradient stability and high predictive performance while operating with minimal simulation timesteps, highlighting efficiency gains over traditional ANN-SNN conversion methods.

Datasets

• CIFAR-10 — total 60000; splits: train (50000), test (10000)
• ImageNet — total 1330000; splits: train (1280000), val (50000)
• DVS-Gesture — total ?; splits: train (-1), test (-1)
• DVS-CIFAR10 — total 10000; splits: train (-1), test (-1)

Metrics

• top-1 accuracy (primary) — range: [0, 1]
  • Fraction of correctly classified instances out of the total number of instances in the evaluation set. Computed as (number of correct predictions) / (total number of predictions).

Input / output format

Input: For static datasets: RGB images of fixed resolution (32x32 for CIFAR-10, standard/large for ImageNet). For neuromorphic datasets: Event streams or temporal slices of spike trains processed sequentially over a fixed number of timesteps (T).

Output: A single predicted class label from the dataset's predefined class set (e.g., 10 classes for CIFAR-10/DVS-CIFAR10, 1000 for ImageNet, 11 for DVS-Gesture).

Scoring recipe

correct = 0
total = len(predictions)
for pred, gold in zip(predictions, gold_labels):
    if pred == gold:
        correct += 1
accuracy = correct / total

Common pitfalls

• Comparing results across papers without matching the number of simulation timesteps (T), as accuracy and latency are highly timestep-dependent.
• Confusing directly-trained SNNs with ANN-to-SNN converted models, which typically require >100 timesteps to achieve comparable accuracy.
• Ignoring the temporal alignment and slicing strategy for neuromorphic event streams, which significantly impacts performance on DVS-Gesture and DVS-CIFAR10.

Evidence (verbatim from paper)

In this work, our model achieves state-of-the-art performance (93.15% top-1 accuracy with only 6 timesteps) on CIFAR-10, which not only greatly reduces latency and computation cost compared with other SNN models.

Citation

@misc{zheng2020goingdeeper,
  title={Going Deeper With Directly-Trained Larger Spiking Neural Networks},
  author={Hanle Zheng et al. (2020)},
  year={2020},
  note={arXiv:2011.05280}
}

• arXiv: 2011.05280