Ma Snn Eval

ma-snn-eval

Attention Spiking Neural Networks — Man Yao et al. (2022) (arXiv:2209.13929, 2022)

What this evaluates

Evaluates the effectiveness and energy efficiency of Multi-dimensional Attention (MA) modules integrated into Spiking Neural Networks (SNNs) for event-based action recognition and static image classification.

Datasets

• DVS128 Gesture — total 1342; splits: test (-1)
• DVS128 Gait — total 4200; splits: test (-1)
• ImageNet-1K — total 1330000; splits: train (1280000), test (50000)

Metrics

• Top-1 Accuracy (%) (primary) — range: percent
  • Percentage of correctly classified samples out of the total test set.
• Energy Efficiency ($r_{EE}$) — range: ratio
  • Ratio of baseline or ANN energy consumption to the SNN's energy consumption.
• NASAR — range: [0, 1]
  • Normalized Average Spiking Activity Rate, representing the average fraction of neurons firing per time step.

Input / output format

Input: Event streams (DVS datasets) or static images (ImageNet-1K) processed by SNN backbones.

Output: Class predictions (for classification tasks).

Scoring recipe

def compute_metrics(predictions, gold_labels, spike_counts, baseline_energy, num_neurons, time_steps):
    accuracy = (predictions == gold_labels).sum() / len(gold_labels) * 100
    energy_efficiency = baseline_energy / spike_counts
    nasar = spike_counts / (num_neurons * time_steps)
    return {'accuracy': accuracy, 'energy_efficiency': energy_efficiency, 'nasar': nasar}

Common pitfalls

• Energy efficiency is reported as a relative ratio ($r_{EE}$) against a baseline or ANN, not absolute hardware energy consumption.
• NASAR (Normalized Average Spiking Activity Rate) measures average spiking activity per neuron per time step, which differs from total spike counts.
• Latency ($T$) is not fixed across experiments; varying $T$ drastically changes both accuracy and energy metrics, so comparisons must control for time steps.

Evidence (verbatim from paper)

In TableI, we report the accuracy of each vanilla model and its attention counterpart, and compare TCSA-SNN with previous works. We observe that in every comparison, TCSA-SNN outperforms the vanilla architectures, suggesting that the benefits of attention modules are not confined to a single event-based dataset, limited base architecture, or fixed output latency.

Citation

@misc{yao2022attention,
  title={Attention Spiking Neural Networks},
  author={Man Yao et al. (2022)},
  year={2022},
  note={arXiv:2209.13929}
}

• arXiv: 2209.13929