Odin Mnist Eval

odin-mnist-eval

A 0.086-mm$^2$ 12.7-pJ/SOP 64k-Synapse 256-Neuron Online-Learning Digital Spiking Neuromorphic Processor in 28nm CMOS — Frenkel et al. (2018) (arXiv:1804.07858, 2018)

What this evaluates

Evaluates the classification accuracy and energy efficiency of a 256-neuron spiking neuromorphic processor (ODIN) on the MNIST handwritten digit dataset. It compares offline gradient-based weight training against online spike-driven synaptic plasticity (SDSP) learning, while characterizing hardware power consumption and energy per spike operation.

Datasets

• MNIST — total ?; splits: train (-1), test (-1)

Metrics

• classification accuracy (primary) — range: percent
  • Percentage of correctly classified MNIST digits by the 10-neuron SNN output.
• energy per SOP (E_SOP) — range: pJ
  • Incremental energy per spike operation calculated as (P - P_leak - P_idle * f_clk) / r_SOP, excluding static leakage and idle power.
• global energy per SOP (E_tot,SOP) — range: pJ
  • Total chip power divided by SOP rate (P / r_SOP), including leakage and idle power contributions.

Input / output format

Input: 16x16 downsampled MNIST images converted to rate-based Poisson-distributed spike trains, fed into a single-layer fully-connected SNN of 10 LIF neurons.

Output: Spike trains from 10 output neurons, one per digit class. Classification determined by the neuron with the highest firing rate or spike count.

Scoring recipe

def compute_accuracy(predictions, gold):
    correct = sum(1 for p, g in zip(predictions, gold) if p == g)
    return (correct / len(gold)) * 100

def compute_energy_metrics(P, P_leak, P_idle, f_clk, r_SOP):
    E_SOP = (P - P_leak - P_idle * f_clk) / r_SOP
    E_tot_SOP = P / r_SOP
    return E_SOP, E_tot_SOP

Common pitfalls

• Confusing incremental energy per SOP (E_SOP) with global energy per SOP (E_tot,SOP), which differ significantly due to leakage and idle power contributions.
• Assuming the MNIST accuracy represents state-of-the-art performance; the paper explicitly states the goal is to compare learning strategies, not to break accuracy records.
• Overlooking that spike coding uses rate-based Poisson trains during training, which may differ from inference coding schemes.

Evidence (verbatim from paper)

Benchmark for testing accuracy on image classification: pre-processing steps of the MNIST dataset of handwritten digits and the two considered setups for training the weights of a LIF-based 10-neuron spiking neural network implemented in the ODIN chip. (a) Off-chip offline weight training is carried out with quantization-aware stochastic gradient descent on a 10-neuron single-layer artificial neural network (ANN) with softmax units, implemented using Keras with a TensorFlow backend. The chosen optimizer and loss function are Adam with categorical cross-entropy. (b) On-chip online teacher-based weight training with the local SDSP learning rule.

Citation

@misc{frenkel2018odin,
  title={A 0.086-mm$^2$ 12.7-pJ/SOP 64k-Synapse 256-Neuron Online-Learning Digital Spiking Neuromorphic Processor in 28nm CMOS},
  author={Frenkel et al. (2018)},
  year={2018},
  note={arXiv:1804.07858}
}

• arXiv: 1804.07858