spiNNaker2-benchmarks-eval

spiNNaker2-benchmarks-eval

The SpiNNaker 2 Processing Element Architecture for Hybrid Digital Neuromorphic Computing — Hoppner et al. (2021) (arXiv:2103.08392, 2021)

What this evaluates

Evaluates the energy efficiency and computational capability of the SpiNNaker2 processing element architecture across a suite of neuromorphic and deep learning workloads, including classical spiking neural networks, hybrid SNN/DNN frameworks, and standard DNN layers.

Datasets

• SpiNNaker2 Benchmark Suite — total ?; splits: test (-1)

Metrics

• energy_efficiency (primary) — range: other
  • Total energy consumed per operation or per benchmark run, measured during hardware execution. Exact formula and units (e.g., pJ/op or mW) are not specified in the provided excerpt.

Input / output format

Input: Hardware configuration of the SpiNNaker2 PE loaded with benchmark workloads: CoreMark, matrix multiplication, Synfire Chain (SNN), NEF (SNN/DNN), and convolutional/fully connected layers (DNN).

Output: Measured energy consumption, power draw, and execution metrics for each benchmark workload.

Scoring recipe

def compute_energy_efficiency(benchmark_workload, execution_trace):
    total_energy_joules = sum(power_samples * time_delta for power_samples, time_delta in execution_trace)
    total_ops = count_operations(benchmark_workload)
    return total_energy_joules / total_ops if total_ops > 0 else float('inf')

Common pitfalls

• Energy measurements are highly sensitive to the dynamic voltage and frequency scaling (DVFS) and adaptive body biasing settings used during execution.
• Reporting only aggregate energy without specifying the execution mode (event-triggered vs. frame-based) or hardware configuration makes cross-study comparisons unreliable.
• Hardware benchmarks like CoreMark measure integer performance, while neuromorphic workloads measure spiking dynamics; comparing them directly without normalization is misleading.

Evidence (verbatim from paper)

To show both, the energy efficiency of the Processing Element (PE) and the capability of hybrid digital neuromorphic of SpiNNaker2 we implement various benchmarks. This includes plain CoreMark and matrix multiplication cases and three diverse benchmark networks ranging from Synfire Chain representing the classical Spiking Neural Network (SNN) (section VI-B), through the Neural Engineering Framework (NEF) representing the combined SNN/DNN approach (section VI-C), to convolutional layers and fully connected layers as examples of Deep Neural Networks (DNN) (section ??).

Citation

@misc{hoppner2021spiNNaker2,
  title={The SpiNNaker 2 Processing Element Architecture for Hybrid Digital Neuromorphic Computing},
  author={Hoppner et al. (2021)},
  year={2021},
  note={arXiv:2103.08392}
}

• arXiv: 2103.08392