Vega Hardware Bench

vega-hardware-bench

Vega: A 10-Core SoC for IoT End-Nodes with DNN Acceleration and Cognitive Wake-Up From MRAM-Based State-Retentive Sleep Mode — Rossi et al. (2021) (arXiv:2110.09101, 2021)

What this evaluates

Measures the performance, energy efficiency, and latency of the Vega SoC on floating-point near-sensor analytic applications (NSAA) and deep neural network (DNN) inference workloads.

Datasets

• FP NSAA Kernels — total ?; splits: test (-1)
• MobileNetV2 — total ?; splits: test (-1)
• RepVGG-A0/A1/A2 — total ?; splits: test (-1)

Metrics

• Energy Efficiency (primary) — range: other
  • Total operations (MACs or FLOPs) divided by total energy consumption in joules, reported as GOPS/W or GFLOPS/W.
• Latency — range: other
  • Total execution time for a workload or inference pass, measured in milliseconds or microseconds.
• Energy per Inference — range: other
  • Total energy consumed during a single forward pass of a DNN, measured in millijoules (mJ).
• ImageNet Top-1 Accuracy — range: percent
  • Percentage of correctly classified images in the ImageNet validation set.

Input / output format

Input: C/C++ source code for FP kernels; DNN model definitions (MobileNetV2, RepVGG) compiled to 8-bit integer tensors using PULP-NN library.

Output: Execution latency (ms/μs), energy consumption (mJ/pJ), throughput (MFLOPS/MACs), and classification accuracy (%).

Scoring recipe

# Calculate energy efficiency for DNN inference
total_macs = sum(layer_macs for layer in network)
total_energy_j = energy_compute + energy_mram_access + energy_hyperram_access
efficiency_gops_w = total_macs / (total_energy_j * soc_clock_hz)

# Calculate accuracy
correct_preds = sum(1 for p, g in zip(predictions, ground_truth) if p == g)
accuracy_pct = (correct_preds / len(ground_truth)) * 100

Common pitfalls

• Confusing software-only execution with hardware accelerator (HWCE) results, which differ by ~3x speedup.
• Overlooking that MRAM energy savings only apply to weight storage, not activation memory.
• Assuming uniform compute-bound layers; early network layers are often memory-bound due to large activation sizes.

Evidence (verbatim from paper)

The results are presented for both HWCE-based and SW-based computation, using the same PULP-NN layers of the MobileNetV2 case study in the latter case. Almost all layers are compute-dominated, except for the final fully connected layer. In such conditions, HWCE-based execution delivers a 3x speedup over SW-based; and a 60-90% boost in system-level energy efficiency, depending on how much is the energy impact of HyperRAM traffic.

Citation

@misc{rossi2021vega,
  title={Vega: A 10-Core SoC for IoT End-Nodes with DNN Acceleration and Cognitive Wake-Up From MRAM-Based State-Retentive Sleep Mode},
  author={Rossi et al. (2021)},
  year={2021},
  note={arXiv:2110.09101}
}

• arXiv: 2110.09101