tpu-workload-eval
In-Datacenter Performance Analysis of a Tensor Processing Unit — Jouppi et al. (2017) (arXiv:1704.04760, 2017)
What this evaluates
Evaluates the performance and energy efficiency of a Tensor Processing Unit (TPU) and alternative hardware designs across six specific neural network workloads. It probes how architectural parameters like memory bandwidth, clock rate, and matrix multiply unit size impact throughput and power consumption.
Datasets
- TPU Benchmark Workloads (MLP0, MLP1, LSTM0, LSTM1, CNN0, CNN1) — total ?; splits: test (-1)
Metrics
Watt/die (primary) — range: other
- Measures power consumption per TPU die, normalized by the number of devices per server (2 CPUs, 8 GPUs, or 4 TPUs). Calculated as total server power divided by the device count.
Performance Speedup — range: other
- Ratio of baseline execution time (CPU/GPU) to TPU execution time, or relative throughput comparison between architectures.
Input / output format
Input: Hardware design parameters (memory bandwidth, clock rate, matrix unit dimensions, accumulator count) and workload specifications (neural network architecture, matrix sizes, batch sizes).
Output: Performance metrics including clock cycles, power consumption (Watt/die), and relative speedup ratios compared to baseline CPUs/GPUs.
Scoring recipe
def calculate_watt_per_die(total_power, num_devices):
return total_power / num_devices
def calculate_speedup(baseline_time, tpu_time):
return baseline_time / tpu_time
def calculate_relative_perf_watt(tpu_val, baseline_val):
return tpu_val / baseline_val
Common pitfalls
- The evaluation relies on a performance model for sensitivity analysis rather than physical hardware counters, which may not capture all real-world overheads or compiler optimizations.
- Host server interaction overhead is added to TPU times, but the model omits CPU time for running the application share, making the speedup comparison slightly optimistic.
- Weighted mean performance masks individual workload behavior; MLPs/LSTMs are memory-bound while CNNs are compute-bound, so scaling clock rate or memory bandwidth affects them differently.
Evidence (verbatim from paper)
Figure 9 above shows the relative total-performance/Watt/die of TPU' leaps to 31X - 86X over Haswell and 25X - 41X over the K80. The incremental metric soars to an amazing 69X - 196X over Haswell and 42X - 68X over the K80.
Citation
@misc{jouppi2017tpu,
title={In-Datacenter Performance Analysis of a Tensor Processing Unit},
author={Jouppi et al. (2017)},
year={2017},
note={arXiv:1704.04760}
}
1---2name: tpu-workload-eval3description: Evaluates the performance and energy efficiency of a Tensor Processing Unit (TPU) and alternative hardware designs across six specific neural network workloads. It probes how architectural parameters like memory bandwidth, clock rate, and matrix multiply unit size impact throughput and power consumption. Use when the user wants to benchmark on TPU Benchmark Workloads (MLP0, MLP1, LSTM0, LSTM1, CNN0, CNN1), or asks about evaluating this task. Reports Watt/die.4---56# tpu-workload-eval78> In-Datacenter Performance Analysis of a Tensor Processing Unit — Jouppi et al. (2017) (arXiv:1704.04760, 2017)910## What this evaluates1112Evaluates the performance and energy efficiency of a Tensor Processing Unit (TPU) and alternative hardware designs across six specific neural network workloads. It probes how architectural parameters like memory bandwidth, clock rate, and matrix multiply unit size impact throughput and power consumption.1314## Datasets1516- **TPU Benchmark Workloads (MLP0, MLP1, LSTM0, LSTM1, CNN0, CNN1)** — total ?; splits: test (-1)1718## Metrics1920- `Watt/die` **(primary)** — range: other21 - Measures power consumption per TPU die, normalized by the number of devices per server (2 CPUs, 8 GPUs, or 4 TPUs). Calculated as total server power divided by the device count.22- `Performance Speedup` — range: other23 - Ratio of baseline execution time (CPU/GPU) to TPU execution time, or relative throughput comparison between architectures.2425## Input / output format2627**Input**: Hardware design parameters (memory bandwidth, clock rate, matrix unit dimensions, accumulator count) and workload specifications (neural network architecture, matrix sizes, batch sizes).2829**Output**: Performance metrics including clock cycles, power consumption (Watt/die), and relative speedup ratios compared to baseline CPUs/GPUs.3031## Scoring recipe3233```python34def calculate_watt_per_die(total_power, num_devices):35 return total_power / num_devices3637def calculate_speedup(baseline_time, tpu_time):38 return baseline_time / tpu_time3940def calculate_relative_perf_watt(tpu_val, baseline_val):41 return tpu_val / baseline_val42```4344## Common pitfalls4546- The evaluation relies on a performance model for sensitivity analysis rather than physical hardware counters, which may not capture all real-world overheads or compiler optimizations.47- Host server interaction overhead is added to TPU times, but the model omits CPU time for running the application share, making the speedup comparison slightly optimistic.48- Weighted mean performance masks individual workload behavior; MLPs/LSTMs are memory-bound while CNNs are compute-bound, so scaling clock rate or memory bandwidth affects them differently.4950## Evidence (verbatim from paper)5152> Figure 9 above shows the relative total-performance/Watt/die of TPU' leaps to 31X - 86X over Haswell and 25X - 41X over the K80. The incremental metric soars to an amazing 69X - 196X over Haswell and 42X - 68X over the K80.5354## Citation5556```bibtex57@misc{jouppi2017tpu,58 title={In-Datacenter Performance Analysis of a Tensor Processing Unit},59 author={Jouppi et al. (2017)},60 year={2017},61 note={arXiv:1704.04760}62}63```6465- arXiv: 1704.04760
qhjqhj00