qhjqhj00 blas-performance-benchmark
A New Vectorization Technique for Expression Templates in C++ — Progsch et al. (2011) (arXiv:1109.1264, 2011)
What this evaluates
Evaluates the runtime performance and throughput of a C++ expression template library (SALT) against optimized BLAS implementations (Intel MKL) and other template libraries (Eigen) for standard vector operations.
Datasets
- Synthetic Vector Operations Benchmark — total ?; splits: test (-1)
Metrics
performance (GFLOPS)(primary) — range: GFLOPS- Measured in Giga-Floating Point Operations per Second. Calculated as (2*N)/time for dot product and AXPY, and N/time for scaling. Normalized against theoretical CPU peak.
Input / output format
Input: Single-precision floating-point vectors of varying length N, and scalar alpha for AXPY.
Output: Throughput in GFLOPS for each vector size and operation type.
Scoring recipe
def compute_gflops(operation, N, time_seconds):
if operation in ['dot', 'axpy']:
ops = 2 * N
elif operation == 'scal':
ops = N
else:
ops = 0
return (ops / time_seconds) / 1e9
Common pitfalls
- Results are highly sensitive to CPU cache hierarchy and memory bandwidth, which differ across hardware generations.
- Compiler flags (-O3 -msse3) and specific versions drastically affect auto-vectorization and loop unrolling, making cross-version comparisons difficult.
- Out-of-place operations require separate copy and scale steps in BLAS, inflating their baseline time compared to expression templates that fuse them.
Evidence (verbatim from paper)
Measurements were taken for three common vector operations. These are the dot product (xDOT), vector scaling (xSCAL) and operations of the form y=y+alpha x (xAXPY). The results were compared to Intel’s MKL library (BLAS interface) and to Eigen (expression templates). The results for a Intel Core i5-580M CPU (2.66 GHz, 3.33 GHz TurboBoost) are shown in figures [3], [4] and [5]. The grey lines indicate the sizes of the different caches and the 17.1 GB/s memory bandwidth and 13.3 Gflop/s theoretical peak performance of the used processor.
Citation
@misc{progsch2011salt,
title={A New Vectorization Technique for Expression Templates in C++},
author={Progsch et al. (2011)},
year={2011},
note={arXiv:1109.1264}
}
- arXiv: 1109.1264