Sparsert Spmm Conv Eval

sparsert-spmm-conv-eval

SparseRT: Accelerating Unstructured Sparsity on GPUs for Deep Learning Inference — Wang et al. (2020) (arXiv:2008.11849, 2020)

What this evaluates

This evaluation probes the computational efficiency and throughput of GPU inference kernels under unstructured sparsity. It measures how well a sparse matrix multiplication and sparse convolution implementation scales across different matrix dimensions and sparsity levels compared to dense and existing sparse baselines.

Datasets

• SparseRT SpMM & Convolution Benchmark — total 257; splits: test (257)

Metrics

• speedup (primary) — range: fold
  • Calculated as the runtime of a baseline library (cuBLAS, cuSPARSE, or cuDNN) divided by the runtime of SparseRT. Reported as geometric mean fold speedup across problem instances.
• TFLOPs — range: other
  • Total floating point operations required for the SpMM operation divided by the execution time in seconds.

Input / output format

Input: Sparse weight matrices (pruned from real neural networks at 90% or 95% sparsity) and corresponding dense activation matrices, specified by dimensions (M, N, K) and sparsity patterns.

Output: Computed dense result matrix; evaluation focuses on execution time and throughput rather than numerical correctness.

Scoring recipe

def compute_speedup(baseline_runtime, sparset_rt_runtime):
    return baseline_runtime / sparset_rt_runtime

def compute_tflops(total_flops, runtime_seconds):
    return total_flops / (runtime_seconds * 1e12)

# For each problem instance:
# speedup = compute_speedup(cuBLAS_runtime, sparset_rt_runtime)
# tflops = compute_tflops(total_flops, sparset_rt_runtime)
# Report geometric mean speedup across all instances.

Common pitfalls

• Using cuSPARSE as the primary baseline is misleading, as the authors note it is ill-suited for the specific matrix shapes and sparsity levels common in deep learning.
• Comparing speedups without acknowledging that dense baselines (cuBLAS) perform significantly wasted computation (e.g., 90% wasted work at 90% sparsity).
• Assuming higher sparsity (95%) always yields proportionally higher speedups; the paper notes efficiency slightly degrades at 95% compared to 90%.

Evidence (verbatim from paper)

We calculate the TFLOPs by dividing how many total floating point operations are required to perform the SpMM by the time it took each library to execute the operation. For cuBLAS, we perform the equivalent GeMM by representing the sparse matrix as a dense matrix. This results in 90% wasted work at this sparsity level.

Citation

@misc{wang2020sparsert,
  title={SparseRT: Accelerating Unstructured Sparsity on GPUs for Deep Learning Inference},
  author={Wang et al. (2020)},
  year={2020},
  note={arXiv:2008.11849}
}

• arXiv: 2008.11849