Sparse Gpu Kernel Eval

sparse-gpu-kernel-eval

Sparse GPU Kernels for Deep Learning — Gale et al. (2020) (arXiv:2006.10901, 2020)

What this evaluates

Evaluates the performance and efficiency of custom sparse GPU kernels for SpMM and SDDMM operations against standard libraries like cuSPARSE on deep learning workloads. It measures computational throughput, memory usage, and end-to-end speedups across various model architectures and batch sizes.

Datasets

• Sparse Matrix Dataset from DNNs — total 3012; splits: benchmark (3012)

Metrics

• Geometric mean speedup (primary) — range: other
  • Geometric mean of the ratio (baseline_time / kernel_time) computed across all benchmark problems.
• Peak throughput (TFLOPs) — range: other
  • Measured floating-point operations per second divided by 1e12, reported as peak achieved performance.
• Bits per dimension — range: other
  • Standard metric for image generation quality; lower values indicate better reconstruction accuracy.

Input / output format

Input: Sparse matrix in CSR format and a dense matrix, along with specified batch size and precision configuration (single or mixed).

Output: Kernel execution time or computed throughput (TFLOPs) on the GPU.

Scoring recipe

def compute_speedup(baseline_times, kernel_times):
    ratios = [b / k for b, k in zip(baseline_times, kernel_times) if k > 0]
    geometric_mean = exp(mean(log(ratios)))
    return geometric_mean

Common pitfalls

• For convolution benchmarks (ResNet-50), the time for the im2col transformation is explicitly excluded from the SpMM/SDDMM benchmark timings.
• When comparing against cusparseConstrainedGeMM for SDDMM, the time to explicitly transpose the right-hand operand using cuBLAS is included in the baseline's timing.
• Pre-processing overhead for the ASpT baseline is excluded from the reported benchmark times.

Evidence (verbatim from paper)

Across all benchmarks, our SpMM and SDDMM kernels show significant advantages over Nvidia cuSPARSE. For single-precision SpMM, our kernel achieves a geometric mean speedup of 3.58× and reaches 4.29TFLOPs, representing 27.3% of single-precision peak. Our kernel outperforms cuSPARSE on 99.75% of the problems in our dataset.

Citation

@misc{gale2020sparse,
  title={Sparse GPU Kernels for Deep Learning},
  author={Gale et al. (2020)},
  year={2020},
  note={arXiv:2006.10901}
}

• arXiv: 2006.10901