Shapley Explanation Latency

shapley-explanation-latency

ONNXExplainer: an ONNX Based Generic Framework to Explain Neural Networks Using Shapley Values — Yong Zhao et al. (2023) (arXiv:2309.16916, 2023)

What this evaluates

Evaluates the computational efficiency (latency and memory) and explanation quality of a Shapley value-based neural network explainer framework against baseline implementations across standard vision models.

Datasets

• ILSVRC-2012 — total ?; splits: test (-1)

Metrics

• latency (primary) — range: other
  • Cold start time (time to explain the first image) plus the average per-image latency for the remaining 99 images in a batch of 100.
• agreement (%) — range: percent
  • Percentage of human users who visually agree that Shapley value simulation images from two explainers are equally close to the ground truth contribution scores.
• max_reference_images — range: other
  • Maximum number of reference samples that can be batched during explanation without exceeding GPU memory limits, measured separately for FP32 and FP16 precision.

Input / output format

Input: RGB images of size 3x224x224. A reference input tensor of all zeros is provided for baseline comparison.

Output: Shapley value simulation images (attribution maps) and per-image explanation latency/memory metrics.

Scoring recipe

cold_start = time(explain(image_1, ref_zeros))
avg_latency = mean(time(explain(image_i, ref_zeros)) for i in 2..100)
total_latency = cold_start + avg_latency

user_agreement = count(users_say_same) / total_users * 100

max_ref_images = max_batch_size_that_fits_in_v100_memory(explainer, precision)

Common pitfalls

• Failing to exclude the cold start/warmup time from average latency measurements, which inflates performance metrics due to framework initialization overhead.
• Comparing memory usage across different hardware configurations without normalizing for GPU architecture or precision (FP32 vs FP16).
• Using non-zero or dataset-specific reference inputs, which violates the paper's protocol of using an all-zero reference tensor for fair comparison.

Evidence (verbatim from paper)

We infer and explain 100 images in each benchmark. After loading the models, the first few inference requests can be significantly slower at run-time due to deferred initialization and optimizations. Thus we consider the time to explain the first image out of 100 as the cold start time (or warmup time) and average the per-image latency of the rest for each benchmark.

Citation

@misc{zhao2023onnxexplainer,
  title={ONNXExplainer: an ONNX Based Generic Framework to Explain Neural Networks Using Shapley Values},
  author={Yong Zhao et al. (2023)},
  year={2023},
  note={arXiv:2309.16916}
}

• arXiv: 2309.16916