Attribution Eval

attribution-eval

Explaining image classifiers by removing input features using generative models — Agarwal et al. (2019) (arXiv:1910.04256, 2019)

What this evaluates

Evaluates the faithfulness and robustness of perturbation-based image attribution methods by measuring how well generated heatmaps localize objects, predict probability drops upon feature removal, and maintain consistency under hyperparameter variations.

Datasets

• ImageNet — total ?; splits: test (1000)
• Places365 — total ?; splits: test (1000)

Metrics

• deletion metric (primary) — range: percent
  • Area under the curve (AUC) of the target-class prediction probability as input pixels with the highest attribution scores are sequentially zeroed out. Lower scores indicate higher faithfulness.
• object localization error — range: percent
  • Derived bounding boxes from heatmap thresholding are compared to ground-truth boxes using Intersection over Union (IoU). Error is calculated by thresholding IoU at 0.5 and averaging across images. Lower is better.
• saliency metric — range: other
  • log(max(a, 0.05)) - log(s(x_p)), where a is the patch size ratio and s(x_p) is the classification probability of the most salient patch. Lower scores indicate more accurate explanations.

Input / output format

Input: Original image, classifier model, and ground-truth bounding boxes (for localization). The evaluation takes the attribution heatmap generated by a method as the prediction.

Output: Attribution heatmap (2D tensor of same spatial dimensions as input image) indicating pixel importance scores.

Scoring recipe

def compute_deletion_metric(heatmap, image, classifier, step_size=224*8):
    flat_heatmap = heatmap.flatten()
    sorted_indices = np.argsort(flat_heatmap)[::-1]
    probs = []
    for i in range(0, len(sorted_indices), step_size):
        masked_img = image.copy()
        masked_img.flatten()[sorted_indices[:i]] = 0
        probs.append(classifier.predict(masked_img))
    return np.trapz(probs)

Common pitfalls

• Using naive perturbations (blur, gray, noise) leaves residual discriminative features, artificially inflating attribution accuracy.
• Hyperparameter sensitivity (patch size, superpixel count, mask resolution) drastically alters heatmaps; evaluating at a single setting misrepresents method robustness.
• The saliency metric uses log(max(a, 0.05)) to prevent undefined values for very small patches, which can skew scores if ignored.
• Localization error thresholding IoU at 0.5 and averaging can obscure poor performance on ambiguous or multi-object images.

Evidence (verbatim from paper)

Intuitively, if the attributions in an explanation correctly reflect the importance of input pixels, removing the input pixels of highest attributions should cause a substantial probability drop. The deletion metric [42] measures the area under the curve of the target-class probability as we gradually zero out input pixels of the highest attributions in descending order. The deletion scores are widely used to compare attribution methods [43,18,44,45] i.e., lower deletion scores are considered more accurate.

Citation

@misc{agarwal2019explaining,
  title={Explaining image classifiers by removing input features using generative models},
  author={Agarwal et al. (2019)},
  year={2019},
  note={arXiv:1910.04256}
}

• arXiv: 1910.04256