Black Box Attribution Eval

black-box-attribution-eval

Less is More: Efficient Black-box Attribution via Minimal Interpretable Subset Selection — Chen et al. (2025) (arXiv:2504.00470, 2025)

What this evaluates

Evaluates the faithfulness of black-box attribution methods by measuring how well identified input regions align with the model's decision-making process. It tests the ability of explanation algorithms to pinpoint critical features that drive correct predictions or cause errors.

Datasets

• ImageNet — total 7000; splits: test (5000), test_error (2000)
• CUB-200-2011 — total 1000; splits: test (600), test_error (400)
• CelebA — total 2000; splits: test (2000)
• VGG-Face2 — total 2000; splits: test (2000)
• LC25000 (Lung) — total 1000; splits: test_error (1000)
• VGG-Sound — total 927; splits: test (618), test_error (309)

Metrics

• Deletion AUC (primary) — range: [0, 1]
  • Measures the reduction in model output when the most important regions are progressively replaced with a baseline ($x_0=0$). Lower values indicate better faithfulness. Computed as the trapezoidal integral of the performance drop curve.
• Insertion AUC (primary) — range: [0, 1]
  • Measures the increase in model output as the most important regions are progressively revealed (others set to baseline $x_0=0$). Higher values indicate better faithfulness. Computed as the trapezoidal integral of the performance rise curve.
• average highest confidence — range: [0, 1]
  • Measures the maximum model confidence score achieved within a constrained search region of top-k important variables. Higher values indicate better error attribution. Formula: max_{T in T_[k]} f(x_{[x_T_bar = x_0]}).
• μ Fidelity — range: [0, 1]
  • Measures the correlation between the reduction in the model's score when variables are set to baseline and the importance of those variables. Higher values indicate better faithfulness.

Input / output format

Input: Image or audio input tensor, model architecture, and target prediction (correct or incorrect class).

Output: A ranked list or mask of input regions (e.g., superpixels or SAM segments) indicating their attribution importance scores.

Scoring recipe

def compute_auc(perturbed_scores, baseline=0.0):
    n = len(perturbed_scores)
    auc = 0.0
    for i in range(1, n):
        auc += (perturbed_scores[i] + perturbed_scores[i-1]) * (i - (i-1)) / (2 * n)
    return auc

# Deletion: mask top-k regions, score drops -> lower AUC is better
deletion_scores = [model(x_masked_top_k) for k in range(N)]
deletion_auc = compute_auc(deletion_scores)

# Insertion: reveal top-k regions, score rises -> higher AUC is better
insertion_scores = [model(x_masked_bottom_k) for k in range(N)]
insertion_auc = compute_auc(insertion_scores)

Common pitfalls

• Deletion AUC is lower-better, while Insertion AUC is higher-better; confusing the direction can invert results.
• The baseline value $x_0$ is fixed to 0, which may not be appropriate for all data distributions or model outputs.
• SAM-based segmentation can produce overly large regions, negatively impacting fine-grained attribution scores compared to SLICO superpixels.

Evidence (verbatim from paper)

We employ four fidelity metrics to evaluate our attribution methods. The first is the Deletion AUC score [23], which quantifies the reduction in the model's output when important regions are replaced with a baseline value... The second metric is the Insertion AUC score [23], which quantifies the increase in the model's output as important regions are progressively revealed... The third metric is the average highest confidence [8]... The final metric is μ Fidelity [80]...

Citation

@misc{chen2025lima,
  title={Less is More: Efficient Black-box Attribution via Minimal Interpretable Subset Selection},
  author={Chen et al. (2025)},
  year={2025},
  note={arXiv:2504.00470}
}

• arXiv: 2504.00470