Backx Attribution Eval

backx-attribution-eval

A Backdoor-based Explainable AI Benchmark for High Fidelity Evaluation of Attributions — Yang et al. (2024) (arXiv:2405.02344, 2024)

What this evaluates

This benchmark evaluates the fidelity and reliability of explainable AI (XAI) attribution methods in identifying backdoor triggers versus natural image features. It tests whether attribution techniques can consistently highlight injected trigger patterns across different visibility levels and attack types, while remaining invariant to clean input distributions.

Datasets

• CIFAR-10 — total ?; splits: test (-1)
• GTSRB — total ?; splits: test (-1)
• ImageNet 2012 — total ?; splits: validation (-1)

Metrics

• trigger recall (primary) — range: [0, 1]
  • Measures the proportion of ground-truth trigger pixels correctly identified by the attribution method. Calculated as the intersection between the attribution mask and the trigger mask divided by the total trigger mask area.
• attack success rate — range: [0, 1]
  • The fraction of poisoned samples where the model predicts the target backdoor class. Used here to assess how well attribution methods detect or mitigate backdoor influence.
• fractional probability change — range: [0, 1]
  • The relative change in the model's output probability for the target class when the trigger is present versus absent, scaled by attribution performance to evaluate attribution-based defense efficacy.

Input / output format

Input: RGB image samples (tensors) that are either clean or poisoned with fixed visible triggers (Blend attack) or input-specific invisible triggers (ISSBA attack).

Output: Attribution maps/masks of the same spatial dimensions as the input, indicating the importance of each pixel/feature for the model's prediction.

Scoring recipe

def compute_trigger_recall(attribution_map, trigger_mask, threshold=0.5):
    pred_pos = attribution_map > threshold
    true_pos = trigger_mask > 0.5
    overlap = np.logical_and(pred_pos, true_pos).sum()
    recall = overlap / true_pos.sum() if true_pos.sum() > 0 else 0.0
    return recall

Common pitfalls

• Confusing trigger recall (measuring attribution fidelity) with attack success rate (measuring backdoor efficacy), leading to misinterpretation of method performance.
• Failing to standardize reference inputs and interpolation counts across different attribution families (e.g., CAM vs. IG), which skews comparison fairness.
• Evaluating only visible triggers while ignoring input-specific invisible triggers, missing critical robustness gaps in attribution methods.

Evidence (verbatim from paper)

Fig. 17. The comparison of (a) attack success rate and (b) trigger recall across different model architectures using different attribution methods on ImageNet. A lower attack success rate indicates better results, while a higher trigger recall indicates better results.

Citation

@misc{yang2024backx,
  title={A Backdoor-based Explainable AI Benchmark for High Fidelity Evaluation of Attributions},
  author={Yang et al. (2024)},
  year={2024},
  note={arXiv:2405.02344}
}

• arXiv: 2405.02344