Fidelity

fidelity

Robust Infidelity: When Faithfulness Measures on Masked Language Models Are Misleading — Crothers et al. (2023) (arXiv:2308.06795, 2023)

What this evaluates

Probes whether input attribution methods accurately reflect token importance for model predictions, particularly when inputs are adversarially perturbed or masked out-of-distribution. It evaluates the consistency of fidelity scores across different model architectures and under various adversarial attacks.

Datasets

• Stanford Sentiment Treebank (SST-2) — total ?; splits: test (-1)
• Twitter climate change sentiment — total ?; splits: test (-1)
• Wikipedia Toxic Comments — total ?; splits: test (-1)
• Civil Comments — total ?; splits: test (-1)

Metrics

• fidelity (primary) — range: [-1, 1]
  • Measures the correlation between per-token attribution scores (computed via layer integrated gradients) and the actual change in model prediction probability when those tokens are iteratively masked.

Input / output format

Input: Raw text sequence (e.g., review, comment) fed to a pre-trained BERT or RoBERTa classifier.

Output: Predicted class label and per-token attribution scores (salience) computed via layer integrated gradients (N=30 steps).

Scoring recipe

def compute_fidelity(model, input_text, attributions, k=10):
    sorted_indices = np.argsort(attributions)
    masked_text = mask_tokens(input_text, sorted_indices[:k])
    pred_drop = model.predict(input_text) - model.predict(masked_text)
    return pearsonr(attributions, pred_drop)

Common pitfalls

• Masked inputs frequently fall outside the training distribution, causing spurious fidelity scores.
• Fidelity measures are highly model-specific and do not reliably correlate with adversarial robustness or true feature salience across architectures.

Evidence (verbatim from paper)

fidelity calculations in Table 4 are based on a sample of 1,600 total records evenly split across dataset-model combinations. Initial experiments indicated this was sufficient to observe consistent patterns in fidelity scores, while freeing computational resources to explore multiple combinations. We use N=30 for the number of steps for layer integrated gradient calculation, as this approximates the largest value that fits within the memory constraints of the system, and higher step counts typically produce more accurate explanations. To evaluate models in the presence of class imbalance, we use the macro F1 score.

Citation

@misc{crothers2023robustinfidelity,
  title={Robust Infidelity: When Faithfulness Measures on Masked Language Models Are Misleading},
  author={Crothers et al. (2023)},
  year={2023},
  note={arXiv:2308.06795}
}

• arXiv: 2308.06795