Validitysoft

validitysoft

SCENE: Evaluating Explainable AI Techniques Using Soft Counterfactuals — Zheng et al. (2024) (arXiv:2408.04575, 2024)

What this evaluates

Evaluates the faithfulness and semantic plausibility of model-agnostic XAI techniques by generating soft counterfactuals via token-level perturbations. It measures whether perturbations actually change model predictions and whether the generated explanations align with the true causal impact of those changes.

Datasets

• Unspecified NLP dataset (CNN/RNN/Transformer baselines) — total ?; splits: test (-1); repo https://github.com/HaoranZhengRaul/SCENE.git

Metrics

• Validitysoft (primary) — range: other
  • Measures whether soft counterfactual perturbations successfully change the model's prediction. Computed as the difference between original and perturbed prediction probabilities.
• Csoft (primary) — range: other
  • Measures whether the XAI explanations align with the actual causal impacts of the perturbations. Computed as the Spearman correlation between explanation attributions and prediction changes.
• Infidelity — range: [0, 1]
  • Measures the stability of the model's behavior after perturbations. Lower values indicate higher fidelity to the original model.
• Human Agreement — range: [0, 1]
  • Measures alignment between XAI attributions and human interpretation of important attributes.

Input / output format

Input: Natural language text instance, model architecture type (CNN/RNN/Transformer), and XAI method attribution scores.

Output: Soft counterfactual text, perturbed model prediction/probability, and computed metric values (Validitysoft, Csoft, Infidelity, Human Agreement, Average Time).

Scoring recipe

def compute_validitysoft(orig_pred, perturbed_pred):
    return orig_pred - perturbed_pred

def compute_csoft(attribution_scores, pred_changes):
    return spearman_corr(attribution_scores, pred_changes)

def compute_infidelity(orig_pred, perturbed_pred):
    return abs(orig_pred - perturbed_pred)

# Aggregate across dataset
validitysoft_mean = mean([compute_validitysoft(o, p) for o, p in zip(orig_preds, pert_preds)])
csoft_mean = mean([compute_csoft(attr, change) for attr, change in zip(attributions, changes)])
infidelity_mean = mean([compute_infidelity(o, p) for o, p in zip(orig_preds, pert_preds)])

Common pitfalls

• Using hard token replacement instead of soft counterfactuals, which breaks semantic plausibility and contextual coherence.
• Assuming architecture-agnostic applicability (e.g., Guided GradCAM fails on RNNs lacking internal embedding layers).
• Prioritizing Human Agreement over Infidelity/Validitysoft, as human intuition does not always correlate with model faithfulness.

Evidence (verbatim from paper)

We assess the performance of SCENE by comparing its Validitysoft and Csoft metrics, along with Infidelity, against the ground truth across three neural architectures using Spearman's correlation.

Citation

@misc{zheng2024scene,
  title={SCENE: Evaluating Explainable AI Techniques Using Soft Counterfactuals},
  author={Zheng et al. (2024)},
  year={2024},
  note={arXiv:2408.04575}
}

• arXiv: 2408.04575