Openxai Eval

openxai-eval

OpenXAI: Towards a Transparent Evaluation of Model Explanations — Agarwal et al. (2022) (arXiv:2206.11104, 2022)

What this evaluates

Evaluates the faithfulness, stability, and fairness of post-hoc feature attribution explanation methods (e.g., LIME, SHAP, gradient-based) on tabular datasets to enable reproducible and transparent comparisons.

Datasets

• Popular tabular datasets for XAI and fairness research — total ?; splits: (unstated)

Metrics

• faithfulness (primary) — range: [0, 1]
  • Quantitative measure of how well the explanation aligns with the model's actual decision process and feature importance.
• stability — range: [0, 1]
  • Quantitative measure of the consistency of explanations under small perturbations or repeated runs.
• fairness — range: [0, 1]
  • Quantitative measure assessing whether explanations exhibit bias or disparate impact across protected attributes.

Input / output format

Input: Tabular data instances processed by post-hoc explanation methods to generate feature attributions, alongside model predictions and ground-truth labels.

Output: Quantitative scores for faithfulness, stability, and fairness.

Scoring recipe

def evaluate_explanations(model_preds, gold_labels, explanations, protected_attrs):
    scores = {}
    scores['faithfulness'] = compute_faithfulness(model_preds, explanations)
    scores['stability'] = compute_stability(explanations)
    scores['fairness'] = compute_fairness(explanations, protected_attrs)
    return scores

Common pitfalls

• Lack of clear understanding of which explanation methods perform well on which metrics, even on simple tabular data
• Evaluation is initially restricted to tabular data, with text and image modalities planned for future releases
• Metrics are generic across modalities but require specific implementations for different data types

Evidence (verbatim from paper)

OpenXAI introduces a unified, open-source framework for systematically benchmarking post hoc explanation methods across diverse datasets and models, offering eleven quantitative metrics for faithfulness, stability, and fairness—enabling reproducible, transparent comparisons of methods like LIME, SHAP, and Integrated Gradients.

Citation

@misc{agarwal2022openxai,
  title={OpenXAI: Towards a Transparent Evaluation of Model Explanations},
  author={Agarwal et al. (2022)},
  year={2022},
  note={arXiv:2206.11104}
}

• arXiv: 2206.11104