Histopathology Explainer Eval

histopathology-explainer-eval

KS-GNNExplainer: Global Model Interpretation Through Instance Explanations On Histopathology images — Abdous et al. (2023) (arXiv:2304.08240, 2023)

What this evaluates

Evaluates graph neural network explainers on histopathology images by measuring how well they identify critical tumor nuclei and preserve model fidelity. It probes the explainer's ability to extract global, class-specific patterns and produce accurate instance-level importance maps for downstream nuclei classification.

Datasets

• BRACS — total ?; splits: test (-1)
• BACH — total ?; splits: test (-1)
• BreCaHAD — total ?; splits: test (-1)
• CRC — total ?; splits: test (-1)

Metrics

• macro-averaged F1 score (primary) — range: [0, 1]
  • Standard macro-averaged F1 score computed over binary tumor vs. non-tumor predictions derived from importance scores.
• Fidelity — range: [0, 1]
  • Measures the drop in model prediction confidence when the most important nodes (per explainer) are masked, evaluated across a range of masking thresholds.

Input / output format

Input: Histopathology image patches or nuclei represented as graph nodes/edges, processed by a GNN to produce class predictions and node importance scores.

Output: Per-nucleus importance scores (continuous values), thresholded at 0.5 to yield binary tumor/non-tumor predictions.

Scoring recipe

# F1 Score Calculation
preds = [1 if score >= 0.5 else 0 for score in importance_scores]
macro_f1 = f1_score(y_true, preds, average='macro')

# Fidelity Calculation (standard explainer protocol)
fidelity_scores = []
for threshold in thresholds:
    masked_graph = mask_top_k_nodes(graph, importance_scores, k=threshold)
    pred_change = abs(original_prediction - model(masked_graph).prediction)
    fidelity_scores.append(pred_change)

Common pitfalls

• Fidelity metric definition and computation vary significantly across different explainer papers; the exact masking procedure and threshold sweep must be replicated.
• The F1 score relies on a fixed 0.5 threshold for importance scores, which may not be optimal for all datasets or explainers.
• Instance-level importance maps are aggregated to compute global patterns, but the exact aggregation method for the KS test is not detailed in the evaluation section.

Evidence (verbatim from paper)

We also use the mentioned metrics to further analyze previous explainers quantitatively. First, we plot changes in the Fidelity for all explainers, as well as a random one, for different thresholds on each dataset in Fig. [2]... In addition, the macro-averaged F1 score results are shown in Table [2]... Here, we considered a binary classification of tumor vs. non-tumor nuclei through the importance score. We assign nuclei with the importance score of 0.5 and greater as tumor, and else as non-tumor, and compare against the ground truth nuclei label.

Citation

@misc{abdous2023ksgnnexplainer,
  title={KS-GNNExplainer: Global Model Interpretation Through Instance Explanations On Histopathology images},
  author={Abdous et al. (2023)},
  year={2023},
  note={arXiv:2304.08240}
}

• arXiv: 2304.08240