Saebench Eval

saebench-eval

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability — Karvonen et al. (2025) (arXiv:2503.09532, 2025)

What this evaluates

Evaluates sparse autoencoder (SAE) architectures across multiple dimensions including reconstruction fidelity, feature disentanglement, concept detection, and practical interpretability tasks. It systematically compares how different SAE designs, dictionary sizes, and sparsity levels impact these capabilities.

Datasets

• Gemma-2-2B — total ?; splits: train (-1)
• Pythia-160M — total ?; splits: train (-1)

Metrics

• Loss Recovered (primary) — range: [0, 1]
  • Measures reconstruction fidelity by calculating the proportion of variance or loss recovered by the SAE compared to the original residual stream activations.
• Automated Interpretability — range: [0, 1]
  • Scores the interpretability of individual SAE latents using automated evaluation pipelines, typically involving classifier accuracy or human-alignment proxies.
• Absorption — range: [0, 1]
  • Quantifies feature absorption, where a single latent captures multiple concepts or subsumes other latents, measured via intervention-based concept isolation tests.
• SCR — range: [0, 1]
  • Sparse Concept Removal: measures concept detection by zero-ablating a fixed number of top latents and evaluating the drop in concept activation or model performance.
• RAVEL — range: [0, 1]
  • Evaluates feature disentanglement by measuring how cleanly individual latents correspond to distinct, non-overlapping concepts or features.
• Sparse Probing — range: [0, 1]
  • Assesses concept detection by training sparse linear probes (1-sparse or K-sparse) on SAE latents to predict target concepts or properties.
• TPP — range: [0, 1]
  • Targeted Probe Perturbation: measures disentanglement by perturbing specific latents and observing the isolated effect on model outputs or concept representations.
• sparsity-fidelity frontier — range: other
  • A proxy metric plotting reconstruction fidelity against sparsity (L0) to evaluate the trade-off between reconstruction quality and activation sparsity.

Input / output format

Input: Residual stream activations extracted from specific layers of language models (e.g., layer 12 of Gemma-2-2B, layer 8 of Pythia-160M).

Output: Sparse latent activations and reconstructed activations from the SAE. For metric evaluation, scalar scores representing fidelity, disentanglement, or concept detection performance.

Scoring recipe

def compute_sae_metrics(sae_latents, original_activations, concept_labels):
    # Loss Recovered (reconstruction fidelity)
    loss_recovered = 1 - mse(sae_latents, original_activations) / var(original_activations)
    
    # SCR (Sparse Concept Removal)
    top_latents = get_top_k_indices(sae_latents, k=10)
    zero_ablated = sae_latents.copy()
    zero_ablated[:, top_latents] = 0
    scr_score = concept_accuracy(original_activations) - concept_accuracy(zero_ablated)
    
    # Sparse Probing
    probe = train_sparse_probe(sae_latents, concept_labels, sparsity=1)
    probing_score = probe.evaluate(concept_labels)
    
    return loss_recovered, scr_score, probing_score

Common pitfalls

• Relying solely on the sparsity-fidelity frontier, which does not reliably correlate with interpretability or disentanglement performance.
• Restricting sparsity sweeps to the conventional L0 in [20, 200] range, which misses optimal settings for disentanglement metrics that prefer L0 > 400.
• Scaling dictionary size with a fixed compute budget rather than fixed training steps/data, which confounds scaling trend analysis.

Evidence (verbatim from paper)

Matryoshka Batch TopK SAEs perform best on concept detection and feature disentanglement tasks, especially in the typical L0 range of 40-200 (5 of 8 metrics). Most notably, the Matryoshka SAE obtains best scores on several metrics (Absorption, RAVEL, Sparse Probing, and SCR in Figure 2 and TPP in Figure J) while performing worse than TopK and BatchTopK on the sparsity-fidelity frontier (Figure 2, upper left).

Citation

@misc{karvonen2025saebench,
  title={SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability},
  author={Karvonen et al. (2025)},
  year={2025},
  note={arXiv:2503.09532}
}

• arXiv: 2503.09532