Probing

Probing methods interpret model signals by training an auxiliary predictor (often linear) to decode a labeled property y from an internal vector $z$ (e.g., the residual stream state $x_l$ at layer $l$). Operationally, probing treats the model as a frozen feature extractor and assesses decodability: whether $y$ is recoverable from $z$ by a restricted hypothesis class (commonly linear), which supports localization by comparison across candidate objects (layers/heads/FFNs) via decoding performance or information-theoretic surrogates, typically followed by Causal Attribution to test functional necessity.

zjunlp Updated 1.5k repo stars

File contents

zjunlp/mechanist commit 4b5e391be6

Frequently asked questions

npx get-skill add zjunlp/probing