Research Model Representations
Start With A Causal Question
Name the behavior, candidate representation, intervention point, predicted behavioral change, and falsifying result. A decodable probe or correlated activation is evidence of information, not evidence that the model uses it causally.
Workflow
- Pin the checkpoint, tokenizer, template, framework, and source-code revision.
- Construct matched positive, negative, and neutral examples. Control length, topic, syntax, and token positions where practical.
- Define hook names by verified model architecture rather than assuming layer paths from a related model.
- Decide local versus remote instrumentation. Before remote execution, classify prompts and activations, confirm authorization, retention/logging behavior, network exposure, and paid-compute budget; keep proprietary or sensitive inputs local unless explicitly approved.
- Collect activations with explicit batch, dtype, device, token-selection, pooling, and normalization rules.
- Split probe training and evaluation examples by the true contamination boundary.
- Establish selectivity controls: random labels, random directions, held-out concepts, and simple surface-feature baselines.
- Test causality through ablation, patching, steering, or counterfactual replacement at held-out examples.
- Measure target behavior and unrelated capability guardrails across layers, positions, and intervention strengths.
- Report unstable seeds, negative results, multiple-comparison choices, and architecture-specific limitations.
Tool Selection
- TransformerLens provides activation caching and hook-oriented analysis for supported architectures.
- NNsight provides model instrumentation and remote-capable intervention workflows.
Verify the pinned model and operation against the selected tool. Use direct framework hooks when a small, explicit intervention is clearer than adding a large abstraction.
References
Read references/causal-representation-evidence.md before interpreting a probe or direction as a mechanism.
1---2name: research-model-representations3description: Design causal research into model activations, features, attention, residual streams, probes, and circuits. Use when locating behavior, testing a direction or feature, comparing layers, or reproducing interpretability research.4---56# Research Model Representations78## Start With A Causal Question910Name the behavior, candidate representation, intervention point, predicted behavioral change, and falsifying result. A decodable probe or correlated activation is evidence of information, not evidence that the model uses it causally.1112## Workflow13141. Pin the checkpoint, tokenizer, template, framework, and source-code revision.152. Construct matched positive, negative, and neutral examples. Control length, topic, syntax, and token positions where practical.163. Define hook names by verified model architecture rather than assuming layer paths from a related model.174. Decide local versus remote instrumentation. Before remote execution, classify prompts and activations, confirm authorization, retention/logging behavior, network exposure, and paid-compute budget; keep proprietary or sensitive inputs local unless explicitly approved.185. Collect activations with explicit batch, dtype, device, token-selection, pooling, and normalization rules.196. Split probe training and evaluation examples by the true contamination boundary.207. Establish selectivity controls: random labels, random directions, held-out concepts, and simple surface-feature baselines.218. Test causality through ablation, patching, steering, or counterfactual replacement at held-out examples.229. Measure target behavior and unrelated capability guardrails across layers, positions, and intervention strengths.2310. Report unstable seeds, negative results, multiple-comparison choices, and architecture-specific limitations.2425## Tool Selection2627- [TransformerLens](https://github.com/TransformerLensOrg/TransformerLens) provides activation caching and hook-oriented analysis for supported architectures.28- [NNsight](https://github.com/ndif-team/nnsight) provides model instrumentation and remote-capable intervention workflows.2930Verify the pinned model and operation against the selected tool. Use direct framework hooks when a small, explicit intervention is clearer than adding a large abstraction.3132## References3334Read `references/causal-representation-evidence.md` before interpreting a probe or direction as a mechanism.
gaelic-ghost