Results for “sparse4d”

Train and analyze Sparse Autoencoders (SAEs) using SAELens to decompose neural network activations into interpretable features for mechanistic interpretability research.

Trains and analyzes Sparse Autoencoders (SAEs) with SAELens to decompose neural network activations into interpretable features, covering loading pre-trained SAEs, training custom ones, and feature steering.

Compress large language models by 40-60% with minimal accuracy loss using one-shot pruning techniques like Wanda and SparseGPT, enabling faster inference and deployment on constrained hardware.

Provides expert guidance for distributed training with DeepSpeed, covering ZeRO optimization stages, pipeline parallelism, FP16/BF16/FP8, 1-bit Adam, and sparse attention.

Add distributed training support to any PyTorch script with minimal code changes using a unified API for DDP, DeepSpeed, FSDP, and mixed precision.