Results for “sparse-activation”

Trains and analyzes Sparse Autoencoders (SAEs) with SAELens to decompose neural network activations into interpretable features, covering loading pre-trained SAEs, training custom ones, and feature steering.

Trains, evaluates, exports, quantizes, and runs inference for Sparse4D multi-camera temporal 3D object detection and tracking models using TAO.

Provides expert guidance for distributed training with DeepSpeed, covering ZeRO optimization stages, pipeline parallelism, FP16/BF16/FP8, 1-bit Adam, and sparse attention.

Train Mixture of Experts (MoE) models using DeepSpeed or HuggingFace, covering architectures, routing, load balancing, and expert parallelism.