Results for “density-modes”

Quantize LLMs to 8-bit or 4-bit for 50-75% memory reduction with minimal accuracy loss using bitsandbytes. Supports INT8, NF4, FP4 formats, QLoRA training, and 8-bit optimizers.

Accelerate LLM inference using speculative decoding, Medusa multiple heads, and lookahead decoding techniques for 1.5-3.6× speedup without quality loss.

Provides guidance for training Mixture-of-Experts models with long context windows, covering context parallelism sizing, selective recomputation, dispatcher choices, and practical patterns from recent experiments.

Fine-tune Cosmos Reason video QA models using supervised fine-tuning with FSDP parallelism, including dataset preparation, spec construction, and AutoML support.

Compress large language models using knowledge distillation from teacher to student models, covering temperature scaling, soft targets, reverse KLD, logit distillation, and MiniLLM training strategies.