Results for “llm-mutation”

Run Megatron-LM (MLM) and Megatron Bridge training with mock or real data, covering correlation testing, available recipes, and multi-GPU examples.

Quantize LLMs to 8-bit or 4-bit for 50-75% memory reduction with minimal accuracy loss using bitsandbytes. Supports INT8, NF4, FP4 formats, QLoRA training, and 8-bit optimizers.

Accelerate LLM inference using speculative decoding, Medusa multiple heads, and lookahead decoding techniques for 1.5-3.6× speedup without quality loss.

Compress large language models using knowledge distillation from teacher to student models, covering temperature scaling, soft targets, reverse KLD, logit distillation, and MiniLLM training strategies.

Provides battle-tested PyTorch training recipes for LLMs, vision, diffusion, and biomedical domains, covering training loops, optimizer selection, LR scheduling, mixed precision, and debugging.

Reduce per-token latency on Jetson vLLM servers by appending speculative decoding configuration, with guidance on when to enable and how to benchmark the improvement.