Results for “h100”
7 skills
orchestra-research Distributed LLM Pretraining Torchtitan
Pretrains large language models from scratch using PyTorch-native distributed training with 4D parallelism (FSDP2, TP, PP, CP) and Float8 support on H100 GPUs.
10.4k · bundle
orchestra-research Optimizing Attention Flash
Optimizes transformer attention with Flash Attention for 2-4x speedup and 10-20x memory reduction. Supports PyTorch native SDPA, flash-attn library, H100 FP8, and sliding window attention.
10.4k · bundle
orchestra-research Tensorrt LLM
Optimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency on NVIDIA GPUs (A100/H100).
10.4k · bundle
More results
qhjqhj00 Distributed LLM Pretraining Torchtitan
Pretrains large language models at scale using PyTorch-native torchtitan with 4D parallelism, Float8, and distributed checkpointing.
3 · bundle
orchestra-research Hqq Quantization
Quantize large language models to 8/4/3/2/1-bit precision without calibration data, using multiple optimized backends for deployment with vLLM or HuggingFace Transformers.
10.4k · bundle
orchestra-research Training Llms Megatron
Trains large language models (2B-462B parameters) using NVIDIA Megatron-Core with advanced parallelism strategies for maximum GPU efficiency.
10.4k · bundle
nvidia Nemo Mbridge Perf Moe Long Context
Provides guidance for training Mixture-of-Experts models with long context windows, covering context parallelism sizing, selective recomputation, dispatcher choices, and practical patterns from recent experiments.
2.2k · bundle