SkillMD SkillMD
light-dark-mode
Profile My skills Saved Collections Edit profile Submit a skill

Results for “sharding”

2 skills
orchestra-research
Pytorch Fsdp2
Adds PyTorch FSDP2 (fully_shard) to training scripts with correct init, sharding, mixed precision/offload config, and distributed checkpointing. Use when models exceed single-GPU memory or when you need DTensor-based sharding with DeviceMesh.
10.4k · bundle
More results
ichichuang
Pytorch Fsdp
Provides expert guidance on PyTorch Fully Sharded Data Parallel (FSDP) training, covering parameter sharding, mixed precision, CPU offloading, and FSDP2.
0 · bundle