Results for “distributed-checkpoint”

4 skills

orchestra-research

Adds PyTorch FSDP2 (fully_shard) to training scripts with correct init, sharding, mixed precision/offload config, and distributed checkpointing. Use when models exceed single-GPU memory or when you need DTensor-based sharding with DeviceMesh.

10.4k · bundle

More results

Scales AI and Python applications across clusters with distributed computing primitives for ML workloads.

cloudthinker-ai

Manages Ray clusters, jobs, Serve deployments, and distributed workloads via the Dashboard API and CLI, with discovery-first checks and safety guardrails.

distributed-llm-pretraining-torchtitan

Pretrains large language models at scale using PyTorch-native torchtitan with 4D parallelism, Float8, and distributed checkpointing.