Results for “pytorch-cuda-allocator”

Provides idiomatic PyTorch patterns and best practices for building robust, efficient, and reproducible deep learning applications, covering model architecture, training loops, data pipelines, and checkpointing.

PyTorch deep learning patterns and best practices for building robust, efficient, and reproducible training pipelines, model architectures, and data loading.

Run PyTorch training across GPUs with minimal changes.

Fixes common PyTorch bugs including percentile calculations, LayerNorm for Conv1d, and buffer edge cases in reinforcement learning and neural network code.

PyTorch deep learning development with transformers, diffusion models, and GPU optimization.

Optimizes transformer attention with Flash Attention for 2-4x speedup and 10-20x memory reduction. Supports PyTorch native SDPA, flash-attn library, H100 FP8, and sliding window attention.

Simplest distributed training API. 4 lines to add distributed support to any PyTorch script. Unified API for DeepSpeed/FSDP/Megatron/DDP. Automatic device placement, mixed precision (FP16/BF16/FP8). Interactive config, single launch command. HuggingFace ecosystem standard.

Optimizes transformer attention with Flash Attention for 2-4x speedup and 10-20x memory reduction. Use when training/running transformers with long sequences (>512 tokens), encountering GPU memory issues with attention, or need faster inference. Supports PyTorch native SDPA, flash-attn library, H100 FP8, and sliding window attention.

Build and train graph neural networks with PyTorch Geometric, covering node/link/graph classification, message passing layers, heterogeneous graphs, and custom datasets.

Organize PyTorch code into LightningModules, configure Trainers for multi-GPU/TPU, implement data pipelines, callbacks, logging (W&B, TensorBoard, MLflow), and distributed training (DDP, FSDP, DeepSpeed) for scalable neural network training.

Add distributed training support to any PyTorch script with minimal code changes using a unified API for DDP, DeepSpeed, FSDP, and mixed precision.

Organizes PyTorch code with a Trainer class, automatic distributed training (DDP/FSDP/DeepSpeed), callbacks, and minimal boilerplate. Scales from laptop to supercomputer with the same code.

Optimizes transformer attention with Flash Attention for 2-4x speedup and 10-20x memory reduction. Use when training/running transformers with long sequences (>512 tokens), encountering GPU memory issues with attention, or need faster inference. Supports PyTorch native SDPA, flash-attn library, H100 FP8, and sliding window attention.

Provides guidance on using PyTorch for deep learning, covering tensors, autograd, nn.Module, DataLoaders, and best practices.

Simplest distributed training API. 4 lines to add distributed support to any PyTorch script. Unified API for DeepSpeed/FSDP/Megatron/DDP. Automatic device placement, mixed precision (FP16/BF16/FP8). Interactive config, single launch command. HuggingFace ecosystem standard.

Provides battle-tested PyTorch training recipes for LLMs, vision, diffusion, and biomedical domains, covering training loops, optimizer selection, LR scheduling, mixed precision, and debugging.

Convert single-node PyTorch distributed scripts into multi-node Slurm sbatch jobs and debug common multi-node failures, covering srun-native and torch.distributed approaches, container setup, NCCL timeouts, and interactive allocation.

Provides PyTorch-native distributed LLM pretraining using torchtitan with 4D parallelism (FSDP2, TP, PP, CP). Use when pretraining Llama 3.1, DeepSeek V3, or custom models at scale from 8 to 512+ GPUs with Float8, torch.compile, and distributed checkpointing.

Distributed training orchestration across clusters. Scales PyTorch/TensorFlow/HuggingFace from laptop to 1000s of nodes. Built-in hyperparameter tuning with Ray Tune, fault tolerance, elastic scaling. Use when training massive models across multiple machines or running distributed hyperparameter sweeps.

Distributed training orchestration across clusters. Scales PyTorch/TensorFlow/HuggingFace from laptop to 1000s of nodes. Built-in hyperparameter tuning with Ray Tune, fault tolerance, elastic scaling. Use when training massive models across multiple machines or running distributed hyperparameter sweeps.

Build and train graph neural networks for drug discovery, protein modeling, and molecular science using PyTorch-native tools.

GPU-accelerate Python code using CuPy, Numba CUDA, Warp, cuDF, cuML, cuGraph, and other RAPIDS libraries for dramatic speedups on numerical, data, ML, graph, and simulation workloads.

Provides guidance for performing causal interventions on PyTorch models using pyvene's declarative intervention framework. Use when conducting causal tracing, activation patching, interchange intervention training, or testing causal hypotheses about model behavior.

Computes the AUROC metric using torchmetrics, handling binary, multiclass, and multilabel tasks with configurable thresholds and averaging.

Computer vision engineering skill for object detection, image segmentation, and visual AI systems. Covers CNN and Vision Transformer architectures, YOLO/Faster R-CNN/DETR detection, Mask R-CNN/SAM segmentation, and production deployment with ONNX/TensorRT. Includes PyTorch, torchvision, Ultralytics, Detectron2, and MMDetection frameworks. Use when building detection pipelines, training custom models, optimizing inference, or deploying vision systems.

Perform causal interventions on PyTorch models using pyvene's declarative framework for causal tracing, activation patching, and interchange intervention training.

Write, debug, and test NumPyro probabilistic programs on JAX with correct shapes, PRNG keys, and inference choice.

Provides PyTorch-native distributed LLM pretraining using torchtitan with 4D parallelism (FSDP2, TP, PP, CP). Use when pretraining Llama 3.1, DeepSeek V3, or custom models at scale from 8 to 512+ GPUs with Float8, torch.compile, and distributed checkpointing.

Computes the SQuAD metric using torchmetrics, given predictions and ground truth. Use when evaluating question-answering outputs with exact match and F1 scores.

Provides guidance for performing causal interventions on PyTorch models using pyvene's declarative intervention framework. Use when conducting causal tracing, activation patching, interchange intervention training, or testing causal hypotheses about model behavior.

Pretrains large language models from scratch using PyTorch-native distributed training with 4D parallelism (FSDP2, TP, PP, CP) and Float8 support on H100 GPUs.

Pretrains large language models at scale using PyTorch-native torchtitan with 4D parallelism, Float8, and distributed checkpointing.

Replaces GIoU with Complete IoU (CIoU) loss in PyTorch object tracking or detection tasks, combining overlap area, center-point distance, and aspect-ratio similarity for improved bounding-box regression.