Results for “gpu”

Deploy and serve AI/ML models on Modal's serverless cloud platform with on-demand GPUs, autoscaling containers, persistent storage, and scheduled jobs.

Benchmarks inference latency and computational runtime of transformer models and MLX operations across Apple Silicon and NVIDIA GPU backends, with configurable input lengths and batch sizes.

Run LLM inference on CPU, Apple Silicon, and consumer GPUs without NVIDIA hardware. Use for edge deployment, M1/M2/M3 Macs, AMD/Intel GPUs, or when CUDA is unavailable. Supports GGUF quantization (1.5-8 bit) for reduced memory and 4-10× speedup vs PyTorch on CPU.

Run Megatron-LM (MLM) and Megatron Bridge training with mock or real data, covering correlation testing, available recipes, and multi-GPU examples.

Walks users from a bare AKS cluster to a running AI model deployment, covering cluster verification, controller install, GPU assessment, provider setup, and first deployment.

Quantize large language models to 4-bit using activation-aware weight quantization, achieving ~3x speedup with minimal accuracy loss for deployment on limited GPU memory.

Submit and manage TAO training, evaluation, and inference jobs on SLURM GPU clusters over SSH with sbatch/srun, Pyxis/Enroot containers, and Lustre-backed storage.

Assesses NumPy code for cuPyNumeric migration readiness by analyzing source code against an API support manifest and GPU-scaling idioms, producing a structured verdict with per-finding reasoning.

Submit TAO jobs to Lepton managed GPU compute on DGX Cloud, with run/status/cancel interface and multi-node distributed training support.

Run TAO SDK jobs as Docker containers on a local or remote Docker daemon with NVIDIA GPU support, including preflight checks and credential handling.

Trains and fine-tunes vision models for object detection, image classification, and segmentation using Hugging Face Transformers on cloud GPUs, with automatic dataset validation and Hub persistence.

Reduces peak GPU memory in Megatron Bridge training by applying expandable segments, parallelism resizing, activation recompute, and CPU offloading constraints.

GPU-accelerated data curation for LLM training, supporting text, image, video, and audio with fuzzy deduplication, quality filtering, semantic deduplication, PII redaction, and NSFW detection.

Fine-tune HuggingFace CV, VLM, or LLM models on local NVIDIA GPUs using an NGC PyTorch container, with support for full or LoRA training, dataset handling, and optional model push to the Hub.

Quantize large language models to 4-bit with minimal accuracy loss using GPTQ, enabling deployment of 70B+ models on consumer GPUs with 4× memory reduction and 3-4× faster inference.

Organize PyTorch code into LightningModules, configure Trainers for multi-GPU/TPU, implement data pipelines, callbacks, logging (W&B, TensorBoard, MLflow), and distributed training (DDP, FSDP, DeepSpeed) for scalable neural network training.

Trains large language models (2B-462B parameters) using NVIDIA Megatron-Core with advanced parallelism strategies for maximum GPU efficiency.

Evaluates computational throughput and real-time efficiency of embedded CPU and GPU platforms by measuring peak FLOPS via a matrix rotation kernel and assessing inference latency and power consumption on a robotic vision pipeline.

Adds PyTorch FSDP2 (fully_shard) to training scripts with correct init, sharding, mixed precision/offload config, and distributed checkpointing. Use when models exceed single-GPU memory or when you need DTensor-based sharding with DeviceMesh.

Profile a DeepStream pipeline with Nsight Systems and derive its configs from the measurement.

Provides guidance on using PyTorch for deep learning, covering tensors, autograd, nn.Module, DataLoaders, and best practices.

Indexes Megatron Bridge recipes and recommends the best starting config based on model, GPU count, and training goal.

Deploy Meta's LlamaGuard moderation model to filter LLM inputs and outputs across 6 safety categories using HuggingFace, vLLM, or FastAPI.

Provides expert guidance on PyTorch Fully Sharded Data Parallel (FSDP) training, covering parameter sharding, mixed precision, CPU offloading, and FSDP2.

Quantize large language models to 4-bit precision using activation-aware weight quantization, reducing memory footprint and speeding up inference with minimal accuracy loss.

Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware, with backend selection between vLLM, Transformers, and accelerate.

Run ML training and batch jobs across multiple clouds with automatic cost optimization, spot instance recovery, and unified orchestration.

Train large language models (7B-70B+) with RLHF using PPO, GRPO, DPO, and other algorithms, accelerated by Ray and vLLM for distributed multi-GPU setups.

Finetune the NV-Generate-CTMR MAISI VAE/autoencoder on user-supplied CT or MRI NIfTI volumes using a staged config and datalist workflow.

Add distributed training support to any PyTorch script with minimal code changes using a unified API for DDP, DeepSpeed, FSDP, and mixed precision.