Results for “gpu-memory”
19 skills
nvidia nemo-mbridge-perf-memory-tuning
Reduces peak GPU memory in Megatron Bridge training by applying expandable segments, parallelism resizing, activation recompute, and CPU offloading constraints.
2.2k · bundle
orchestra-research awq-quantization
Quantize large language models to 4-bit using activation-aware weight quantization, achieving ~3x speedup with minimal accuracy loss for deployment on limited GPU memory.
10.4k · bundle
antigravity hf-mem
Estimates GPU memory required to load Safetensors or GGUF model weights for inference from the Hugging Face Hub using HTTP Range requests, without downloading weights locally.
42.4k
samyakjhaveri cuda-omp-translator
Reference guide for evaluating LLM-generated translations between CUDA and OpenMP, covering memory model mapping, kernel launch patterns, shared memory, atomics, and common failure modes.
0
orchestra-research llama-cpp
Run LLM inference on CPU, Apple Silicon, and consumer GPUs without NVIDIA hardware. Use for edge deployment, M1/M2/M3 Macs, AMD/Intel GPUs, or when CUDA is unavailable. Supports GGUF quantization (1.5-8 bit) for reduced memory and 4-10× speedup vs PyTorch on CPU.
10.4k · bundle
orchestra-research optimizing-attention-flash
Optimizes transformer attention with Flash Attention for 2-4x speedup and 10-20x memory reduction. Supports PyTorch native SDPA, flash-attn library, H100 FP8, and sliding window attention.
10.4k · bundle
More results
orchestra-research pytorch-fsdp2
Adds PyTorch FSDP2 (fully_shard) to training scripts with correct init, sharding, mixed precision/offload config, and distributed checkpointing. Use when models exceed single-GPU memory or when you need DTensor-based sharding with DeviceMesh.
10.4k · bundle
huggingface hf-mem
Estimates the memory required to load Safetensors or GGUF model weights for inference from the Hugging Face Hub, using HTTP Range requests without downloading weights.
10.8k
k-dense-ai optimize-for-gpu
GPU-accelerate Python code using CuPy, Numba CUDA, Warp, cuDF, cuML, cuGraph, and other RAPIDS libraries for dramatic speedups on numerical, data, ML, graph, and simulation workloads.
30.2k · bundle
lingxling hf-mem
Estimates memory requirements for running Hugging Face models, including optional KV cache, using HTTP range requests without downloading weights.
253
nvidia nemo-mbridge-perf-cpu-offloading
Configure and validate CPU offloading for Megatron Bridge training, including activation offloading and optimizer state offloading with HybridDeviceOptimizer.
2.2k · bundle
orchestra-research quantizing-models-bitsandbytes
Quantize LLMs to 8-bit or 4-bit for 50-75% memory reduction with minimal accuracy loss using bitsandbytes. Supports INT8, NF4, FP4 formats, QLoRA training, and 8-bit optimizers.
10.4k · bundle
orchestra-research gptq
Quantize large language models to 4-bit with minimal accuracy loss using GPTQ, enabling deployment of 70B+ models on consumer GPUs with 4× memory reduction and 3-4× faster inference.
10.4k · bundle
orchestra-research nemo-curator
GPU-accelerated data curation for LLM training, supporting text, image, video, and audio with fuzzy deduplication, quality filtering, semantic deduplication, PII redaction, and NSFW detection.
10.4k · bundle
lord1egypt modal-serverless-gpu
Run ML workloads on Modal's serverless GPU cloud: deploy models as auto-scaling APIs, run batch jobs, and schedule tasks with pay-per-second GPU pricing.
2
majiayu000 awq-quantization
Quantize large language models to 4-bit precision using activation-aware weight quantization, reducing memory footprint and speeding up inference with minimal accuracy loss.
567 · bundle
lord1egypt peft-fine-tuning
Fine-tune large language models by training less than 1% of parameters using LoRA, QLoRA, and 25+ adapter methods, enabling efficient adaptation on limited GPU memory.
2
affaan-m pytorch-patterns
Provides idiomatic PyTorch patterns and best practices for building robust, efficient, and reproducible deep learning applications, covering model architecture, training loops, data pipelines, and checkpointing.
226k
orchestra-research ml-training-recipes
Provides battle-tested PyTorch training recipes for LLMs, vision, diffusion, and biomedical domains, covering training loops, optimizer selection, LR scheduling, mixed precision, and debugging.
10.4k · bundle