Results for “gpu-memory”
51 skills
nvidia nemo-mbridge-perf-memory-tuning
Reduces peak GPU memory in Megatron Bridge training by applying expandable segments, parallelism resizing, activation recompute, and CPU offloading constraints.
2.2k · bundle
matlab matlab-optimize-gpu-codegen
Optimize MATLAB design files for GPU Coder to generate faster CUDA code. Iteratively profiles, rewrites, and benchmarks until performance targets are met or diagnostics are resolved. Use when asked to: optimize for GPU Coder, improve GPU codegen performance, profile generated GPU/CUDA code, profile GPU MEX, fix gpuPerformanceAnalyzer diagnostics, speed up GPU MEX, reduce GPU memory transfers, improve kernel parallelism, rewrite MATLAB for CUDA, or run gpuPerformanceAnalyzer.
920 · bundle
orchestra-research awq-quantization
Quantize large language models to 4-bit using activation-aware weight quantization, achieving ~3x speedup with minimal accuracy loss for deployment on limited GPU memory.
10.4k · bundle
antigravity hf-mem
Estimates GPU memory required to load Safetensors or GGUF model weights for inference from the Hugging Face Hub using HTTP Range requests, without downloading weights locally.
42.4k
samyakjhaveri cuda-omp-translator
Reference guide for evaluating LLM-generated translations between CUDA and OpenMP, covering memory model mapping, kernel launch patterns, shared memory, atomics, and common failure modes.
0
orchestra-research llama-cpp
Run LLM inference on CPU, Apple Silicon, and consumer GPUs without NVIDIA hardware. Use for edge deployment, M1/M2/M3 Macs, AMD/Intel GPUs, or when CUDA is unavailable. Supports GGUF quantization (1.5-8 bit) for reduced memory and 4-10× speedup vs PyTorch on CPU.
10.4k · bundle
More results
orchestra-research optimizing-attention-flash
Optimizes transformer attention with Flash Attention for 2-4x speedup and 10-20x memory reduction. Supports PyTorch native SDPA, flash-attn library, H100 FP8, and sliding window attention.
10.4k · bundle
qcmuu llama-cpp
Runs LLM inference on CPU, Apple Silicon, and consumer GPUs without NVIDIA hardware. Use for edge deployment, M1/M2/M3 Macs, AMD/Intel GPUs, or when CUDA is unavailable. Supports GGUF quantization (1.5-8 bit) for reduced memory and 4-10× speedup vs PyTorch on CPU.
0 · bundle
tianhao909 llama-cpp
Runs LLM inference on CPU, Apple Silicon, and consumer GPUs without NVIDIA hardware. Use for edge deployment, M1/M2/M3 Macs, AMD/Intel GPUs, or when CUDA is unavailable. Supports GGUF quantization (1.5-8 bit) for reduced memory and 4-10× speedup vs PyTorch on CPU.
1 · bundle
ichichuang llama-cpp
Runs LLM inference on CPU, Apple Silicon, and consumer GPUs without NVIDIA hardware. Use for edge deployment, M1/M2/M3 Macs, AMD/Intel GPUs, or when CUDA is unavailable. Supports GGUF quantization (1.5-8 bit) for reduced memory and 4-10× speedup vs PyTorch on CPU.
0 · bundle
orchestra-research pytorch-fsdp2
Adds PyTorch FSDP2 (fully_shard) to training scripts with correct init, sharding, mixed precision/offload config, and distributed checkpointing. Use when models exceed single-GPU memory or when you need DTensor-based sharding with DeviceMesh.
10.4k · bundle
orchestra-research quantizing-models-bitsandbytes
Quantize LLMs to 8-bit or 4-bit for 50-75% memory reduction with minimal accuracy loss using bitsandbytes. Supports INT8, NF4, FP4 formats, QLoRA training, and 8-bit optimizers.
10.4k · bundle
tianhao909 quantizing-models-bitsandbytes
Quantizes LLMs to 8-bit or 4-bit for 50-75% memory reduction with minimal accuracy loss. Use when GPU memory is limited, need to fit larger models, or want faster inference. Supports INT8, NF4, FP4 formats, QLoRA training, and 8-bit optimizers. Works with HuggingFace Transformers.
1 · bundle
qcmuu quantizing-models-bitsandbytes
Quantizes LLMs to 8-bit or 4-bit for 50-75% memory reduction with minimal accuracy loss. Use when GPU memory is limited, need to fit larger models, or want faster inference. Supports INT8, NF4, FP4 formats, QLoRA training, and 8-bit optimizers. Works with HuggingFace Transformers.
0 · bundle
tianhao909 optimizing-attention-flash
Optimizes transformer attention with Flash Attention for 2-4x speedup and 10-20x memory reduction. Use when training/running transformers with long sequences (>512 tokens), encountering GPU memory issues with attention, or need faster inference. Supports PyTorch native SDPA, flash-attn library, H100 FP8, and sliding window attention.
1 · bundle
qcmuu optimizing-attention-flash
Optimizes transformer attention with Flash Attention for 2-4x speedup and 10-20x memory reduction. Use when training/running transformers with long sequences (>512 tokens), encountering GPU memory issues with attention, or need faster inference. Supports PyTorch native SDPA, flash-attn library, H100 FP8, and sliding window attention.
0 · bundle
majiayu000 awq-quantization
Quantize large language models to 4-bit precision using activation-aware weight quantization, reducing memory footprint and speeding up inference with minimal accuracy loss.
567 · bundle
affaan-m pytorch-patterns
Provides idiomatic PyTorch patterns and best practices for building robust, efficient, and reproducible deep learning applications, covering model architecture, training loops, data pipelines, and checkpointing.
226k
lord1egypt peft-fine-tuning
Fine-tune large language models by training less than 1% of parameters using LoRA, QLoRA, and 25+ adapter methods, enabling efficient adaptation on limited GPU memory.
2
google gke-inference
Deploys and optimizes AI/ML inference workloads on GKE, using GPUs, TPUs, and model servers.
14.4k
huggingface hf-mem
Estimates the memory required to load Safetensors or GGUF model weights for inference from the Hugging Face Hub, using HTTP Range requests without downloading weights.
10.8k
orchestra-research gptq
Quantize large language models to 4-bit with minimal accuracy loss using GPTQ, enabling deployment of 70B+ models on consumer GPUs with 4× memory reduction and 3-4× faster inference.
10.4k · bundle
k-dense-ai optimize-for-gpu
GPU-accelerate Python code using CuPy, Numba CUDA, Warp, cuDF, cuML, cuGraph, and other RAPIDS libraries for dramatic speedups on numerical, data, ML, graph, and simulation workloads.
30.2k · bundle
lingxling hf-mem
Estimates memory requirements for running Hugging Face models, including optional KV cache, using HTTP range requests without downloading weights.
253
francostino hf-mem
Hugging Face CLI to estimate the required memory to load Safetensors or GGUF model weights for inference from the Hugging Face Hub
63
ranbot-ai hf-mem
Hugging Face CLI to estimate the required memory to load Safetensors or GGUF model weights for inference from the Hugging Face Hub
6
huggingface huggingface-zerogpu
Build ML demos on Hugging Face Spaces with ZeroGPU hardware, covering @spaces.GPU decorator usage, duration and quota tuning, process isolation, CUDA availability model, concurrency safety, and build constraints.
10.8k · bundle
nvidia nemo-mbridge-perf-cpu-offloading
Configure and validate CPU offloading for Megatron Bridge training, including activation offloading and optimizer state offloading with HybridDeviceOptimizer.
2.2k · bundle
sickn33 hf-mem
Hugging Face CLI to estimate the required memory to load Safetensors or GGUF model weights for inference from the Hugging Face Hub
45.1k
intelli-verse-x ivx-cf-person-gpu
GPU / MLOps person pack for Content Factory. Use when the user says person gpu, @person-gpu, GPU person, RunPod person, or MLOps person. Auto-loads gpu-infrastructure-engineer and mlops-engineer plus gpu-optimization, cf-llm-model-usage, cost-optimization.
0 · bundle
muratcankoylan memory-systems
Designs persistent memory architectures for AI agents, covering cross-session knowledge retention, entity tracking, temporal validity, graph/vector retrieval, and memory consolidation.
16.9k · bundle
26bb hf-mem
Hugging Face CLI to estimate the required memory to load Safetensors or GGUF model weights for inference from the Hugging Face Hub
0
qcmuu nemo-curator
GPU-accelerated data curation for LLM training. Supports text/image/video/audio. Features fuzzy deduplication (16× faster), quality filtering (30+ heuristics), semantic deduplication, PII redaction, NSFW detection. Scales across GPUs with RAPIDS. Use for preparing high-quality training datasets, cleaning web data, or deduplicating large corpora.
0 · bundle
orchestra-research nemo-curator
GPU-accelerated data curation for LLM training, supporting text, image, video, and audio with fuzzy deduplication, quality filtering, semantic deduplication, PII redaction, and NSFW detection.
10.4k · bundle
lord1egypt modal-serverless-gpu
Run ML workloads on Modal's serverless GPU cloud: deploy models as auto-scaling APIs, run batch jobs, and schedule tasks with pay-per-second GPU pricing.
2
tianhao909 gptq
Post-training 4-bit quantization for LLMs with minimal accuracy loss. Use for deploying large models (70B, 405B) on consumer GPUs, when you need 4× memory reduction with <2% perplexity degradation, or for faster inference (3-4× speedup) vs FP16. Integrates with transformers and PEFT for QLoRA fine-tuning.
1 · bundle