Results for “quantization”
22 skills
majiayu000 awq-quantization
Quantize large language models to 4-bit precision using activation-aware weight quantization, reducing memory footprint and speeding up inference with minimal accuracy loss.
567 · bundle
orchestra-research awq-quantization
Quantize large language models to 4-bit using activation-aware weight quantization, achieving ~3x speedup with minimal accuracy loss for deployment on limited GPU memory.
10.4k · bundle
orchestra-research gguf-quantization
Convert and quantize models to GGUF format for efficient CPU/GPU inference with llama.cpp, supporting 2-8 bit quantization and Apple Silicon acceleration.
10.4k · bundle
orchestra-research hqq-quantization
Quantize large language models to 8/4/3/2/1-bit precision without calibration data, using multiple optimized backends for deployment with vLLM or HuggingFace Transformers.
10.4k · bundle
qhjqhj00 hqq-quantization
Quantize LLMs to 8/4/3/2/1-bit precision without calibration data, using multiple backends and HuggingFace/vLLM integration.
3 · bundle
oyi77 llm-deployment
Deploy and serve LLMs in production with vLLM, Ollama, TGI, and llama.cpp, including quantization and GPU optimization.
10
More results
majiayu000 hqq-quantization
Quantize large language models to 8/4/3/2/1-bit precision without calibration data, using multiple optimized backends and integrations with HuggingFace Transformers, vLLM, and PEFT/LoRA.
567 · bundle
huggingface huggingface-local-models
Search the Hugging Face Hub for llama.cpp-compatible GGUF models, select the right quantization, and run them locally with llama-cli or llama-server.
10.8k · bundle
orchestra-research serving-llms-vllm
Deploy and serve LLMs with high throughput using vLLM's PagedAttention and continuous batching. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism for production inference.
10.4k · bundle
orchestra-research llama-cpp
Run LLM inference on CPU, Apple Silicon, and consumer GPUs without NVIDIA hardware. Use for edge deployment, M1/M2/M3 Macs, AMD/Intel GPUs, or when CUDA is unavailable. Supports GGUF quantization (1.5-8 bit) for reduced memory and 4-10× speedup vs PyTorch on CPU.
10.4k · bundle
orchestra-research quantizing-models-bitsandbytes
Quantize LLMs to 8-bit or 4-bit for 50-75% memory reduction with minimal accuracy loss using bitsandbytes. Supports INT8, NF4, FP4 formats, QLoRA training, and 8-bit optimizers.
10.4k · bundle
orchestra-research tensorrt-llm
Optimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency on NVIDIA GPUs (A100/H100).
10.4k · bundle
nvidia tao-train-rtdetr
Train, evaluate, distill, quantize, export, and run inference for RT-DETR object detection models using NVIDIA TAO.
2.2k · bundle
orchestra-research gptq
Quantize large language models to 4-bit with minimal accuracy loss using GPTQ, enabling deployment of 70B+ models on consumer GPUs with 4× memory reduction and 3-4× faster inference.
10.4k · bundle
orchestra-research implementing-llms-litgpt
Train, fine-tune, and deploy LLMs using LitGPT's clean implementations of 20+ architectures like Llama, Gemma, and Phi.
10.4k · bundle
qhjqhj00 qutip
Simulate open and closed quantum systems with QuTiP, covering master equations, Lindblad dynamics, decoherence, and quantum optics.
3 · bundle
nvidia tao-train-image-classification
Train, evaluate, distill, quantize, export, and run inference for PyTorch-based TAO image classification models with support for multiple backbones.
2.2k · bundle
infinition llama-cpp
Run GGUF models locally with llama.cpp, including finding the right file on the Hugging Face Hub, installing, quantizing, serving, and using Python bindings.
2 · bundle
nvidia tao-train-dino
Train, evaluate, export, distill, quantize, or run inference for a TAO DINO 2D object detector using transformer-based detection with denoising training and multi-scale features.
2.2k · bundle
google-gemma gemma-dev
Selects the right Gemma model for a task, recommends deployment tooling (Gradio, Transformers.js, Vertex AI, MLX), and applies optimizations like MTP and QAT.
· bundle
nvidia tao-train-ocdnet
Trains, evaluates, exports, prunes, quantizes, retrains, and runs inference for OCDNet scene text detection models using TAO, detecting arbitrary-oriented text regions in natural images.
2.2k · bundle
nvidia tao-train-deformable-detr
Train, evaluate, export, quantize, and run inference for a Deformable DETR 2D object detection model using TAO, with deformable attention for efficient multi-scale feature processing.
2.2k · bundle