Results for “gfm”
6 skills
orchestra-research Gguf Quantization
Convert and quantize models to GGUF format for efficient CPU/GPU inference with llama.cpp, supporting 2-8 bit quantization and Apple Silicon acceleration.
10.4k · bundle
huggingface Hf Mem
Estimates the memory required to load Safetensors or GGUF model weights for inference from the Hugging Face Hub, using HTTP Range requests without downloading weights.
10.8k
antigravity Hf Mem
Estimates GPU memory required to load Safetensors or GGUF model weights for inference from the Hugging Face Hub using HTTP Range requests, without downloading weights locally.
42.4k
orchestra-research Gptq
Quantize large language models to 4-bit with minimal accuracy loss using GPTQ, enabling deployment of 70B+ models on consumer GPUs with 4× memory reduction and 3-4× faster inference.
10.4k · bundle
lingxling Hf Mem
Estimates memory requirements for running Hugging Face models, including optional KV cache, using HTTP range requests without downloading weights.
253
infinition Llama Cpp
Run GGUF models locally with llama.cpp, including finding the right file on the Hugging Face Hub, installing, quantizing, serving, and using Python bindings.
2 · bundle