SkillMD SkillMD
light-dark-mode
Profile My skills Saved Collections Edit profile Submit a skill

Results for “llm-compression”

6 skills
orchestra-research
Knowledge Distillation
Compress large language models using knowledge distillation from teacher to student models, covering temperature scaling, soft targets, reverse KLD, logit distillation, and MiniLLM training strategies.
10.4k · bundle
More results
orchestra-research
Quantizing Models Bitsandbytes
Quantize LLMs to 8-bit or 4-bit for 50-75% memory reduction with minimal accuracy loss using bitsandbytes. Supports INT8, NF4, FP4 formats, QLoRA training, and 8-bit optimizers.
10.4k · bundle
orchestra-research
Tensorrt LLM
Optimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency on NVIDIA GPUs (A100/H100).
10.4k · bundle
orchestra-research
Llamaguard
Deploy Meta's LlamaGuard moderation model to filter LLM inputs and outputs across 6 safety categories using HuggingFace, vLLM, or FastAPI.
10.4k
orchestra-research
Implementing Llms Litgpt
Train, fine-tune, and deploy LLMs using LitGPT's clean implementations of 20+ architectures like Llama, Gemma, and Phi.
10.4k · bundle
orchestra-research
Model Pruning
Compress large language models by 40-60% with minimal accuracy loss using one-shot pruning techniques like Wanda and SparseGPT, enabling faster inference and deployment on constrained hardware.
10.4k · bundle