intelli-verse-x GPU Optimization Skill
Purpose
Maximize GPU utilization and minimize inference latency.
Techniques
- Quantization: FP32 → FP16/FP8/INT8 (2-4x speedup)
- Kernel Fusion: Combine operations to reduce memory traffic
- Continuous Batching: vLLM-style batching for LLMs
- Speculative Decoding: Draft model + verification
- KV Cache Optimization: PagedAttention, prefix caching
- Tensor Parallelism: Multi-GPU for large models
- Pipeline Parallelism: Overlap compute and communication
Profiling
- PyTorch Profiler: Kernel-level timing
- Nsight Systems: End-to-end trace
- DCGM: GPU-level metrics (utilization, memory, bandwidth)
Targets
| Metric | Target |
|---|---|
| GPU utilization | > 70% |
| Memory bandwidth | > 60% of theoretical |
| Tensor core usage | > 50% |
| Inference latency | Within SLO budget |
CF-Specific
- Use vLLM for self-hosted LLM serving
- Warm GPU pools for cold start elimination
- MIG for sharing H100s across services
- Spot instances for batch workloads
- Monitor with DCGM + Prometheus