Results for “cpu-inference”
6 skills
orchestra-research Gguf Quantization
Convert and quantize models to GGUF format for efficient CPU/GPU inference with llama.cpp, supporting 2-8 bit quantization and Apple Silicon acceleration.
10.4k · bundle
orchestra-research Llama Cpp
Run LLM inference on CPU, Apple Silicon, and consumer GPUs without NVIDIA hardware. Use for edge deployment, M1/M2/M3 Macs, AMD/Intel GPUs, or when CUDA is unavailable. Supports GGUF quantization (1.5-8 bit) for reduced memory and 4-10× speedup vs PyTorch on CPU.
10.4k · bundle
More results
nvidia Jetson Inference Mem Tune
Recommends an inference runtime and memory-related launch flags for LLM/VLM workloads on NVIDIA Jetson devices, based on a live memory audit snapshot.
2.2k · bundle
qhjqhj00 Flops
Evaluates computational throughput and real-time efficiency of embedded CPU and GPU platforms by measuring peak FLOPS via a matrix rotation kernel and assessing inference latency and power consumption on a robotic vision pipeline.
3
orchestra-research Lambda Labs Gpu Cloud
Manage and use Lambda Labs GPU cloud instances for ML training and inference with SSH access, persistent filesystems, and multi-node clusters.
10.4k · bundle
qhjqhj00 Ray Data
Process large ML datasets in parallel across CPU or GPU clusters, with streaming execution, multi-format I/O, and integration with Ray Train, PyTorch, and TensorFlow for batch inference and preprocessing pipelines.
3 · bundle