Results for “inference-latency”
9 skills
qhjqhj00 flops
Evaluates computational throughput and real-time efficiency of embedded CPU and GPU platforms by measuring peak FLOPS via a matrix rotation kernel and assessing inference latency and power consumption on a robotic vision pipeline.
3
qhjqhj00 latency
Measures inference latency of binarized, 8-bit, and 32-bit convolutional layers on edge devices to evaluate the efficiency and speedup of the Larq Compute Engine framework compared to standard implementations.
3
orchestra-research tensorrt-llm
Optimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency on NVIDIA GPUs (A100/H100).
10.4k · bundle
More results
qhjqhj00 runtime
Benchmarks inference latency and computational runtime of transformer models and MLX operations across Apple Silicon and NVIDIA GPU backends, with configurable input lengths and batch sizes.
3
orchestra-research speculative-decoding
Accelerate LLM inference using speculative decoding, Medusa multiple heads, and lookahead decoding techniques for 1.5-3.6× speedup without quality loss.
10.4k · bundle
qhjqhj00 ndcg-10
Evaluates how well internal model representations (hidden states) predict token-level information importance in summarization tasks, using NDCG@10 and Spearman's rank correlation.
3
nvidia jetson-speculative-decoding
Reduce per-token latency on Jetson vLLM servers by appending speculative decoding configuration, with guidance on when to enable and how to benchmark the improvement.
2.2k · bundle
nvidia tao-train-fast-foundation-stereo
Trains, evaluates, exports, and runs inference for FastFoundationStereo (FFS) stereo depth estimation models, a distilled variant of FoundationStereo with lower latency.
2.2k · bundle
huggingface hf-cloud-sagemaker-deployment-planner
Plans and coordinates the deployment of a model to Amazon SageMaker AI, selecting the appropriate pathway (real-time, serverless, async, batch, or Bedrock CMI) based on model type, traffic, latency, and cost constraints.
10.8k