Plugins
1 pluginResults for “batch-inference”
10 skills
orchestra-research Serving Llms Vllm
Deploy and serve LLMs with high throughput using vLLM's PagedAttention and continuous batching. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism for production inference.
10.4k · bundle
qhjqhj00 Ray Data
Process large ML datasets in parallel across CPU or GPU clusters, with streaming execution, multi-format I/O, and integration with Ray Train, PyTorch, and TensorFlow for batch inference and preprocessing pipelines.
3 · bundle
lord1egypt Modal Serverless Gpu
Run ML workloads on Modal's serverless GPU cloud: deploy models as auto-scaling APIs, run batch jobs, and schedule tasks with pay-per-second GPU pricing.
2
More results
orchestra-research Modal Serverless Gpu
Run ML workloads on Modal's serverless GPU cloud platform with auto-scaling, pay-per-second pricing, and Python-native infrastructure.
10.4k · bundle
samyakjhaveri Eval Run
Launches a model evaluation batch with parameter collection, pre-flight checks, execution, and post-run analysis for interactive or foreground runs.
0
k-dense-ai Modal
Deploy and serve AI/ML models on Modal's serverless cloud platform with on-demand GPUs, autoscaling containers, persistent storage, and scheduled jobs.
30.2k · bundle
qhjqhj00 Bbh Eval
Benchmarks zero-shot in-context learning on BIG-Bench Hard multiple-choice tasks, comparing self-generated demonstrations against direct prompting and chain-of-thought baselines, and reports accuracy.
3
k-dense-ai Scvi Tools
Provides deep generative models for single-cell omics analysis, including probabilistic batch correction, transfer learning, differential expression, and multi-modal integration.
30.2k · bundle
qhjqhj00 Runtime
Benchmarks inference latency and computational runtime of transformer models and MLX operations across Apple Silicon and NVIDIA GPU backends, with configurable input lengths and batch sizes.
3
huggingface Hf Cloud Sagemaker Deployment Planner
Plans and coordinates the deployment of a model to Amazon SageMaker AI, selecting the appropriate pathway (real-time, serverless, async, batch, or Bedrock CMI) based on model type, traffic, latency, and cost constraints.
10.8k