Results for “continuous-batching”
14 skills
orchestra-research serving-llms-vllm
Deploy and serve LLMs with high throughput using vLLM's PagedAttention and continuous batching. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism for production inference.
10.4k · bundle
q2805187159 serving-llms-vllm
Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.
3 · bundle
More results
tianhao909 serving-llms-vllm
Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.
1 · bundle
inference-sh ai-automation-workflows
Build automated AI workflows combining multiple models and services for batch processing, scheduled tasks, event-driven pipelines, and agent loops using the inference.sh CLI.
584
muratcankoylan latent-briefing
Shares memory between agents at the representation level by compacting the orchestrator's KV cache for efficient worker handoff, reducing token costs without summarization or retrieval.
16.9k · bundle
a5c-ai agent-dispatch
Agent Dispatch
1.7k · bundle
microsoft continual-learning
Implements a continual learning loop for AI coding agents using hooks, two-tier memory (global and local), and automatic pattern detection to persist and apply learnings across sessions.
2.7k
lucassantana-dev rag
Build and debug Retrieval-Augmented Generation pipelines — chunking, embedding, retrieval, reranking
1 · bundle
lucassantana-dev debug-deep
Composite skill — full debugging workflow from "this is broken" to root cause and fix. Chains systematic-debugging (root-cause hypotheses) → tracer agent (evidence walk) → sentry (production correlation if applicable) → ci-watch (regression check) → incident-response (if production-impacting). Use when a bug needs deep investigation, not just a quick fix.
1 · bundle
samyakjhaveri eval-run
Launches a model evaluation batch with parameter collection, pre-flight checks, execution, and post-run analysis for interactive or foreground runs.
0
fradser executing-plans
Executes a written implementation plan via per-batch sub-agent coordinators with verification and commits between batches.
580 · bundle
affaan-m continuous-agent-loop
Provides patterns for continuous autonomous agent loops with quality gates, evals, and recovery controls.
226k
qcmuu serving-llms-vllm
Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.
0 · bundle
ichichuang serving-llms-vllm
Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.
0 · bundle