Results for “json-decoding”
7 skills
orchestra-research sglang
Serve LLMs and VLMs with structured outputs, prefix caching, and high throughput using RadixAttention.
10.4k · bundle
tianhao909 sglang
Fast structured generation and serving for LLMs with RadixAttention prefix caching. Use for JSON/regex outputs, constrained decoding, agentic workflows with tool calls, or when you need 5× faster inference than vLLM with prefix sharing. Powers 300,000+ GPUs at xAI, AMD, NVIDIA, and LinkedIn.
1 · bundle
More results
nvidia jetson-speculative-decoding
Reduce per-token latency on Jetson vLLM servers by appending speculative decoding configuration, with guidance on when to enable and how to benchmark the improvement.
2.2k · bundle
jrennie99-glitch stream-chain
Stream-JSON chaining for multi-agent pipelines, data transformation, and sequential workflows
0
yanacuti1121 headroom
Context compression for YAMTAM — nén JSON/structured tool output trước khi vào LLM. Hiệu quả với JSON (50-72% tiết kiệm); text thuần cần bản [all].
2
qcmuu sglang
Fast structured generation and serving for LLMs with RadixAttention prefix caching. Use for JSON/regex outputs, constrained decoding, agentic workflows with tool calls, or when you need 5× faster inference than vLLM with prefix sharing. Powers 300,000+ GPUs at xAI, AMD, NVIDIA, and LinkedIn.
0 · bundle
jackychenlu sglang
Fast structured generation and serving for LLMs with RadixAttention prefix caching. Use for JSON/regex outputs, constrained decoding, agentic workflows with tool calls, or when you need 5× faster inference than vLLM with prefix sharing. Powers 300,000+ GPUs at xAI, AMD, NVIDIA, and LinkedIn.
0 · bundle