Compute Mamba Ratio

Compute the optimal --mamba-full-memory-ratio (or --max-mamba-cache-size pin) for a hybrid attention + linear-attention (Mamba / GDN / KDA) model's two serving memory pools, from the workload and serving config. Use when a user asks what ratio to set, why concurrency is clamped, or how to size the state vs KV pools for a hybrid model.

sgl-project Updated 38.9k repo stars
Organizations adopting SGLang

File contents

sgl-project/sglang commit 2134a2dff7

Frequently asked questions

npx get-skill add sgl-project/compute-mamba-ratio