Heteroscale Autoscaling

Scale disaggregated LLM inference (prefill-decode) via topology-aware scheduling and metric-driven policies, achieving 26.6% GPU utilization improvement and conserving hundreds of thousands GPU-hours daily

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit 64babdedb9

Frequently asked questions

npx get-skill add adu2021/heteroscale-autoscaling