Inference Spec Decode Service

Coordinate a measured speculative decoding workflow for a served model. Profile representative traffic, build a matched corpus, run an operator-provided draft trainer, compare the draft with the standing config, and promote only a measured win. Use for profile-matched speculative decoding work that needs one evidence trail and fail-closed gates.

cfregly Updated 4 repo stars

File contents

cfregly/claude-gpu-perf-tune commit b8ba90a4df

Frequently asked questions

npx get-skill add cfregly/inference-spec-decode-service