sgl-project Speculative Decoding — Naming Conventions
Apply this skill when adding, renaming, or reviewing identifiers in speculative decoding code (anything under python/sglang/srt/speculative/, related attention backends, scheduler accumulators, IPC fields, observability metrics, or CLI flags).
Rule 1 — Verb form, drop -ed
Use the verb form accept everywhere. Don't use the past-participle form accepted.
| Don't | Do |
|---|---|
num_accepted_tokens |
num_accept_tokens |
accepted_indices |
accept_indices |
accepted_token_ids |
accept_tokens (also see Rule 3) |
Rule 2 — The extra/bonus token is bonus_token / bonus_tokens
The "+1" token that the target model always emits in addition to verifying drafts is the bonus token. Use bonus_token / bonus_tokens per Rule 7.
| Don't | Do |
|---|---|
verified_id / verified_ids |
bonus_token / bonus_tokens |
output_id / output_ids (when referring to the bonus) |
bonus_token / bonus_tokens |
req.output_ids (the full output history of a request) is unrelated and stays as is.
Rule 3 — accept includes bonus; correct excludes bonus
The semantic distinction lives in the verb, not the noun. Don't enumerate noun pairs.
| Verb | Meaning |
|---|---|
accept_* |
Includes the bonus token |
correct_* |
Drafts only, no bonus |
Pair with whatever noun fits the data (tokens, drafts, indices, …). No required pairing, but preferred default nouns: accept_tokens and correct_drafts — correct semantically describes drafts (what got verified), accept describes the resulting token sequence (incl. bonus).
| Form | Meaning |
|---|---|
accept_tokens / accept_indices |
Include bonus |
correct_drafts |
Drafts only, no bonus |
num_accept_tokens |
Count incl. bonus |
num_correct_drafts |
Count excl. bonus |
Exception: accept_rate / accept_length follow paper convention
These two metric names are entrenched in the spec-decoding literature and in external-facing fields (meta_info, Prometheus). Their semantics are paper-defined, not Rule-3-defined:
| Name | Paper term | Bonus? | Definition |
|---|---|---|---|
accept_rate |
$\alpha$ (Leviathan 2023) | No | per-draft-token acceptance probability = correct_drafts / proposed_drafts |
accept_length |
$\tau$ (EAGLE) | Yes | avg tokens per verify step = completion_tokens / verify_ct |
Internal counters still follow Rule 3 strict semantics: num_correct_drafts (no bonus), num_accept_tokens (with bonus).
Rule 4 — num_ for counts; _ct for counters; _rate for rates; no prefix for IDs
Each form has its own marker. Never mix (no num_X_ct, no num_accept_rate).
| Form | Pattern | Meaning | Examples |
|---|---|---|---|
| Count | num_X |
Snapshot quantity at one point in time (often a tensor or scalar) | num_accept_tokens, num_correct_drafts, num_proposed_drafts |
| Counter | X_ct |
Monotonically incrementing accumulator over time | spec_verify_ct, forward_ct |
| Rate / ratio | X_rate |
Fractional value in [0, 1] |
accept_rate |
| Tokens / content array | no prefix | The actual token data, not a count | accept_tokens, correct_drafts, bonus_token |
Rule 5 — Drop redundant _token_id / _token_ids suffix in spec scope
_id / _ids and _token / _tokens are both fine. But don't combine — _token_id / _token_ids is redundant inside spec decoding, because spec code only ever deals with vocab integers.
The semantic differs by scope:
| Scope | Example | What _token_id means |
|---|---|---|
| Framework / multimodal / tokenizer | image_token_id, pad_token_id, eos_token_id, mask_token_id, bos_token_id |
A specific named/role token's vocab ID. The prefix names the role; _token_id says it's the integer ID for that role. Both halves carry information. |
| Spec decoding | accepted_token_ids, curr_token_id, out_token_ids |
Redundant. Spec only deals with vocab integers; _id adds nothing beyond _token. |
Renames
| Don't | Do |
|---|---|
accepted_token_ids |
accept_tokens (Rule 1 + 3) |
curr_token_id |
current_token |
out_token_ids |
out_tokens |
_resolve_spec_overlap_token_ids |
_resolve_spec_overlap_tokens |
Rule 6 — Singular vs plural
Plural for any non-scalar tensor ([bs]-shaped, flat, or multi-dim); singular only for scalars (kernel tl.load results, single-int locals). Applies to all spec-decoding tensors (tokens, indices, etc.).
accept_tokens: torch.Tensor # [total_accepted] flat - plural
accept_indices: torch.Tensor # [bs, num_draft_tokens] - plural
draft_tokens: torch.Tensor # [bs * num_draft_tokens] flat - plural
bonus_tokens: torch.Tensor # [bs] - plural
accept_token = tl.load(...) # int32 scalar in a kernel iteration - singular
bonus_token = tl.load(...) # int32 scalar inside a kernel - singular
Out of scope (these names stay as is)
These rules apply to spec-decoding-specific identifiers. Pre-existing or framework-level names are kept.
- PyTorch / ecosystem:
seq_lens,extend_seq_lens,cu_seqlens_q - Framework / multimodal vocab:
image_token_id,pad_token_id,eos_token_id,mask_token_id,hot_token_id,bos_token_id,topk_id - Request-level state:
req.input_ids,req.output_ids,req.origin_input_ids,next_token_ids(model_runner.sampleoutput) - Frozen C++ kwargs:
accept_token_num(sgl-kernel) - Non-token IDs:
req_id,gpu_id,layer_id,program_id _len/_lensnames:num_Xis preferred for counts (Rule 4), but_len/_lensnames are acceptable. Triton kernel params in particular often use_lens/_lento align with the PyTorch ecosystem (seq_lens,cu_seqlens_q). Rule 1 still requires the-ed-less form (accept_lengthOK,accepted_lengthnot).