Code Style for Scheduler / TokenizerManager / ModelRunner
Conventions for SGLang's three large classes:
Scheduler — python/sglang/srt/managers/scheduler.py
TokenizerManager — python/sglang/srt/managers/tokenizer_manager.py
ModelRunner — python/sglang/srt/model_executor/model_runner.py
1. Frozen Code
- Some core files are frozen: orchestration-only — a thin composition root that constructs collaborators, wires them, delegates to them, and coordinates the calls. They must stay that way.
- Domain logic does not belong in a frozen file; it lives in a collaborator class in its own module.
1.1 Why
- The file is a thin orchestrator over collaborator classes; freezing keeps it that way and stops it growing back into a god class.
- Keeping domain logic in collaborators (their own files) is what makes per-file code ownership, single responsibility, and unit testing possible.
- The orchestrator is the composition root: it may know about every collaborator, because wiring and sequencing them is its job. Coordination stays here — domain logic does not.
1.2 Frozen files
python/sglang/srt/model_executor/model_runner.py
1.3 Allowed: orchestration
Every statement refers to a collaborator and is one of:
- Construct — a short
init_<thing> helper whose body is essentially a single construction (follows §2); use maybe_init_<thing> with a one-line gate when conditional.
- Wire — a short call that runs the helper from the orchestrator (e.g. in
__init__).
- Delegate — calls to a collaborator's methods at the necessary call sites (
self.foo.run(...)).
- Coordinate — the minimal control flow that selects or orders the above: an
if choosing whether / which collaborator to wire or call, the order of calls, threading one call's result into the next.
- Heuristic: a statement is allowed only if it constructs, wires, delegates, or selects/orders those — never if it computes or transforms a value beyond passing arguments and results through.
# model_runner.py — orchestration only.
def init_foo(self): # construct
self.foo = FooManager(server_args=self.server_args, device=self.device)
self.init_foo() # wire (in __init__)
if self.server_args.enable_bar: # coordinate: select
self.bar.prepare(forward_batch) # delegate
out = self.foo.run(forward_batch) # delegate
self.baz.consume(out) # coordinate: thread result into next delegate
1.4 Not allowed: domain logic
- Config building, data transformation, algorithm bodies, math, post-processing — any branch or loop that computes rather than coordinates.
- It belongs in the collaborator.
# NOT allowed in a frozen file: domain logic inlined.
self.foo = None
if self.server_args.enable_foo:
config = build_foo_config(self.model_config, self.device) # config logic in frozen file
self.foo = FooManager(config) # inline construction, not via (maybe_)init_foo
out = [step(x) for x in batch] # computation, not coordination
- Fix: move that body into
FooManager (its __init__ or a factory) plus a (maybe_)init_foo helper.
1.5 Where coordination logic goes
- Default: extract. Pull cohesive coordination into a low-coupling collaborator (an initializer, a forward pipeline) and delegate to it.
- Residue stays. Coordination that can't be cohesively extracted may remain — but only the minimal Coordinate form above, kept pseudocode-readable. This is the explicit exception, not a fallback; note why it stays.
- When the residue outgrows pseudocode, that is the signal to extract a dedicated coordinator — not to keep inlining.
1.6 Pass what the collaborator needs, not the god object
When you extract domain logic into a collaborator (a factory, an initializer, a pipeline), give it the specific values it needs — model_config, device, the sizes — not the whole frozen object (ModelRunner, Scheduler).
Passing the god object back re-creates the coupling the split was meant to remove: the module still reads dozens of attributes off it, can't be unit-tested without building the whole class, and every field rename ripples back in.
Default to narrow, keyword args. Reference shape: layer_setup.resolve_layer_indices(*, model, model_config, is_draft_worker, spec_algorithm).
Return a small frozen struct and let the orchestrator assign it onto its own fields. The collaborator should not reach back in and mutate the god object.
If a leaf genuinely needs the live object — its constructor contract already takes the runner, or it reads state that mutates after init — confine that dependency to the smallest leaf and pass narrow args everywhere above it. Note why it can't be narrowed.
1.7 If you do pass the god object, keep it read-only
- A callee that genuinely takes the live object should read fields off it and return results; it writes fields back only when there is genuinely no other way.
- The orchestrator owns the assignment onto its own fields.
- Why: a callee that mutates the god object scatters its writes across other modules — you can no longer see what
ModelRunner owns by reading model_runner.py, the hidden writes race with the orchestrator's own ordering, and the callee silently depends on being invoked at exactly the right moment.
# Good — callee reads the runner and returns a small frozen struct; the orchestrator
# owns the writes.
# model_runner.py
class ModelRunner:
def bar(self):
self.foo_result = foo(self)
# another_file.py
def foo(model_runner) -> FooResult:
return FooResult(a=xx, b=yy, c=zz)
# Avoid — callee reaches back in and writes the runner's fields.
# model_runner.py
class ModelRunner:
def bar(self):
foo(self)
# another_file.py
def foo(model_runner):
model_runner.a = xx
model_runner.b = yy
model_runner.c = zz
2. __init__ style
Apply when modifying the __init__ of the three classes above.
2.1 Why
- Downstream forks override one piece (tokenizer, KV cache, IPC, …).
- Inline logic forces them to copy the whole
__init__, which rots against upstream.
- Splitting into
init_* helpers lets them override exactly what they need.
- Reference shape:
TokenizerManager.__init__ in python/sglang/srt/managers/tokenizer_manager.py.
2.2 Rules
__init__ is an orchestrator. Sequence of self.init_*(...) calls + minimal glue. No non-trivial construction inlined.
- One helper per overridable unit. Each
init_* = one concern a subclass might swap. Don't lump.
- Naming:
init_<thing> (snake_case, names the component). Conditional construction → maybe_init_<thing>, gate inside the helper.
- No silent state coupling. A helper only reads
self.* set by earlier helpers. Ordering lives in __init__. Shared intermediates → pass as args, not via self.*.
- New logic = new helper. Default to adding
init_<thing>, not another inline block. One-line self.foo = server_args.foo is fine; structured logic is not.
- Preserve override points. Prefer additive changes to existing
init_* signatures. Breaking changes → call out in PR.
2.3 Scope
- Only the three classes listed above.
- Not other manager-style classes, not small dataclass/utility constructors.
1---
2name: large-class-style
3description: 'Code style for SGLang large classes `Scheduler`, `TokenizerManager`, and `ModelRunner`: frozen-code conventions and `__init__` orchestration style. Use when modifying any of these three classes or reviewing changes to them.'
4metadata:
5 source: https://skillmd.com/skills/sgl-project/large-class-style
6---
7
8# Code Style for Scheduler / TokenizerManager / ModelRunner
9
10Conventions for SGLang's three large classes:
11
12- `Scheduler` — `python/sglang/srt/managers/scheduler.py`
13- `TokenizerManager` — `python/sglang/srt/managers/tokenizer_manager.py`
14- `ModelRunner` — `python/sglang/srt/model_executor/model_runner.py`
15
16## 1. Frozen Code
17
18- Some core files are **frozen**: *orchestration-only* — a thin composition root that constructs collaborators, wires them, delegates to them, and coordinates the calls. They must stay that way.
19- **Domain logic does not belong in a frozen file**; it lives in a collaborator class in its own module.
20
21### 1.1 Why
22
23- The file is a thin orchestrator over collaborator classes; freezing keeps it that way and stops it growing back into a god class.
24- Keeping domain logic in collaborators (their own files) is what makes per-file code ownership, single responsibility, and unit testing possible.
25- The orchestrator is the composition root: it may know about every collaborator, because wiring and sequencing them is its job. Coordination stays here — domain logic does not.
26
27### 1.2 Frozen files
28
29- `python/sglang/srt/model_executor/model_runner.py`
30
31### 1.3 Allowed: orchestration
32
33Every statement refers to a collaborator and is one of:
34
351. **Construct** — a short `init_<thing>` helper whose body is essentially a single construction (follows §2); use `maybe_init_<thing>` with a one-line gate when conditional.
362. **Wire** — a short call that runs the helper from the orchestrator (e.g. in `__init__`).
373. **Delegate** — calls to a collaborator's methods at the necessary call sites (`self.foo.run(...)`).
384. **Coordinate** — the minimal control flow that *selects or orders* the above: an `if` choosing whether / which collaborator to wire or call, the order of calls, threading one call's result into the next.
39
40- Heuristic: a statement is allowed only if it constructs, wires, delegates, or selects/orders those — never if it *computes or transforms* a value beyond passing arguments and results through.
41
42```python
43# model_runner.py — orchestration only.
44def init_foo(self): # construct
45 self.foo = FooManager(server_args=self.server_args, device=self.device)
46
47self.init_foo() # wire (in __init__)
48
49if self.server_args.enable_bar: # coordinate: select
50 self.bar.prepare(forward_batch) # delegate
51out = self.foo.run(forward_batch) # delegate
52self.baz.consume(out) # coordinate: thread result into next delegate
53```
54
55### 1.4 Not allowed: domain logic
56
57- Config building, data transformation, algorithm bodies, math, post-processing — any branch or loop that *computes* rather than *coordinates*.
58- It belongs in the collaborator.
59
60```python
61# NOT allowed in a frozen file: domain logic inlined.
62self.foo = None
63if self.server_args.enable_foo:
64 config = build_foo_config(self.model_config, self.device) # config logic in frozen file
65 self.foo = FooManager(config) # inline construction, not via (maybe_)init_foo
66 out = [step(x) for x in batch] # computation, not coordination
67```
68
69- Fix: move that body into `FooManager` (its `__init__` or a factory) plus a `(maybe_)init_foo` helper.
70
71### 1.5 Where coordination logic goes
72
731. **Default: extract.** Pull cohesive coordination into a low-coupling collaborator (an initializer, a forward pipeline) and delegate to it.
742. **Residue stays.** Coordination that can't be cohesively extracted may remain — but only the minimal **Coordinate** form above, kept pseudocode-readable. This is the explicit exception, not a fallback; note why it stays.
75
76- When the residue outgrows pseudocode, that is the signal to extract a dedicated coordinator — not to keep inlining.
77
78### 1.6 Pass what the collaborator needs, not the god object
79
80- When you extract domain logic into a collaborator (a factory, an initializer, a pipeline), give it the **specific values** it needs — `model_config`, `device`, the sizes — not the whole frozen object (`ModelRunner`, `Scheduler`).
81- Passing the god object back re-creates the coupling the split was meant to remove: the module still reads dozens of attributes off it, can't be unit-tested without building the whole class, and every field rename ripples back in.
82
83- Default to **narrow, keyword args**. Reference shape: `layer_setup.resolve_layer_indices(*, model, model_config, is_draft_worker, spec_algorithm)`.
84- Return a small **frozen struct** and let the orchestrator assign it onto its own fields. The collaborator should not reach back in and mutate the god object.
85- If a leaf genuinely needs the live object — its constructor contract already takes the runner, or it reads state that mutates after init — confine that dependency to the **smallest leaf** and pass narrow args everywhere above it. Note why it can't be narrowed.
86
87### 1.7 If you do pass the god object, keep it read-only
88
89- A callee that genuinely takes the live object should **read** fields off it and **return** results; it writes fields back only when there is genuinely no other way.
90- The orchestrator owns the assignment onto its own fields.
91- Why: a callee that mutates the god object scatters its writes across other modules — you can no longer see what `ModelRunner` owns by reading `model_runner.py`, the hidden writes race with the orchestrator's own ordering, and the callee silently depends on being invoked at exactly the right moment.
92
93```python
94# Good — callee reads the runner and returns a small frozen struct; the orchestrator
95# owns the writes.
96# model_runner.py
97class ModelRunner:
98 def bar(self):
99 self.foo_result = foo(self)
100
101# another_file.py
102def foo(model_runner) -> FooResult:
103 return FooResult(a=xx, b=yy, c=zz)
104
105# Avoid — callee reaches back in and writes the runner's fields.
106# model_runner.py
107class ModelRunner:
108 def bar(self):
109 foo(self)
110
111# another_file.py
112def foo(model_runner):
113 model_runner.a = xx
114 model_runner.b = yy
115 model_runner.c = zz
116```
117
118## 2. `__init__` style
119
120Apply when modifying the `__init__` of the three classes above.
121
122### 2.1 Why
123
124- Downstream forks override one piece (tokenizer, KV cache, IPC, …).
125- Inline logic forces them to copy the whole `__init__`, which rots against upstream.
126- Splitting into `init_*` helpers lets them override exactly what they need.
127- Reference shape: `TokenizerManager.__init__` in `python/sglang/srt/managers/tokenizer_manager.py`.
128
129### 2.2 Rules
130
131- **`__init__` is an orchestrator.** Sequence of `self.init_*(...)` calls + minimal glue. No non-trivial construction inlined.
132- **One helper per overridable unit.** Each `init_*` = one concern a subclass might swap. Don't lump.
133- **Naming:** `init_<thing>` (snake_case, names the component). Conditional construction → `maybe_init_<thing>`, gate inside the helper.
134- **No silent state coupling.** A helper only reads `self.*` set by earlier helpers. Ordering lives in `__init__`. Shared intermediates → pass as args, not via `self.*`.
135- **New logic = new helper.** Default to adding `init_<thing>`, not another inline block. One-line `self.foo = server_args.foo` is fine; structured logic is not.
136- **Preserve override points.** Prefer additive changes to existing `init_*` signatures. Breaking changes → call out in PR.
137
138### 2.3 Scope
139
140- Only the three classes listed above.
141- Not other manager-style classes, not small dataclass/utility constructors.