Sglang Runtime Context

SGLang runtime-context architecture

One container owns process-static runtime state: sglang.srt.runtime_context.RuntimeContext (a process singleton reached via get_context()). Everything below is a tier on it.

reset_context() (unit-test teardown) drops the published config and installs fresh flags/resources/forward tiers.

Config: publish + namespace bags

ServerArgs holds the raw input and nothing else. Resolution writes no field: it declares, and the declarations are what the namespace bags are projected from. Business code never reads the record for a decision: a field read there answers with what the operator typed, not with what resolution decided.

• Every publishing process entry calls publish(server_args, role=...) (run_scheduler_process, the Ray SchedulerActor, the DP controller, tokenizer, detokenizer, encoder, weight-cache daemon, the multi-tokenizer worker, the spawned encoder TP/DP workers, the benchmark work functions, ...); constructors do not publish — ModelRunner, TokenizerManager and MMEncoder call assert_published and fail loudly if an entry forgot. The roles are enumerated once, as the keys of ROLE_NAMESPACE_SETS — there is no launcher role, the launch path publishes as tokenizer. The remaining non-publisher is run_multi_detokenizer_router_process: it is handed a ServerArgs, and uses it only for configure_logger(server_args) today, so it has nothing to publish for — a bag read added under that entry needs a publish at the entry first. publish projects the config bags from the declarations over the record's raw fields; the accessors (get_exec() etc.) fail closed before it runs. role records which process type published, and keys per-role namespace enforcement: SGLANG_ROLE_NAMESPACES=record audits which namespaces each role's process actually reads (per-pair persisted via SGLANG_ROLE_NAMESPACES_OUT; reads inside torch.compile-traced code are NOT observed — audit with compilation disabled before restricting a role), and =enforce fails closed on bag reads outside the role's ROLE_NAMESPACE_SETS entry (None = full tree; only audited roles are restricted).
• Bag membership is where the field is declared: one class per namespace under arg_groups/fields/, each carrying the _NS_PATH it stands for, and ServerArgs is assembled from them (collect_input_fields). The per-field NS("path") marker survives only for a class that cannot express this — an ad-hoc dataclass spanning namespaces, which is what the config-bag tests build. Coverage is linted two-way (test_server_args_namespaces.py, test_runtime_context_config_bags.py).
• Reading config: get_<ns>()[.sub].field — e.g. get_exec().moe.moe_a2a_backend, get_schedule().max_running_requests. Bag leaves are plain instance attributes, safe inside torch.compile-traced code.
• Mutating config after publish: the ONLY entry point is get_context().override(source, **fields). It writes the bag leaves in place (namespace readers see the new value) and records provenance in the overrides log. There is no write-through to the ServerArgs instance — it stays pristine. There is no in-place mutation entry on the instance at all: it is read-only after resolution.
• Reading a leaf when the caller holds the field name (a readback endpoint, a control-plane handler): get_context().config_leaf(name) — the read side of override. It resolves the flat name through the same NS map the write side uses and raises on a name that is not a config leaf. Code that knows its field when it is written reads the bag leaf directly; config_leaf is for name-driven code, not a way around the seed ratchet.
• Post-startup control-plane changes — a weight update, a HiCache mirror attach, a parser resolved from the chat template — go through TokenizerManager.record_config_updates(source, **fields), a named wrapper over get_context().override. One process keeps one log: the request dumps ship get_context().overrides_log(), and config_value(name) / resolved_config_dict(base) answer from the bags. The exposure ratchet resolves the wrapper, so a field recorded through it joins the post-publish override surface exactly like a direct override and needs the same ordering judgment against any supplied-instance read of it (test_supplied_instance_exposure_ratchet.py).
• model_path and served_model_name are answered off the manager. Both are NS leaves and override accepts them, but the tokenizer-side weight reload records only load_format and writes the two path fields as TokenizerManager attributes (_MANAGER_OWNED_FIELDS); config_value and resolved_config_dict overlay them on top of the bags. Bags do not cross a process boundary (above), so recording those two in the tokenizer process would leave every other process's bag on the old path while the log claimed a process-wide change. The scheduler rewrites its own copy where the reload happens — ModelRunner.update_model_fields overrides model_path / load_format for the target runner.
• Late launcher-stage resolution (pre-publish): a few rules cannot run inside __post_init__ — LoRA normalization, and the auto-parser detection that needs a tokenizer/chat-template load. They are resolution, not mutation, and they declare via arg_groups.overrides.declare_resolution(server_args, source, **fields), the same call the rest of the pipeline makes; there is no declare_late_resolution any more. When a declaration is made is not something the code marks — the guardrails that used to read that marker cover these sites through the ordinary keyword scan instead. The declaration lands in the stash on that very object, so every holder of it carries the decision — the HTTP server, the multi-tokenizer workers it is serialized for, the schedulers it forks — and each of them publishes bags projected from it. The fields stay the operator's input; resolution_result(sa, field) and the bags are what answer for the decision. Returning a variant here is a bug: the launcher rebinds its local and everyone else keeps the unresolved object.
• A value another runner / worker owns is a constructor argument, not a config copy. The draft worker's context_length, load format and attention backend travel as arguments to TpModelWorker / ModelRunner and live on the runner (ModelRunner.draft_attention_backend, kv_cache_dtype_str, …); the encoder DP worker's device is MMEncoder(gpu_id=...). There is no ServerArgs.derive any more — a config object is never copied-and-edited; test doubles that need a modified copy use sglang.test.test_utils.server_args_variant.

Why a bag override cannot stand in for late resolution or per-runner construction. The bags are projected at publish from the declarations over the instance's raw fields, so anything the runtime must read has to be declared before publish — an override afterwards puts instance and bags back out of agreement, and whole-object readers (ModelConfig.from_server_args, build_load_config, MMEncoder's own self.server_args.X) never see it. And bags do not cross a process boundary: a child publishes from the object it receives and re-projects its own bags, so a parent-side override is lost. Values that feed construction before any bag exists (group init reads server_args.tp_size) have no bag to override at all.

Reads that legitimately stay on a ServerArgs instance

• Per-runner values — there is no per-runner ServerArgs any more. The draft-worker config copy is gone: every worker (TpModelWorker, the draft workers in speculative/) is handed the same instance the process published, so a bag leaf is the decision and self.server_args.X is the operator's input — a post-publish override moves only the bag, which is exactly why a field that is process-wide config (attention_backend, skip_tokenizer_init, kv_cache_dtype) reads from the bags like any other, and why a residual instance read on this path is stale the moment someone overrides that leaf. What is genuinely per-runner travels two ways, neither of them a config instance: constructor arguments (ModelRunner(draft_attention_backend=...), MMEncoder(gpu_id=...)) and runner attributes holding the resolved value (model_runner.kv_cache_dtype_str, prefill_attention_backend_str, num_fused_shared_experts, linear_attn_backends) — threaded to consumers as arguments, never backfilled onto a shared object. A per-runner choice also stays out of the bags: recording it there is how a second runner inherits the first one's answer, which is exactly the bug linear_attn_backends replaced. The one sanctioned bend in that rule is scoped: ModelRunner._load_format_scope exposes the draft's --speculative-draft-load-format through get_model().override(load_format=...) for exactly the duration of the draft build, because model construction reads that bag leaf — the override restores on exit, so nothing outlives the scope. When there is a runner in hand, read its stamp; that is a different rule from "read the instance".
• Per-instance boundaries — the tokenizer-manager family, everything under entrypoints/, and the tokenizer-process multimodal processors read the bags. The old justification for keeping them on self.server_args ("several Engines can share one process, bags are last-publish-wins across them") is retracted — owner ruling (2026-08-15): a process holds at most one live config at a time (concurrent multi-Engine is unsupported; sequential rebuild stays legal, unit tests rely on it). Nothing in those files reads the instance any more -- the exposure ratchet's pin set is empty, so the next such read is a new entry that has to argue for itself. What genuinely stays per-instance is what differs per worker within one engine: base_gpu_id travels as a constructor argument (MMEncoder(gpu_id=...); BaseMultimodalProcessor._fast_image_processor_device is the shape to copy).
• Whole-object passes (f(server_args) handing the instance along) keep the supplied-instance contract; don't rewrite the parameter reads unless the field is runtime-mutated (see the elastic-EP ep_size case in eplb/expert_location.py) — or the field is one that resolution fills in and the callee runs in a process that has published. That second case is a decision, not a style question: the record carries the user's raw input, so a resolution-filled field read off it inside a runner-owned constructor answers with the pre-resolution value instead of the effective one. The answer is not automatically a bag read: pick where the value should come from — usually the get_*() bag, sometimes a runner stamp or a constructor argument (the per-mode attention pair and the encode-server gpu_id above are both this). The per-instance boundaries above are not exempt from this unless-clause (the multi-Engine exemption is retracted); each one gets its own disposition. test_supplied_instance_exposure_ratchet.py pins that set (empty today) — three spellings of the read: server_args.field, literal-name getattr(server_args, "field", default), and the parked form (self.x = server_args in a method that takes the parameter, read as self.x.field anywhere in the class) — and fails on a new one, so the disposition gets picked when the read is written. Two shapes stay parameter-form on purpose: a helper the resolution pipeline calls with a resolved_view (its parameter happens to be named server_args), and a factory whose contract is "build X from the record you are handed" (create_kt_config_from_server_args, DllmConfig.from_server_args).

Four ways a config sweep breaks something no test runs

Each of these shipped in a review round and cost a real defect; each now has a guard, named here so the next sweep checks the same four things by hand first.

1. The other implementations of an interface. Dropping a parameter means auditing implementers, not just callers: CustomSpecAlgo is the plugin base for speculative algorithms, and the dispatch calls it with the built-in's argument list. Nothing in the tree implements it, so only a plugin user hits the TypeError. Guard: test_plugin_hook_signatures.py.
2. Publish order inside a process entry, not per file. A file containing a publish says nothing about whether a given read runs before it. Spawned workers (MMEncoder for encoder DP/TP, the Ray scheduler actor) start with an empty context, so a bag read above the publish raises only there. Guard: test_publish_precedes_bag_reads.py.
3. The role namespace a process publishes under. ROLE_NAMESPACE_SETS narrows what each role may read; the DP controller is audited for exec alone. A helper that reaches for another namespace passes every default-mode test and aborts startup under SGLANG_ROLE_NAMESPACES=enforce. Prefer answering from the caller's own namespaces over widening the set.
4. Sibling surfaces of a readback. Changing what one entry point reports means enumerating the others: HTTP, gRPC and in-process Engine each have their own server-info and model-info, and each passes its own tests while its users lose the field. Guard: test_effective_state_surfaces.py.

A fifth, from the same rounds: the accessor name itself. Called as an object member (manager.get_disagg()), or shadowed by a same-named import (from model_loader import get_model next to the model bag, where the later import silently wins and the loader call gets a zero-argument bag), it imports fine and fails only when that path runs. The invariant is one line: the name means the process-wide bag, takes no arguments, and is bound once per module. ruff --select F811 catches the import collision; the member-call shapes are an AttributeError at call time only because RuntimeContext has no bag-named member and no __getattr__ -- a delegating __getattr__ would make them silent, and that is when this needs a guard again rather than a rule.

Write these guards over a derived set, never a hand-kept list: an entry naming a function that no longer exists, or a field list missing the one field nobody migrated, passes green forever. Both happened here -- a _ENTRY_POINTS row for a method the Ray actor does not have, and an effective-field set without load_format -- and both were invisible because the assertion had slack (>= len(...) - 1) or compared key names instead of value sources.

get_parallel(): one spelling per name

There is no .config hop. Ranks and group handles are @property read-through over the canonical getters, so they answer with the live process groups. Everything else — tp_size, pp_size, attn_cp_size, dcp_size, moe_dp_size included, alongside config-only leaves like nccl_port, enable_dp_attention, dp_size, ep_size, dwdp_size — is answered from the published parallel bag. Reading a leaf before publish raises a ValueError naming the namespace; an unknown name is an AttributeError.

A size reads from the configuration because the groups are built at exactly the configured widths — checked at every assignment to _TP / _PP / _ATTN_CP / _DCP / _MOE_DP in parallel_state.py. Three things do not follow that rule:

• initialize_model_parallel aliases _MOE_DP to _ATTN_CP when attn_cp_size > moe_dp_size, so a reader that means the MoE communicator's width calls get_moe_cp_size(), not get_parallel().moe_dp_size.
• patch_tensor_parallel_group runs a scope under a different TP group (draft workers), and declares it by overriding tp_size, tp_rank and tp_group for the scope's duration. Readers inside need no special spelling.
• Elastic EP scales ep_size / dp_size on the published bag while the group coordinators keep their construction width. Those are different names, not two answers to one name.

DCP keeps its own pair: get_parallel().attn_dcp_size / .dcp_enabled answer the effective topology (1 / False with no group installed), while dcp_size is what the launch requested.

A process-global seed field-read of one of these sizes (get_server_args().tp_size, or an alias of it) is a read-ratchet failure. A server_args the object was handed is a different thing and not a ratchet matter — see "Reads that legitimately stay on a ServerArgs instance". Fail-loud is narrower: before dist init, a live rank/group read raises. The six parallel quotients are not live reads at all — attn_tp_size, attn_dp_size, attn_dcp_size, moe_ep_size, moe_tp_size, dcp_enabled are a function of the configured leaves, computed once at publish into bag leaves, and answered override → stamp → published leaf. So dcp_enabled means "the launch configured DCP" (dcp_size > 1), not "a DCP group is installed here"; in a scheduler the stamp makes the two identical, in a process that publishes without dist init they differ. test_a_topology_is_stated_by_naming_the_width and its neighbours in test_runtime_context.py pin this; they replaced test_attn_dcp_defaults_when_group_is_uninitialized. One consequence for tests: overriding a leaf no longer moves its quotient — state a topology by publishing a config, or by naming the width. After init, only the DCP group is optional (_DCP exists only when dcp_size > 1; attn-CP and moe-DP always install, as size-1 aliases if unused). The config hop is deliberately dynamo-traceable (a plain property over a slot, no object.__getattribute__); gate helpers like enable_moe_dense_fully_dp() run inside compiled model forwards (test_parallel_config_leaves_trace_under_torch_compile pins this).

A third surface carries the same names: ParallelState (self.ps / mr.ps), the frozen per-process snapshot built once in Scheduler.__init__ from these configured sizes plus this process's ranks, and handed down (draft runners included). Prefer it where an object was handed one; it is not a global accessor.

Reading config: the seed is off limits

get_server_args().field in business code is a ratchet failure. Read:

• a resolved leaf → its namespace bag (get_exec().moe.moe_runner_backend, get_schedule().chunked_prefill_size, …). Bag-backed reads — a leaf directly, or a bag-derived accessor below — are what see post-publish overrides. Only the instance-derived accessors (the ones with no leaf to read) answer from the startup record and therefore do not.
• a leaf the caller names at runtime (a readback reporting a list of fields) → get_context().config_leaf(name); it resolves the name through NS and raises on a non-leaf. A call site that knows its field reads the bag leaf.
• the live topology → get_parallel() (bare names).
• a value derived from published leaves → an accessor in runtime_context that derives it from the bags. The strongest form of this is a Derived(fn=...) declared beside the leaves it is computed from, in the namespace's own arg_groups/fields/ class: publish computes it once and stores it as an ordinary bag leaf, so the read is a plain attribute load and it sees post-publish overrides. enable_mamba_extra_buffer, is_ep_joiner, is_ep_scale_joiner and is_startup_weight_load_overlap are declared that way now — read them where they are declared: get_exec().mamba.enable_mamba_extra_buffer, get_exec().moe.is_ep_joiner, get_model().is_startup_weight_load_overlap. (The namespace is the class that declares the field, not the namespaces its fn happens to read: the mamba one spans exec.mamba and memory, which is exactly why it could not be a method on either bag.) The old mamba_extra_buffer_enabled() / is_ep_joiner() functions and the same-named ServerArgs members are gone. The pre-publish helpers that remain exist for resolution, which has no bag to read yet. attention_backends() derives the (prefill, decode) pair from the three exec.kernel leaves, and max_speculative_num_draft_tokens() / cutedsl_moe_max_num_tokens() derive theirs from spec / schedule / exec.graph.
• a value only the instance can compute → the named accessor in runtime_context, which is the one module allowed to read the slot: mamba_cache_chunk_size(), mamba_state_chunk_size(), uses_mla_backend(), process_model_config(). These have no leaf to read — they combine several fields, the HF config, or a property with no bag of its own. A new derived member gets an accessor here rather than call sites reaching for the record, and only when the bag-derived shape above cannot express it.
• a parallel size → get_parallel().{tp,pp,moe_dp,attn_cp,dcp}_size, which is the parallel bag's own leaf: it answers with the resolved configuration and follows a post-publish override. Two questions are not that, and have their own spelling: the width of the MoE communicator you are about to collectively operate on is get_moe_cp_size() (the _MOE_DP = _ATTN_CP alias makes it differ), and the effective DCP topology is get_parallel().attn_dcp_size / .dcp_enabled (1 / False when no group is installed), which does not need dist init to answer.
• this runner's resolved value → the runner (prefill_attention_backend_str, kv_cache_dtype_str, draft_attention_backend, num_fused_shared_experts on the model).

self.server_args.field is still right for handed per-instance config (see "Reads that legitimately stay on a ServerArgs instance" above for the full set — per-instance boundaries and whole-object passes; there are no per-runner config copies to read any more). The allow-list is GrammarManager and MMEncoder; what sits beside it is residue, not a family — and not for one single reason:

• the tokenizer-manager family and entrypoints/ read the bags; what is left of them in the exposure ratchet is a handful of individually-dispositioned pairs, not a family awaiting conversion. Read the ratchet for the current set rather than assuming a directory is off-limits;
• GrammarManager is a handed instance for its residual self.server_args reads, but backend selection is not on the instance any more: create_grammar_backend reads get_exec().kernel.grammar_backend, and __init__ calls that factory whenever skip_tokenizer_init is false. In production the scheduler process has published; a test that constructs one without publishing has to keep patching the factory (or publish itself);
• MMEncoder publishes the very instance it is handed (publish(server_args, role="encoder")) and takes its per-worker device as a separate gpu_id argument. Its self.server_args reads are on this list as a construction-path convention, and the residual is real: they answer with the raw input, so a leaf resolution decided and a post-publish override both pass them by.

Their tests are not one story: a GrammarManager built standalone turns the factory's bag read into "config namespace not published" unless the test patches it or publishes, while MMEncoder publishes in its own __init__ and so needs no such arrangement.

Test doubles publish, they do not inject. A stand-in that carries server_args=SimpleNamespace(field=...) stops working the moment production reads the bag; seed the value with override_server_args, which publishes only once it is entered or installed — the bare call just builds the override:

override = get_context().override_server_args(field=...)
override.install()
self.addCleanup(override.restore)      # or: with get_context().override_server_args(...):

Five separate test files learned this the hard way during the sweep.

The rule is about a double standing in for config: a SimpleNamespace that pretends to be server_args. Prefer the context override even where a single-accessor stub would work — override_server_args(...) composed with the scoped bag / get_parallel() overrides expresses the cause (the configuration) rather than pinning one helper's answer, and it keeps working when a reader migrates between the accessor and the leaf. The sweep converted the last two accessor stubs to exactly that shape (test_attention_patching.py publishes the non-lazy strategy; test_kimi_k3_vision.py publishes tp_size and forces the live topology through get_parallel().override), so no test stubs an accessor today. Stubbing one named accessor remains a last resort for a case that isolates one branch of one helper where no published config can reach it — if you do it, say so in the test.

Mid-resolution reads (inside the pipeline only)

Resolution runs in __post_init__ and writes nothing onto the record: a handler declares (self._declare / declare_resolution), the declaration goes into the stash, and the fields keep what the caller passed. So a mid-resolution read of a field answers with the raw input — every reader in the pipeline goes through a view instead:

• resolving_view(server_args) / self._resolved() — the live view (walks the stash per read). This is what handlers and hooks bind, conventionally as cfg = resolving_view(self) at the top of the handler.
• resolved_view(server_args) — snapshots the overlay when built, which is what a post-process pass wants: it reads the state at its slot.

test_resolution_reads_the_declarations pins direct field reads at zero over the two scopes it can derive exactly (every arg_groups function taking a config, every ServerArgs handler the dispatcher reaches). Readers the pipeline calls from elsewhere (ModelConfig, the platform defaults, the spec-algo hook) have moved to the view as well — a field read there is the same bug, just one the derivation cannot enumerate.

One consequence worth knowing: because the fields are the raw input, resolving a bare dataclasses.replace copy lands in the same place as the parent — the pipeline reads only its own input. So a resolved record is not copied at all. A caller that needs one field different for the process it is about to hand the record to — the Ray paths and their dist_init_addr — declares it on the record it holds (declare_resolution) and hands that over: the declaration travels inside the object, the receiving process projects its bags from it, and nothing re-resolves. There is no ServerArgs.replace_resolved any more, and the model_config-memo bug that copying used to cause (a copy marked resolved but arriving without the memo cannot refill it, because the guard refuses the write) is gone by construction rather than guarded.

A bag override cannot stand in for this. It is not because overriding needs a publish — set_server_args is what projects the bags and override works as soon as the context holds a record — but because override writes bag leaves and by contract never touches the record, so its effect cannot travel inside an object to another process.

The declaration stash has one writer

Everything that decides configuration goes through declare_resolution(server_args, source, **fields). It validates the names, refuses the published config (the stash is projected at publish and never again, so a later declaration is a silent no-op), and appends. The other names around it are spellings, not mechanisms:

resolution_projection is gone; the whole-object readback is ServerArgs.resolved_dict(), which is what /server_info and its gRPC and in-process twins report.

Adding a model-specific config adjustment

Never assign server_args fields from model code. Declare instead (sglang/srt/arg_groups/overrides.py):

• Constant per-arch values → MODEL_OVERRIDES["MyArchForCausalLM"] = {...}.
• Derived values → @register_model_override("MyArchForCausalLM") returning a dict; the callable receives pristine server_args + hf_config and must not write.
• Normalization that must see earlier declarations → a post-process pass invoked via run_post_process_pass at its slot (reads a view, returns a declaration dict).
• Values only knowable at load time are per-runner state, not declarations: there is no declare_load_time_override any more. A model-family decision that its checkpoint drives (shared-experts fusion) is a question the loader asks the model class — shared_experts_fusion_disable_reason(hf_config, quant_config), a classmethod answering without an instance — at the single model-instantiation point, and install_shared_experts_fusion_decision writes the answer to the ACTIVE moe flag before that model's layers build and read it (is_shared_experts_fusion_disabled, config-intent fallback). draft_model_build_scope brackets every draft build and routes the draft's answer to the speculative leaf, so a draft's decision never overwrites the target's. A process-level load-time fact (the sm80 dtype fallback — device-driven, identical for every runner) records directly via get_context().override.

Declarable fields form a whitelist: Arg(..., resolvable=True) in the ServerArgs dataclass. A declaration against a non-whitelisted field fails at its slot.

Load-time vs resolution-time (critical)

__post_init__ runs in the launcher process before any model/platform import. Logic that consults an extensible registry (e.g. out-of-tree platforms registering attention backends in init_backend(), which runs at model_runner import) must stay at load time (ModelRunner init), writing through get_context().override(). Before moving any load-time logic into resolution, verify everything it reads is already complete at construction time.

Runtime flags (get_flags())

For state that init-time code derives and runtime code reads — parsed enums, platform probes, swappable ACTIVE values. Not for config mirrors (read the bag leaf instead).

• Groups are typed dataclasses on Flags (capture / moe / dp): typo-safe writes, transactional test-only override(**kw) context manager.
• flags.moe is materialized by initialize_moe_config() at scheduler init (it reads exec.moe / spec / model, and takes no record); accessors (get_moe_a2a_backend etc.) are thin shims with lazy defaults. The speculative contexts (speculative_moe_backend_context) swap the ACTIVE leaves around draft forwards.
• flags.dp is materialized by initialize_dp_attention; is_dp_attention_enabled() is a shim over flags.dp.enabled.
• Adding a leaf: declare the dataclass field with a default equal to the pre-init behavior, materialize it at the owning subsystem's init, keep any public accessor as a shim.

Resources (get_resources())

Named slots + two keyed-lazy registries:

• get_stream(name) — get-or-create a named CUDA side stream; set_stream(name, stream) installs explicitly. Name leases by subsystem ROLE: all model alternate streams share "alt"; the offloader's copy stream is "offload"; DP-TBO comm is "dp_tbo_comm"; LoRA side stream is "lora_side". Two call sites may share a name only if their work belongs on one stream — sharing across roles serializes intended overlap.
• get_buffer(name, factory) — get-or-create a named persistent buffer. Grow-only or per-device semantics manage their resources.buffers entries directly (see tokenspeed / SM120 split / Marlin workspace). Buffer names are per-backend today; do not silently share.
• Singletons with manager semantics (EP dispatcher buffers, EPLB recorder/metadata, graph memory pool) keep their owning accessors/classes as facades; only the state lives in a resources entry. Preserve exact semantics in the shim: lazy defaults (the EPLB recorder defaults to a Noop instance, not None), publish-once asserts, event-reuse contracts.
• Stream/buffer creation is a driver call — it must happen outside cuda-graph capture; keep lease points at init/warmup time.

Per-forward flags (get_forward())

Contextvar-backed; a new thread sees the defaults; scoped(**kw) is the regular write path (transactional, restores on exit and on exception); set(name, value) exists for legacy sticky setters (is_extend_in_batch is intentionally sticky within a thread). Use this tier for anything set-per-forward and read-within-forward. Before adding cross-thread state here, prove the readers' thread affinity: contextvars do NOT propagate to already- running or newly spawned threads. Note TBO ("two-batch overlap") interleaves ubatches on ONE thread — do not design for TBO threads that don't exist.

Testing idioms

• Force a code path by overriding causes, not effects: compose get_context().override_server_args(**fields) (publishes a fresh dummy-boundary ServerArgs carrying the overrides AND projects the bags — with-scoped, or install()/restore() + addCleanup for fixture-lifetime use) + get_<ns>().override(...) (scoped override of one bag's own leaves) + get_parallel().override(...) (live topology) + get_flags().<group>.override(...) + get_forward().scoped(...). All are scoped and transactional. Tests control execution through the context — do not hand-build and publish config objects.
• Never monkeypatch import bindings (module.get_x = lambda: ...) and never fake a config source with a SimpleNamespace stand-in: production reads the published bags, so a faked accessor silently stops intercepting after any reader migration. Publish for real (override_server_args(...)), then adjust bag leaves with the scoped bag override where the constructed ServerArgs cannot carry the value (e.g. get_device().override(device="meta")). The one carve-out is the deliberate single-accessor stub for isolating one predicate — the terms and the two sanctioned examples live under "Test doubles publish, they do not inject" above; anything wider than one named accessor is this rule.
• Mocked runners/managers still need the per-runner instance attributes the code under test reads (kv_cache_dtype_str, server_args for whole-object passes) — set them explicitly on the mock; MagicMock(spec=...) raises on attributes that only exist post-__init__, which is the fastest way to find a missed stub.
• reset_context() in teardown when a test publishes outside a scoped override.
• ServerArgs(model_path="dummy") early-returns the pipeline (few declarations, no strict guard) — fine for lightweight fixtures.
• Asserting what resolution decided reads resolution_result(sa, "field"), not sa.field: the field is the raw input. Assert the field only when the point of the case is that the record stayed pristine (the FA4 page-size and waterfill cases do exactly that, and say so).
• Run changed test files per-file (own process), the way CI does: a monolithic local pytest run lets a context published by an earlier file mask a missing-publish bug in a later one.

Guardrails (these fail CI; what to do when they fire)

1. Strict mutation guard (always on, and with no exception): bare server_args.x = ... after resolution raises unconditionally in ServerArgs.__setattr__ — the named lift that out-of-tree plugins used to ask for is gone, they assign onto a stand-in instead — this is the guarantee that no writer can desync the bags, so there is no writer ratchet any more. Change resolved config with get_context().override; hand a per-runner value to its runner as a constructor argument. Projected bags are sealed the same way (leaf assignment raises).
2. Mutation ratchet (test_server_args_mutation_ratchet.py, exact pin 0 over the whole package minus the pipeline / multimodal_gen): textual scan for assignment forms. Never raise the baseline.
3. No-copy contract (test_server_args_no_instance_mutation_entry.py): neither ServerArgs.override nor ServerArgs.derive exists, and nothing in the package calls either form. Rerouting a writer to the bags means flipping all its readers in the same commit (no transitional dual-write).
4. **The

…(truncated)