Clean Up SGLang Server Startup Logs
Goal: ensure the server startup log is clean and minimal, with no spurious warnings, deprecation messages, or unformatted prints from third-party libraries.
Workflow
1. Launch a server and capture the log
uv run sglang serve --model-path Qwen/Qwen3-8B 2>&1 | tee /tmp/startup_log.txt
Wait until the server prints The server is fired up and ready to roll!, then Ctrl-C.
For TP>1 testing:
uv run sglang serve --model-path Qwen/Qwen3-8B --tp 2 2>&1 | tee /tmp/startup_log.txt
For MoE / hybrid-SWA models (e.g. gpt-oss), test separately — they exercise different code paths:
uv run sglang serve --model-path openai/gpt-oss-20b 2>&1 | tee /tmp/startup_log.txt
2. Compare against the clean reference log
Read /tmp/startup_log.txt and compare it against the reference log at the bottom of this file. Identify lines that:
- Do NOT have the
[timestamp] or [timestamp TPx] logger prefix
- Contain
WARNING, deprecated, is deprecated, or similar noise
- Are printed by third-party libraries (transformers, torchao, NCCL, Gloo, tqdm, etc.)
- Are duplicate/redundant with information already logged by SGLang
- Appear multiple times due to
ModelConfig being constructed in multiple processes
3. Classify each noisy line
For each noisy line, determine:
| Category |
Action |
| SGLang code using wrong API |
Fix the SGLang code (e.g., replace deprecated API with new one) |
| SGLang code logging at wrong level |
Change log level (e.g., warning -> debug for non-actionable messages) |
| Duplicated across processes |
Downgrade to debug — info logged in one process becomes noise in 3-4 |
| Third-party lib prints at import time |
Suppress the logger or redirect stdout during that import |
| C-level print from .so library |
Redirect fd 1 during the specific C call, or accept it if too invasive |
| Real warning the user should see |
Keep it |
4. Present findings before fixing
List all noisy lines with their source and proposed fix. Ask the user to review before making changes.
5. Apply fixes and verify
After approval, apply fixes one at a time, re-launch the server, and verify each fix works.
Key Architecture: Why Logs Repeat
ModelConfig is constructed 3-4 times during startup across different processes:
- Main process:
ServerArgs.__post_init__() → get_model_config() → ModelConfig()
- Scheduler subprocess:
Scheduler.init_model_config() → ModelConfig.from_server_args()
- Scheduler subprocess:
TpModelWorker._init_model_config() → ModelConfig.from_server_args()
- Main process:
TokenizerManager.init_model_config() → ModelConfig.from_server_args()
Similarly, get_tokenizer() is called 5 times across processes:
resolve_auto_parsers (main) — template_detection.py
Scheduler.init_tokenizer() (scheduler subprocess) — scheduler.py
DetokenizerManager (detokenizer subprocess) — detokenizer_manager.py
TpModelWorker.__init__() (scheduler subprocess) — tp_worker.py
TokenizerManager (main) — tokenizer_manager.py
Any logger.info() or logger.warning() in ModelConfig.__init__() or get_tokenizer() will appear 3-5 times. Keep these at logger.debug().
Known Noise Sources and Fixes (from past sessions)
1. torchao "Skipping import of cpp extensions due to incompatible torch version"
2. "torch_dtype is deprecated! Use dtype instead!" (PARTIALLY FIXED)
- Source:
transformers/configuration_utils.py — the torch_dtype property warns via logger.warning_once()
- Trigger: Model files accessing
config.torch_dtype instead of config.dtype
- Fix applied so far: Only
models/gpt_oss.py (lines 222, 471) — tested with openai/gpt-oss-20b.
- Remaining files that still use
config.torch_dtype (fix each only after testing with the corresponding model):
models/bailing_moe.py (line 302)
models/llada2.py (line 313)
models/qwen3_next.py (lines 192, 209)
models/qwen3_5.py (line 245)
models/nano_nemotron_vl.py (lines 79, 102, 284)
models/llava.py (lines 732, 734-737)
model_loader/loader.py (line 649)
- Note:
common.py was already fixed in a prior session. If new model files are added with config.torch_dtype, the warning will reappear — grep for \.torch_dtype to find them.
- Important: Only change
config.torch_dtype → config.dtype for models you have actually tested. The dtype property should return the same value, but verify per-model to avoid regressions.
3. "BaseImageProcessorFast is deprecated"
- Source:
transformers/utils/import_utils.py — the lazy module __getattr__ warns when BaseImageProcessorFast is accessed
- Trigger:
base_processor.py and ernie45_vl.py have from transformers import BaseImageProcessorFast at top level. These are imported eagerly via tokenizer_manager.py -> multimodal_processor.py -> base_processor.py, even for non-multimodal models.
- Fix: Replace
from transformers import BaseImageProcessorFast with from transformers import BaseImageProcessor and update all isinstance(..., BaseImageProcessorFast) checks to isinstance(..., BaseImageProcessor)
4. "No platform detected. Using base SRTPlatform with defaults."
- Source:
sglang/srt/platforms/__init__.py — logger.warning()
- Fix: Change to
logger.debug() — this is expected on machines without a platform plugin and not actionable.
5. NCCL version 2.27.7+cuda13.0
- Source: C-level print from
libnccl.so during ncclCommInitRank() call
- Status: Accepted as-is. SGLang already logs the version via
sglang is using nccl==X.Y.Z. The C-level print cannot be suppressed without redirecting stdout fd, which is too invasive. NCCL_DEBUG=WARN does not suppress it in NCCL 2.27+.
6. [Gloo] Rank X is connected to Y peer ranks
- Source: C++ Gloo library print during process group init
- Status: Accepted as-is. From C++ code inside PyTorch's Gloo backend.
7. torchao SyntaxWarning: invalid escape sequence
- Source:
torchao/quantization/quant_api.py — a raw string with unescaped \.
- Status: Upstream torchao bug. Cannot fix from SGLang side.
8. tqdm progress bars (e.g., Multi-thread loading shards, Capturing batches)
- Status: These are expected and useful. They show progress during weight loading and CUDA graph capture. Keep them.
9. CUTE_DSL "Unexpected error during package walk" — double-logged (FIXED)
- Source:
nvidia-cutlass-dsl package at .venv/.../cutlass/cutlass_dsl/cutlass.py, line 391. Logger named CUTE_DSL with its own StreamHandler.
- Trigger: During CUDA graph capture, cutlass DSL walks packages and hits an unexpected error for
cutlass.cute.experimental.
- Root cause of double-logging: The CUTE_DSL logger has
propagate=True (default), so the warning is emitted by both the CUTE_DSL handler (with its format) and the root logger (SGLang's format).
- Fix applied: In
entrypoints/engine.py, changed CUTE_DSL_LOG_LEVEL from "30" (WARNING) to "40" (ERROR). This suppresses the WARNING at both the CUTE_DSL logger and root propagation levels. The env var controls both logger.setLevel() and console_handler.setLevel() in cutlass's setup_log().
10. ModelConfig init logs repeated 3x (FIXED)
- Lines:
"Downcasting torch.float32 to ...", "Hybrid swa model: ...", "DeepGemm is enabled but ..."
- Source:
configs/model_config.py — _get_and_verify_dtype() (line 1457), _derive_hybrid_model() (line 497), _verify_quantization() (line 1236)
- Root cause:
ModelConfig.__init__() is called 3-4 times in different processes (see "Key Architecture" above). Each construction fires the same log lines.
- Fix applied: Downgraded all three from
logger.info()/logger.warning() to logger.debug(). The dtype is already visible in server_args and Load weight end. Hybrid SWA info appears in Tree cache initialized. DeepGemm is not actionable.
11. Tokenizer retry/fallback messages repeated 3-4x (FIXED)
- Lines:
"Tokenizer loaded as generic TokenizersBackend ... retrying", "Loading tokenizer ... directly as PreTrainedTokenizerFast", "Tokenizer for ... loaded as generic TokenizersBackend. Set --trust-remote-code"
- Source:
utils/hf_transformers/tokenizer.py — _resolve_tokenizers_backend() (line 215), _load_tokenizer_by_declared_class() (line 110), final warning (line 244)
- Root cause: 5 separate
get_tokenizer() calls across processes (see "Key Architecture" above). Each produces 3 log lines. Concurrent subprocess launches cause interleaved/doubled output.
- Fix applied: Downgraded all three from
logger.warning()/logger.info() to logger.debug().
12. Template detection logs — 5 lines consolidated to 1 (FIXED)
- Lines:
"Detected reasoning config '...' from template rule '...'", "Detected reasoning parser '...' from template rule '...'", "Detected tool-call parser '...' from template rule '...'", "Auto-detected reasoning parser: ...", "Auto-detected tool-call parser: ..."
- Source:
managers/template_detection.py (lines 337, 370) logged each detection rule match. managers/template_manager.py (lines 177-182) logged summary lines that duplicated the detection logs.
- Fix applied: Removed per-rule logs from
template_detection.py. Consolidated the 5 lines in template_manager.py into a single summary: "Auto-detected template features: reasoning_config=..., reasoning_parser=..., tool_call_parser=..."
13. KV cache dtype logged separately from allocation (FIXED)
- Lines:
"Using KV cache dtype: torch.bfloat16" then "KV Cache is allocated. #tokens: ..., K size: ..., V size: ..."
- Source:
model_executor/model_runner.py (line 2217) and mem_cache/memory_pool.py (line 740)
- Fix applied: Removed the standalone dtype log from
model_runner.py. Added dtype field to the allocation log in memory_pool.py: "KV Cache is allocated. dtype: torch.bfloat16, #tokens: ..., K size: ..., V size: ..."
14. CUTLASS backend warning — B200 → SM100, warning → info (FIXED)
- Line:
"CUTLASS backend is disabled when piecewise cuda graph is enabled due to TMA descriptor initialization issues on B200."
- Source:
layers/attention/flashinfer_backend.py (line 249)
- Fix applied: Changed "B200" to "SM100 GPUs" (the condition checks
is_sm100_supported() which matches SM10x, not just B200). Downgraded from logger.warning() to logger.info() since it's an expected automatic fallback.
15. max_total_num_tokens and Tree cache initialized log ordering
- Issue:
max_total_num_tokens=... appears before Tree cache initialized:... even though tree cache is conceptually part of memory setup.
- Root cause:
max_total_num_tokens is logged inside init_model_worker() (scheduler.py:972), which runs before build_kv_cache() (scheduler.py:425) where tree cache is created.
- Status: Not fixed — reordering was reverted. Acceptable as-is.
16. Ignore import error when loading sglang.srt.models.midashenglm
- Source:
models/registry.py (line 109) — logger.warning() during import_model_classes() which iterates all model modules via pkgutil.iter_modules
- Trigger: The
midashenglm model depends on torchaudio, which fails to load
- Status: Should be downgraded to
logger.debug() — not actionable when loading an unrelated model. Same pattern exists in managers/multimodal_processor.py, dllm/algorithm/__init__.py, multimodal_gen/runtime/models/registry.py.
17. Multiple NUMA nodes found for GPU X
- Source:
utils/numa_utils.py (line 112) — logger.warning()
- Status: Could be downgraded to
logger.info(). The situation is handled gracefully ("Using the first one") and not actionable.
18. Warmup /model_info access log
- Source: Uvicorn access log, triggered by SGLang's own warmup at
entrypoints/http_server.py (line 1877)
- Status: SGLang talking to itself. Could suppress uvicorn access logger during warmup, or exclude
/model_info from warmup access logging.
Investigation Techniques
Trace what triggers an import
import sys
_real_import = __builtins__.__import__
def _tracing_import(name, *args, **kwargs):
if 'TARGET_MODULE' in name:
import traceback
print(f'=== Importing {name} ===')
traceback.print_stack()
return _real_import(name, *args, **kwargs)
__builtins__.__import__ = _tracing_import
Trace what triggers a logger warning
import logging, traceback
class TraceHandler(logging.Handler):
def emit(self, record):
if 'SEARCH_STRING' in record.getMessage():
traceback.print_stack()
h = TraceHandler()
h.setLevel(logging.WARNING)
logging.getLogger('TARGET_LOGGER_NAME').addHandler(h)
Find C-level prints in .so files
strings /path/to/library.so | grep "SEARCH_STRING"
Find all config.torch_dtype accesses (for deprecation warning)
grep -rn '\.torch_dtype' python/sglang/srt/models/ python/sglang/srt/model_loader/ python/sglang/srt/utils/hf_transformers/
Reference: Clean Startup Log (TP=1, Qwen3-8B)
[2026-05-24 00:52:39] Attention backend not specified. Use trtllm_mha backend by default.
[2026-05-24 00:52:39] TensorRT-LLM MHA only supports page_size of 16, 32 or 64, changing page_size from None to 64.
[2026-05-24 00:52:40] server_args=ServerArgs(model_path='Qwen/Qwen3-8B', ...)
[2026-05-24 00:52:40] Multiple NUMA nodes found for GPU 0: [...]. Using the first one.
[2026-05-24 00:52:42] Using default HuggingFace chat template with detected content format: string
[2026-05-24 00:52:42] Auto-detected template features: reasoning_config=..., reasoning_parser=qwen3, tool_call_parser=qwen
[2026-05-24 00:52:50] Init torch distributed begin.
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[2026-05-24 00:52:50] Init torch distributed ends. elapsed=0.21 s, mem usage=0.10 GB
[2026-05-24 00:52:51] Load weight begin. avail mem=275.75 GB
[2026-05-24 00:52:51] Found local HF snapshot for Qwen/Qwen3-8B at ...; skipping download.
Multi-thread loading shards: 100% Completed | 5/5 [00:01<00:00, 2.62it/s]
[2026-05-24 00:52:54] Load weight end. elapsed=2.62 s, type=Qwen3ForCausalLM, avail mem=260.48 GB, mem usage=15.28 GB.
[2026-05-24 00:52:54] KV Cache is allocated. dtype: torch.bfloat16, #tokens: 1707904, K size: 117.28 GB, V size: 117.28 GB
[2026-05-24 00:52:54] Memory pool end. avail mem=25.28 GB
[2026-05-24 00:52:54] CUTLASS backend is disabled when piecewise cuda graph is enabled due to TMA descriptor initialization issues on SM100 GPUs. Using auto backend instead for stability.
[2026-05-24 00:52:54] Capture cuda graph begin. This can take up to several minutes. avail mem=24.16 GB
[2026-05-24 00:52:54] Capture cuda graph bs [1, 2, 4, ...]
Capturing batches (bs=1 avail_mem=23.56 GB): 100% | 52/52 [00:05<00:00, 10.36it/s]
[2026-05-24 00:53:00] Capture cuda graph end. Time elapsed: 5.38 s. mem usage=0.60 GB. avail mem=23.56 GB.
[2026-05-24 00:53:00] Capture piecewise CUDA graph begin. avail mem=23.56 GB
[2026-05-24 00:53:00] Capture cuda graph num tokens [4, 8, 12, ...]
Compiling num tokens (num_tokens=4): 100% | 74/74 [00:09<00:00, 7.44it/s]
Capturing num tokens (num_tokens=4 avail_mem=21.24 GB): 100% | 74/74 [00:07<00:00, 10.44it/s]
[2026-05-24 00:53:18] Capture piecewise CUDA graph end. Time elapsed: 18.18 s. mem usage=2.32 GB. avail mem=21.24 GB.
[2026-05-24 00:53:20] Tree cache initialized: source=default impl=RadixCache hybrid_swa=False hybrid_ssm=False hierarchical=False streaming_wrapped=False
[2026-05-24 00:53:20] max_total_num_tokens=1707904, chunked_prefill_size=16384, max_prefill_tokens=16384, max_running_requests=4096, context_len=40960, available_gpu_mem=21.24 GB
[2026-05-24 00:53:20] INFO: Started server process [1964249]
[2026-05-24 00:53:20] INFO: Waiting for application startup.
[2026-05-24 00:53:20] Using default chat sampling params from model generation config: {'temperature': 0.6, 'top_k': 20, 'top_p': 0.95}
[2026-05-24 00:53:20] INFO: Application startup complete.
[2026-05-24 00:53:20] INFO: Uvicorn running on http://127.0.0.1:30000 (Press CTRL+C to quit)
[2026-05-24 00:53:21] Prefill batch, #new-seq: 1, #new-token: 64, ...
[2026-05-24 00:53:21] INFO: 127.0.0.1:... - "POST /generate HTTP/1.1" 200 OK
[2026-05-24 00:53:21] The server is fired up and ready to roll!
Note: [Gloo] messages and tqdm progress bars are acceptable. The key is no warnings or deprecation messages from transformers, torchao, or other third-party libraries. The CUTLASS backend is disabled message is now info level, not a warning.
1---
2name: clean-startup-log
3description: Clean up noisy startup warnings and spurious prints in SGLang server logs. Use when users ask to clean up unwanted warnings, deprecation messages, or third-party noise in the server startup output.
4disable-model-invocation: true
5metadata:
6 source: https://skillmd.com/skills/sgl-project/clean-startup-log
7---
8
9# Clean Up SGLang Server Startup Logs
10
11Goal: ensure the server startup log is clean and minimal, with no spurious warnings, deprecation messages, or unformatted prints from third-party libraries.
12
13## Workflow
14
15### 1. Launch a server and capture the log
16
17```bash
18uv run sglang serve --model-path Qwen/Qwen3-8B 2>&1 | tee /tmp/startup_log.txt
19```
20
21Wait until the server prints `The server is fired up and ready to roll!`, then Ctrl-C.
22
23For TP>1 testing:
24```bash
25uv run sglang serve --model-path Qwen/Qwen3-8B --tp 2 2>&1 | tee /tmp/startup_log.txt
26```
27
28For MoE / hybrid-SWA models (e.g. gpt-oss), test separately — they exercise different code paths:
29```bash
30uv run sglang serve --model-path openai/gpt-oss-20b 2>&1 | tee /tmp/startup_log.txt
31```
32
33### 2. Compare against the clean reference log
34
35Read `/tmp/startup_log.txt` and compare it against the reference log at the bottom of this file. Identify lines that:
36
37- Do NOT have the `[timestamp]` or `[timestamp TPx]` logger prefix
38- Contain `WARNING`, `deprecated`, `is deprecated`, or similar noise
39- Are printed by third-party libraries (transformers, torchao, NCCL, Gloo, tqdm, etc.)
40- Are duplicate/redundant with information already logged by SGLang
41- Appear multiple times due to `ModelConfig` being constructed in multiple processes
42
43### 3. Classify each noisy line
44
45For each noisy line, determine:
46
47| Category | Action |
48|----------|--------|
49| **SGLang code using wrong API** | Fix the SGLang code (e.g., replace deprecated API with new one) |
50| **SGLang code logging at wrong level** | Change log level (e.g., warning -> debug for non-actionable messages) |
51| **Duplicated across processes** | Downgrade to debug — info logged in one process becomes noise in 3-4 |
52| **Third-party lib prints at import time** | Suppress the logger or redirect stdout during that import |
53| **C-level print from .so library** | Redirect fd 1 during the specific C call, or accept it if too invasive |
54| **Real warning the user should see** | Keep it |
55
56### 4. Present findings before fixing
57
58List all noisy lines with their source and proposed fix. Ask the user to review before making changes.
59
60### 5. Apply fixes and verify
61
62After approval, apply fixes one at a time, re-launch the server, and verify each fix works.
63
64## Key Architecture: Why Logs Repeat
65
66`ModelConfig` is constructed **3-4 times** during startup across different processes:
671. Main process: `ServerArgs.__post_init__()` → `get_model_config()` → `ModelConfig()`
682. Scheduler subprocess: `Scheduler.init_model_config()` → `ModelConfig.from_server_args()`
693. Scheduler subprocess: `TpModelWorker._init_model_config()` → `ModelConfig.from_server_args()`
704. Main process: `TokenizerManager.init_model_config()` → `ModelConfig.from_server_args()`
71
72Similarly, `get_tokenizer()` is called **5 times** across processes:
731. `resolve_auto_parsers` (main) — `template_detection.py`
742. `Scheduler.init_tokenizer()` (scheduler subprocess) — `scheduler.py`
753. `DetokenizerManager` (detokenizer subprocess) — `detokenizer_manager.py`
764. `TpModelWorker.__init__()` (scheduler subprocess) — `tp_worker.py`
775. `TokenizerManager` (main) — `tokenizer_manager.py`
78
79Any `logger.info()` or `logger.warning()` in `ModelConfig.__init__()` or `get_tokenizer()` will appear 3-5 times. **Keep these at `logger.debug()`.**
80
81## Known Noise Sources and Fixes (from past sessions)
82
83### 1. torchao "Skipping import of cpp extensions due to incompatible torch version"
84
85- **Source:** `torchao/__init__.py` — printed via `logger.warning()` when torch version < 2.11.0
86- **Trigger:** `sglang/__init__.py` -> `_apply_hf_patches()` -> `_patch_removed_symbols()` -> `from transformers.models.llama import modeling_llama` -> deep import chain -> `transformers/quantizers/auto.py` -> `from .quantizer_torchao import TorchAoHfQuantizer` -> imports torchao
87- **Fix:** In `hf_transformers_patches.py::_patch_removed_symbols()`, temporarily set the `torchao` logger level to `ERROR` around the `modeling_llama` import:
88 ```python
89 _torchao_logger = logging.getLogger("torchao")
90 _prev_level = _torchao_logger.level
91 _torchao_logger.setLevel(logging.ERROR)
92 try:
93 from transformers.models.llama import modeling_llama
94 finally:
95 _torchao_logger.setLevel(_prev_level)
96 ```
97
98### 2. "`torch_dtype` is deprecated! Use `dtype` instead!" (PARTIALLY FIXED)
99
100- **Source:** `transformers/configuration_utils.py` — the `torch_dtype` property warns via `logger.warning_once()`
101- **Trigger:** Model files accessing `config.torch_dtype` instead of `config.dtype`
102- **Fix applied so far:** Only `models/gpt_oss.py` (lines 222, 471) — tested with `openai/gpt-oss-20b`.
103- **Remaining files that still use `config.torch_dtype`** (fix each only after testing with the corresponding model):
104 - `models/bailing_moe.py` (line 302)
105 - `models/llada2.py` (line 313)
106 - `models/qwen3_next.py` (lines 192, 209)
107 - `models/qwen3_5.py` (line 245)
108 - `models/nano_nemotron_vl.py` (lines 79, 102, 284)
109 - `models/llava.py` (lines 732, 734-737)
110 - `model_loader/loader.py` (line 649)
111- **Note:** `common.py` was already fixed in a prior session. If new model files are added with `config.torch_dtype`, the warning will reappear — grep for `\.torch_dtype` to find them.
112- **Important:** Only change `config.torch_dtype` → `config.dtype` for models you have actually tested. The `dtype` property should return the same value, but verify per-model to avoid regressions.
113
114### 3. "`BaseImageProcessorFast` is deprecated"
115
116- **Source:** `transformers/utils/import_utils.py` — the lazy module `__getattr__` warns when `BaseImageProcessorFast` is accessed
117- **Trigger:** `base_processor.py` and `ernie45_vl.py` have `from transformers import BaseImageProcessorFast` at top level. These are imported eagerly via `tokenizer_manager.py` -> `multimodal_processor.py` -> `base_processor.py`, even for non-multimodal models.
118- **Fix:** Replace `from transformers import BaseImageProcessorFast` with `from transformers import BaseImageProcessor` and update all `isinstance(..., BaseImageProcessorFast)` checks to `isinstance(..., BaseImageProcessor)`
119
120### 4. "No platform detected. Using base SRTPlatform with defaults."
121
122- **Source:** `sglang/srt/platforms/__init__.py` — `logger.warning()`
123- **Fix:** Change to `logger.debug()` — this is expected on machines without a platform plugin and not actionable.
124
125### 5. `NCCL version 2.27.7+cuda13.0`
126
127- **Source:** C-level print from `libnccl.so` during `ncclCommInitRank()` call
128- **Status:** Accepted as-is. SGLang already logs the version via `sglang is using nccl==X.Y.Z`. The C-level print cannot be suppressed without redirecting stdout fd, which is too invasive. `NCCL_DEBUG=WARN` does not suppress it in NCCL 2.27+.
129
130### 6. `[Gloo] Rank X is connected to Y peer ranks`
131
132- **Source:** C++ Gloo library print during process group init
133- **Status:** Accepted as-is. From C++ code inside PyTorch's Gloo backend.
134
135### 7. `torchao SyntaxWarning: invalid escape sequence`
136
137- **Source:** `torchao/quantization/quant_api.py` — a raw string with unescaped `\.`
138- **Status:** Upstream torchao bug. Cannot fix from SGLang side.
139
140### 8. tqdm progress bars (e.g., `Multi-thread loading shards`, `Capturing batches`)
141
142- **Status:** These are expected and useful. They show progress during weight loading and CUDA graph capture. Keep them.
143
144### 9. CUTE_DSL "Unexpected error during package walk" — double-logged (FIXED)
145
146- **Source:** `nvidia-cutlass-dsl` package at `.venv/.../cutlass/cutlass_dsl/cutlass.py`, line 391. Logger named `CUTE_DSL` with its own `StreamHandler`.
147- **Trigger:** During CUDA graph capture, cutlass DSL walks packages and hits an unexpected error for `cutlass.cute.experimental`.
148- **Root cause of double-logging:** The CUTE_DSL logger has `propagate=True` (default), so the warning is emitted by both the CUTE_DSL handler (with its format) and the root logger (SGLang's format).
149- **Fix applied:** In `entrypoints/engine.py`, changed `CUTE_DSL_LOG_LEVEL` from `"30"` (WARNING) to `"40"` (ERROR). This suppresses the WARNING at both the CUTE_DSL logger and root propagation levels. The env var controls both `logger.setLevel()` and `console_handler.setLevel()` in cutlass's `setup_log()`.
150
151### 10. ModelConfig init logs repeated 3x (FIXED)
152
153- **Lines:** `"Downcasting torch.float32 to ..."`, `"Hybrid swa model: ..."`, `"DeepGemm is enabled but ..."`
154- **Source:** `configs/model_config.py` — `_get_and_verify_dtype()` (line 1457), `_derive_hybrid_model()` (line 497), `_verify_quantization()` (line 1236)
155- **Root cause:** `ModelConfig.__init__()` is called 3-4 times in different processes (see "Key Architecture" above). Each construction fires the same log lines.
156- **Fix applied:** Downgraded all three from `logger.info()`/`logger.warning()` to `logger.debug()`. The dtype is already visible in `server_args` and `Load weight end`. Hybrid SWA info appears in `Tree cache initialized`. DeepGemm is not actionable.
157
158### 11. Tokenizer retry/fallback messages repeated 3-4x (FIXED)
159
160- **Lines:** `"Tokenizer loaded as generic TokenizersBackend ... retrying"`, `"Loading tokenizer ... directly as PreTrainedTokenizerFast"`, `"Tokenizer for ... loaded as generic TokenizersBackend. Set --trust-remote-code"`
161- **Source:** `utils/hf_transformers/tokenizer.py` — `_resolve_tokenizers_backend()` (line 215), `_load_tokenizer_by_declared_class()` (line 110), final warning (line 244)
162- **Root cause:** 5 separate `get_tokenizer()` calls across processes (see "Key Architecture" above). Each produces 3 log lines. Concurrent subprocess launches cause interleaved/doubled output.
163- **Fix applied:** Downgraded all three from `logger.warning()`/`logger.info()` to `logger.debug()`.
164
165### 12. Template detection logs — 5 lines consolidated to 1 (FIXED)
166
167- **Lines:** `"Detected reasoning config '...' from template rule '...'"`, `"Detected reasoning parser '...' from template rule '...'"`, `"Detected tool-call parser '...' from template rule '...'"`, `"Auto-detected reasoning parser: ..."`, `"Auto-detected tool-call parser: ..."`
168- **Source:** `managers/template_detection.py` (lines 337, 370) logged each detection rule match. `managers/template_manager.py` (lines 177-182) logged summary lines that duplicated the detection logs.
169- **Fix applied:** Removed per-rule logs from `template_detection.py`. Consolidated the 5 lines in `template_manager.py` into a single summary: `"Auto-detected template features: reasoning_config=..., reasoning_parser=..., tool_call_parser=..."`
170
171### 13. KV cache dtype logged separately from allocation (FIXED)
172
173- **Lines:** `"Using KV cache dtype: torch.bfloat16"` then `"KV Cache is allocated. #tokens: ..., K size: ..., V size: ..."`
174- **Source:** `model_executor/model_runner.py` (line 2217) and `mem_cache/memory_pool.py` (line 740)
175- **Fix applied:** Removed the standalone dtype log from `model_runner.py`. Added `dtype` field to the allocation log in `memory_pool.py`: `"KV Cache is allocated. dtype: torch.bfloat16, #tokens: ..., K size: ..., V size: ..."`
176
177### 14. CUTLASS backend warning — B200 → SM100, warning → info (FIXED)
178
179- **Line:** `"CUTLASS backend is disabled when piecewise cuda graph is enabled due to TMA descriptor initialization issues on B200."`
180- **Source:** `layers/attention/flashinfer_backend.py` (line 249)
181- **Fix applied:** Changed "B200" to "SM100 GPUs" (the condition checks `is_sm100_supported()` which matches SM10x, not just B200). Downgraded from `logger.warning()` to `logger.info()` since it's an expected automatic fallback.
182
183### 15. `max_total_num_tokens` and `Tree cache initialized` log ordering
184
185- **Issue:** `max_total_num_tokens=...` appears before `Tree cache initialized:...` even though tree cache is conceptually part of memory setup.
186- **Root cause:** `max_total_num_tokens` is logged inside `init_model_worker()` (scheduler.py:972), which runs before `build_kv_cache()` (scheduler.py:425) where tree cache is created.
187- **Status:** Not fixed — reordering was reverted. Acceptable as-is.
188
189### 16. `Ignore import error when loading sglang.srt.models.midashenglm`
190
191- **Source:** `models/registry.py` (line 109) — `logger.warning()` during `import_model_classes()` which iterates all model modules via `pkgutil.iter_modules`
192- **Trigger:** The `midashenglm` model depends on `torchaudio`, which fails to load
193- **Status:** Should be downgraded to `logger.debug()` — not actionable when loading an unrelated model. Same pattern exists in `managers/multimodal_processor.py`, `dllm/algorithm/__init__.py`, `multimodal_gen/runtime/models/registry.py`.
194
195### 17. `Multiple NUMA nodes found for GPU X`
196
197- **Source:** `utils/numa_utils.py` (line 112) — `logger.warning()`
198- **Status:** Could be downgraded to `logger.info()`. The situation is handled gracefully ("Using the first one") and not actionable.
199
200### 18. Warmup `/model_info` access log
201
202- **Source:** Uvicorn access log, triggered by SGLang's own warmup at `entrypoints/http_server.py` (line 1877)
203- **Status:** SGLang talking to itself. Could suppress uvicorn access logger during warmup, or exclude `/model_info` from warmup access logging.
204
205## Investigation Techniques
206
207### Trace what triggers an import
208```python
209import sys
210_real_import = __builtins__.__import__
211def _tracing_import(name, *args, **kwargs):
212 if 'TARGET_MODULE' in name:
213 import traceback
214 print(f'=== Importing {name} ===')
215 traceback.print_stack()
216 return _real_import(name, *args, **kwargs)
217__builtins__.__import__ = _tracing_import
218```
219
220### Trace what triggers a logger warning
221```python
222import logging, traceback
223class TraceHandler(logging.Handler):
224 def emit(self, record):
225 if 'SEARCH_STRING' in record.getMessage():
226 traceback.print_stack()
227h = TraceHandler()
228h.setLevel(logging.WARNING)
229logging.getLogger('TARGET_LOGGER_NAME').addHandler(h)
230```
231
232### Find C-level prints in .so files
233```bash
234strings /path/to/library.so | grep "SEARCH_STRING"
235```
236
237### Find all config.torch_dtype accesses (for deprecation warning)
238```bash
239grep -rn '\.torch_dtype' python/sglang/srt/models/ python/sglang/srt/model_loader/ python/sglang/srt/utils/hf_transformers/
240```
241
242## Reference: Clean Startup Log (TP=1, Qwen3-8B)
243
244```
245[2026-05-24 00:52:39] Attention backend not specified. Use trtllm_mha backend by default.
246[2026-05-24 00:52:39] TensorRT-LLM MHA only supports page_size of 16, 32 or 64, changing page_size from None to 64.
247[2026-05-24 00:52:40] server_args=ServerArgs(model_path='Qwen/Qwen3-8B', ...)
248[2026-05-24 00:52:40] Multiple NUMA nodes found for GPU 0: [...]. Using the first one.
249[2026-05-24 00:52:42] Using default HuggingFace chat template with detected content format: string
250[2026-05-24 00:52:42] Auto-detected template features: reasoning_config=..., reasoning_parser=qwen3, tool_call_parser=qwen
251[2026-05-24 00:52:50] Init torch distributed begin.
252[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
253[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
254[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
255[2026-05-24 00:52:50] Init torch distributed ends. elapsed=0.21 s, mem usage=0.10 GB
256[2026-05-24 00:52:51] Load weight begin. avail mem=275.75 GB
257[2026-05-24 00:52:51] Found local HF snapshot for Qwen/Qwen3-8B at ...; skipping download.
258Multi-thread loading shards: 100% Completed | 5/5 [00:01<00:00, 2.62it/s]
259[2026-05-24 00:52:54] Load weight end. elapsed=2.62 s, type=Qwen3ForCausalLM, avail mem=260.48 GB, mem usage=15.28 GB.
260[2026-05-24 00:52:54] KV Cache is allocated. dtype: torch.bfloat16, #tokens: 1707904, K size: 117.28 GB, V size: 117.28 GB
261[2026-05-24 00:52:54] Memory pool end. avail mem=25.28 GB
262[2026-05-24 00:52:54] CUTLASS backend is disabled when piecewise cuda graph is enabled due to TMA descriptor initialization issues on SM100 GPUs. Using auto backend instead for stability.
263[2026-05-24 00:52:54] Capture cuda graph begin. This can take up to several minutes. avail mem=24.16 GB
264[2026-05-24 00:52:54] Capture cuda graph bs [1, 2, 4, ...]
265Capturing batches (bs=1 avail_mem=23.56 GB): 100% | 52/52 [00:05<00:00, 10.36it/s]
266[2026-05-24 00:53:00] Capture cuda graph end. Time elapsed: 5.38 s. mem usage=0.60 GB. avail mem=23.56 GB.
267[2026-05-24 00:53:00] Capture piecewise CUDA graph begin. avail mem=23.56 GB
268[2026-05-24 00:53:00] Capture cuda graph num tokens [4, 8, 12, ...]
269Compiling num tokens (num_tokens=4): 100% | 74/74 [00:09<00:00, 7.44it/s]
270Capturing num tokens (num_tokens=4 avail_mem=21.24 GB): 100% | 74/74 [00:07<00:00, 10.44it/s]
271[2026-05-24 00:53:18] Capture piecewise CUDA graph end. Time elapsed: 18.18 s. mem usage=2.32 GB. avail mem=21.24 GB.
272[2026-05-24 00:53:20] Tree cache initialized: source=default impl=RadixCache hybrid_swa=False hybrid_ssm=False hierarchical=False streaming_wrapped=False
273[2026-05-24 00:53:20] max_total_num_tokens=1707904, chunked_prefill_size=16384, max_prefill_tokens=16384, max_running_requests=4096, context_len=40960, available_gpu_mem=21.24 GB
274[2026-05-24 00:53:20] INFO: Started server process [1964249]
275[2026-05-24 00:53:20] INFO: Waiting for application startup.
276[2026-05-24 00:53:20] Using default chat sampling params from model generation config: {'temperature': 0.6, 'top_k': 20, 'top_p': 0.95}
277[2026-05-24 00:53:20] INFO: Application startup complete.
278[2026-05-24 00:53:20] INFO: Uvicorn running on http://127.0.0.1:30000 (Press CTRL+C to quit)
279[2026-05-24 00:53:21] Prefill batch, #new-seq: 1, #new-token: 64, ...
280[2026-05-24 00:53:21] INFO: 127.0.0.1:... - "POST /generate HTTP/1.1" 200 OK
281[2026-05-24 00:53:21] The server is fired up and ready to roll!
282```
283
284Note: `[Gloo]` messages and tqdm progress bars are acceptable. The key is no warnings or deprecation messages from transformers, torchao, or other third-party libraries. The `CUTLASS backend is disabled` message is now `info` level, not a warning.