Run AReno Serving
Read AGENTS.md, CODEMAP.md, and current areno serve --help. Serving uses --model-path, not training's --ckpt.
For a remote model reference, pass --model-hub modelscope. If ModelScope
cannot resolve it, report that failure or request a local model path instead of
silently changing hubs.
Workflow
- Record commit and inspect
areno env --json, areno check, GPUs, model config, and adapter registration.
- Validate TP/world-size constraints from current model code.
- Start with requested cache/context, attention backend, and CUDA graph policy. Do not disable graphs merely to hide capture failures.
- Probe model listing and chat:
python .agents/skills/areno-run-serving/scripts/probe_server.py \
--base-url http://127.0.0.1:8000 [--model <name>]
- Build image requests with scripts/build_image_request.py, piping JSON to the HTTP client rather than shell argv.
- When relevant, validate streaming, cancellation followed by a clean request, image input, and tool calls. Read references/request-contracts.md.
- Classify request, processor, model, cache, scheduler, or transport ownership before editing.
Report command, commit, endpoint, successful probes, and shutdown state. Startup alone is insufficient.
1---2name: areno-run-serving3description: Start, validate, debug, and stop an AReno OpenAI-compatible serving endpoint. Use for areno serve commands, API probes, streaming, cancellation, cache or CUDA graph issues, and supported image or tool-call requests. Do not use for training jobs.4---56# Run AReno Serving78Read `AGENTS.md`, `CODEMAP.md`, and current `areno serve --help`. Serving uses `--model-path`, not training's `--ckpt`.910For a remote model reference, pass `--model-hub modelscope`. If ModelScope11cannot resolve it, report that failure or request a local model path instead of12silently changing hubs.1314## Workflow15161. Record commit and inspect `areno env --json`, `areno check`, GPUs, model config, and adapter registration.172. Validate TP/world-size constraints from current model code.183. Start with requested cache/context, attention backend, and CUDA graph policy. Do not disable graphs merely to hide capture failures.194. Probe model listing and chat:2021```bash22python .agents/skills/areno-run-serving/scripts/probe_server.py \23 --base-url http://127.0.0.1:8000 [--model <name>]24```25265. Build image requests with [scripts/build_image_request.py](scripts/build_image_request.py), piping JSON to the HTTP client rather than shell argv.276. When relevant, validate streaming, cancellation followed by a clean request, image input, and tool calls. Read [references/request-contracts.md](references/request-contracts.md).287. Classify request, processor, model, cache, scheduler, or transport ownership before editing.2930Report command, commit, endpoint, successful probes, and shutdown state. Startup alone is insufficient.
inclusionai