agentic-in Inference
Use this built-in skill when the user needs to run, serve, benchmark, or operationalize model inference.
Core rules
- Separate model choice, hardware fit, serving stack, and request pattern.
- Make latency, throughput, memory, and quality tradeoffs explicit.
- Inspect the real deployment target before recommending a stack.
- Prefer reproducible launch and benchmark steps over hand-wavy performance advice.
Default workflow
- Identify the model, hardware, traffic pattern, and latency target.
- Choose the serving path that fits the deployment constraints.
- Validate startup, request shape, batching, and resource usage.
- Benchmark and tune only after the baseline path is stable.
Guardrails
- Do not imply a model will fit on hardware you have not checked.
- Do not conflate prototype notebook inference with production serving.
- Do not quote performance numbers without workload context.