Add a New Model — Inference Services
Scaffold all the files needed for a new model server.
Steps
- Create
src/models/<name>/model_server/directory. - Create
src/models/<name>/model_server/model.py— subclasskserve.Modelwithload(),preprocess(),predict(), and aif __name__ == "__main__"block that readsMODEL_NAMEandMODEL_PATHfrom the environment and startskserve.ModelServer().start([model]). Reference an existing model likerevert_risk_modelfor the pattern. - Create
src/models/<name>/model_server/requirements.txtwith the model's Python dependencies. The sharedpython/requirements.txtis always installed alongside. - Copy and adapt an existing
blubber.yamlto.pipeline/<name>/. Use a CPU model (e.g.,revertrisk/revertrisk.yaml) as the template unless this model needs a GPU/pyTorch base image. - Add the service to
docker-compose.yml:platform: linux/amd64, build from.pipeline/<name>/blubber.yaml, expose port8080, setMODEL_NAME, mount/mnt/models/from aPATH_TO_<NAME>_MODELenv var. - Add two entries in
.pipeline/config.yaml: a<name>pipeline (test + production stages) and a<name>-publishpipeline (publish to registry). - Create unit tests in
test/unit/<name>/. - Wire CI triggers in the
integration/configrepo:jjb/project-pipelines.yamlandzuul/layout.yaml. - If the model needs local Makefile support, add a target to the
Makefile.
Notes
- Model artifacts are hosted at
https://analytics.wikimedia.org/published/wmf-ml-models/. - The shared Python library at
python/(decorators, preprocess utils, metrics) is available to all models. - Most models need a
.envfile locally withPATH_TO_<NAME>_MODELpointing to downloaded model files.
Input
$ARGUMENTS — the model name in kebab-case (e.g., article-quality, revert-risk-wikidata).