Bentoml Deploy Scriptgen

Generate a production deploy-script bundle

You are a script generator, not a script author: the .py files under this skill's templates/deploy/ are reviewed, e2e-tested artifacts.

HARD RULE — copy the .py files VERBATIM. Never edit, patch, "improve" or regenerate their code, and never write new script logic. Two rendered files carry everything user-specific: config.yml and README.md. No manifests to write — the bundle renders them from config.yml on every run. If a need cannot be expressed in config.yml, say so, point at kubernetes.manifests_dir (the documented escape hatch), and report it as a limitation. Never fork the templates.

This bundle is also the interactive path's renderer and validator: bentoml-k8s-deploy copies templates/deploy/ with the Step 2 commands, then calls deploy.py --target k8s --render-only (and --check-only). _internal/render.py + _internal/config.py are the single source of truth for both flows; no second renderer exists. Two config layouts, both exercised:

• bundle (this skill): deploy/config.yml with project: ..; python3 deploy/deploy.py --target k8s [--render-only] finds it beside deploy.py.
• root-level: config.yml at the project root with project: ., passed explicitly: python3 deploy/deploy.py --target k8s --render-only k8s --config config.yml.

Relative paths (project, kubernetes.manifests_dir) resolve against the config file's own directory; both layouts behave identically.

config.yml v4 is overrides-only: no service list, no entry: flag, no depends:. Service list, entry service and dependency DAG come from the bento's own bento.yaml at RUN time — the local bento store after the build, docker run --rm --entrypoint cat <image> $BENTO_PATH/bento.yaml under --skip-build, or the deploy/.bento-topology.json cache. Nothing to keep in sync with the code. Image tag = bento version; ECR recognized from the image URL's host; cross-architecture builds auto-detected. Four values are enough to deploy: project, image, kubernetes.context, kubernetes.namespace.

Runtime behaviour:

Step 0 — Locate the project and confirm scope

Find the BentoML project (service.py / bentofile.yaml; detection as in bentoml-containerize). The bundle goes into <project>/deploy/; if that exists, show its contents and get explicit confirmation before overwriting. Both k8s and ec2 produce working scripts. Prerequisites the script cannot create:

An older bundle (deploy/deploy.config.json + hand-owned manifests in deploy/k8s/, or a config.yml stamped bentoml-deploy-config/v3) means regeneration, not migration: carry the settings over, drop everything v4 derives. Pre-v4 configs are rejected with exit 2 and a message naming what moved. Old hand-edited manifests still work via kubernetes.manifests_dir; rendering is the default and the recommendation.

Step 1 — Gather parameters (one round of questions)

Detect what you can, then ask the rest in one round; conventions as in the interactive deploy skills. Read service.py for the entry service class, its @bentoml.api methods (the smoke test) and any @bentoml.service(resources=...) worth mirroring; run kubectl config get-contexts — never assume the current context. Never ask about topology: no service list, entry flag, depends, rollout order, dependency URL or slug goes into the config.

Only four values are REQUIRED:

Everything else is an override with a working default — ask only about what the user wants to change:

No platform or registry-type question — both derived. Arch mismatches (the exec format error class) are detected and cross-built automatically; buildx presence is preflighted.

Which config file to write

Both templates ship; make exactly one of them deploy/config.yml, and say which and why:

• config.minimal.yml — every default accepted, nothing beyond the four required values. Four lines, no comments. Say plainly it cannot express a NodePort, Ingress, autoscaling, per-service resources or a smoke test: all-defaults means ClusterIP verified through a port-forward.
• config.yml (annotated) — anything else customized, mirrored decorator resources and NodePort/smoke-test requests included; where most multi-service bentos land. Prune inapplicable blocks and ec2:/kubernetes: sections, keep the comments.

EC2 target parameters

Same round as bentoml-ec2-deploy's Step 0, minus provisioning; all keys sit in the top-level ec2: section. Arch needs no question: preflight compares each host's uname -m with the builder's, cross-builds on a mismatch, and fails before anything mutates if two hosts disagree.

Step 2 — Copy the bundle VERBATIM and render the config

mkdir -p <project>/deploy
cp -R <this-skill>/templates/deploy/. <project>/deploy/
# MANDATORY: drop bytecode caches. Running the templates in place (e2e tests, a stray
# `python3 -m py_compile`) leaves __pycache__/ in the skill; `cp -R` would copy it into
# the user's repo as stale, committable dirt.
find <project>/deploy -name __pycache__ -type d -prune -exec rm -rf {} +
find <project>/deploy -name '*.pyc' -delete
# Verify the copy before rendering; caches on EITHER side would show up as spurious
# differences, hence the exclude. Must report NO differences at all.
diff -r --exclude=__pycache__ <this-skill>/templates/deploy <project>/deploy
# Pick ONE config template, delete the other. EITHER all defaults accepted, the 4-line
# file overwriting the annotated one:
mv <project>/deploy/config.minimal.yml <project>/deploy/config.yml
# OR anything customized, keeping the annotated config.yml:
rm <project>/deploy/config.minimal.yml

Afterwards the only differences from the templates may be config.yml, the removed config template, and README.md. Every .py file stays byte-identical forever, and no k8s/ dir is ever added.

Render placeholders in exactly two files — deploy/config.yml and deploy/README.md — file by file. Never a blanket sed across the bundle: the .py files must stay byte-identical, and some carry literal {{SERVICE_NAME}}-style text in comments a global sed would corrupt.

• Both config templates: {{IMAGE_URL}}, {{K8S_CONTEXT}}, {{NAMESPACE}}.
• Annotated only: {{ENTRY_SERVICE_NAME}}, {{DEP_SERVICE_NAME}}, {{INFERENCE_PATH}}, {{INFERENCE_BODY_TEXT}}, {{EXPECT_SUBSTRING}}, {{EC2_*}}, {{CONTAINER_NAME}}.
• README.md: the same three, plus {{SERVICE_NAME}} (its title — the bento name) and {{ENTRY_SERVICE_SLUG}} / {{DEP_SERVICE_SLUG}} for the rollback recipe (derived slugs: service names snake_cased, _→-). The literal {{PLACEHOLDER}} in its preflight prose is prose — leave it.

Rules for the two rendered files:

Validate: python3 -c "import yaml,sys;yaml.safe_load(open('deploy/config.yml'))" parses; grep -nE '\{\{[A-Z][A-Z0-9_]*\}\}' deploy/config.yml deploy/README.md finds nothing (it matches generator placeholders only, not the ${{ secrets.* }} expressions in the README's GitHub Actions examples); find deploy -name __pycache__ -o -name '*.pyc' is empty.

Step 3 — Render the manifests and review them (k8s target only)

Never hand-write manifests, and never copy bentoml-k8s-deploy's templates in: deploy/_internal/render.py produces those shapes from config.yml. Skip this step for an ec2-only bundle.

# --render-only needs two things. (1) An image ref for the Deployments: --version (with
# `image:`), --image REF, or a git checkout (default tag = short git SHA); nothing is
# built, the ref is only written into the YAML. (2) The topology, from the image (docker
# pulls/reads it) or from deploy/.bento-topology.json; if the image does not exist yet,
# build the bento once (`bentoml build`) or run a full deploy first -- the failure
# message names all three sources.
python3 deploy/deploy.py --target k8s --render-only --version review
kubectl --context <ctx> apply --dry-run=client -f deploy/rendered   # optional but cheap

Confirm what only the user knows: resources per service, exposure. The topology is not theirs to confirm — it came from the bento — but show it: --render-only prints the derived rollout order, and the log line names which source answered. BENTOML_SERVE_DEPENDS on non-leaf services is the renderer's property, not anything the user wrote.

• The deploy path renders in memory and never reads deploy/rendered/; delete it after the review. Re-rendering prunes stale files it owns (renamed slug, deleted service, HPA off) and warns about files it does not.
• The render also (re)writes deploy/.bento-topology.json whenever it discovers the topology freshly.
• Rendered objects carry app.kubernetes.io/managed-by: bentoml-k8s-deploy whichever path applied them, so interactive-then-CI cannot drift and no label is ever rewritten.

Step 4 — Prove the bundle works, then hand it over

Commit the bundle first (git add deploy/ && git commit): the image tag defaults to the project's short git SHA, so an uncommitted deploy/ makes the gate warn working tree is dirty — the git-SHA image tag will not uniquely identify this build, and the build would carry a stale SHA. To review first, run the gate anyway and say the warning is expected until they commit. Run it as CI would, once per generated target:

python3 deploy/deploy.py --target k8s --check-only
python3 deploy/deploy.py --target ec2 --check-only        # connects to every host over SSH

The full ec2 --check-only needs the SSH key, per-host reachability, any env_names values exported, and (with ecr-token-over-ssh) AWS credentials; it probes each host's docker daemon, arch and host port in one SSH round trip per host, changing nothing.

Show the check list and the summary JSON; it must exit 0 before you hand off. Lacking credentials, degrade explicitly and name what was skipped (the summary's skipped_checks lists it):

• no bentoml CLI / registry credentials, cluster access works: python3 deploy/deploy.py --target k8s --check-only --skip-build --image <known-ref>
• no cluster/host/AWS or registry access at all (CI PR gate, or ec2 without AWS credentials here): python3 deploy/deploy.py --target <k8s|ec2> --check-only --local-only

If a check fails, fix the environment or config it names — never the scripts. Then tell the user: