Bentoml Containerize

Containerize a BentoML project and push it to a registry

Build the user's project into a Bento, containerize it, smoke-test the container, push it. Output: a pushed image reference for bentoml-k8s-deploy or bentoml-ec2-deploy. Steps in order; never skip the smoke test.

For production / CI-CD, generate a standalone script bundle (no agent at deploy time) with bentoml-deploy-scriptgen.

references/bentoml-workflow.md is the map of the whole pipeline: project -> runtime environment -> Bento -> image -> deploy target, with the local store paths and the container conventions every later step relies on.

Step 0 — Preflight checks

command -v bentoml && bentoml --version                      # 1. BentoML CLI
docker info --format '{{.ServerVersion}} {{.Architecture}}'  # 2. docker daemon (containerize's default backend)
ls service.py bentofile.yaml 2>/dev/null                     # 3. locate the project
find . -maxdepth 3 -name service.py -not -path '*/.venv/*'   #    ...if not in the current dir

• Missing CLI: pip install bentoml into the user's active environment.
• Docker unreachable: user starts Docker (Docker Desktop / colima); stop until it is. Note Architecture for Step 3: kernel names, x86_64 = amd64, aarch64 = arm64; container platforms use the latter (linux/amd64).
• Build context = directory holding service.py; run later commands there. If the project has neither bentofile.yaml nor an image= spec, create one in Step 1.

Ask both questions in one round:

1. Which registry? Docker Hub, GHCR, ECR, a private registry, kind/minikube local load (no registry), or ttl.sh.
2. Target CPU architecture (cluster nodes / EC2 instance type), amd64 or arm64? Most clouds are linux/amd64, Apple Silicon defaults to arm64, and a mismatch gives exec format error on the target.

Step 1 — Verify / complete the runtime environment spec

Read service.py; confirm a class decorated with @bentoml.service. Its snake_case name is the default Bento name (MyService → my_service).

Declare every Python dependency the service imports, in one of two places:

• In code, preferred for new projects: bentoml.images.Image(python_version="3.11").python_packages(...) passed as @bentoml.service(image=my_image).
• Or bentofile.yaml beside service.py: service: "service:MyService", include: ["*.py"], python: {packages: [...]}.

Add what the imports need and the spec lacks; bentoml is always included. Full Image API (requirements_file(), system_packages(), run(), base_image, distro, lockfiles), bentofile.yaml fields, env vars and models: references/runtime-environment.md.

.bentoignore: bentoml build packages everything under the build context matching include (default *), minus always-excluded .git/, .venv/, venv/, __pycache__/, .DS_Store. Add gitignore-style patterns (data/, checkpoints/, *.ipynb) for large dirs.

Secrets: never give envs a secret value (HF_TOKEN, API keys) in bentofile.yaml or the decorator; values are baked into image layers. Name only, injected at runtime (Step 4 locally, Kubernetes Secret later).

Step 2 — Build the Bento

Build once from the build context, capturing the tag; -o tag prints one __tag__:name:version line. No plain bentoml build first: every build makes a new bento with a fresh version.

BENTO_TAG=$(bentoml build -o tag | grep '^__tag__:' | sed 's/^__tag__://')
echo "$BENTO_TAG"    # e.g. summarization:6oxk5qvott3lsnry
# bentoml list shows Bentos newest first (-o json for scripting); bentoml get "$BENTO_TAG" -o json inspects one
# Bentos live in ~/bentoml/bentos/<name>/<version>/, models in ~/bentoml/models/ (BENTOML_HOME to relocate)

Empty BENTO_TAG: rerun plain bentoml build (no -o tag) for the full error. Usually a service: entry not matching the real module/class name, a package missing at import time, or a model download failure (gated Hugging Face models need export HF_TOKEN=... in this shell).

Step 3 — Containerize

Compose $IMAGE from the chosen registry's pattern in references/registries.md BEFORE running this.

# IMAGE = full registry reference, pushed in Step 5, e.g. docker.io/<user>/summarization:6oxk5qvott3lsnry
bentoml containerize "$BENTO_TAG" -t "$IMAGE"

• Without -t, the image is tagged as the bento tag. With -t (preferred, skips a docker tag) it is tagged ONLY $IMAGE; no $BENTO_TAG image is created.
• Docker image names must be lowercase.
• The image tag must be the bento version. The deploy skills' config.yml holds image: as a bare URL with NO tag and appends the version, so $IMAGE = <URL for config.yml>:<bento version>. Any other tag deploys an image that was never pushed. The registry URL is a user-provided dependency (one their cluster can pull from); the tag is not a choice.

Exception, kind/minikube (no registry): omit -t, nothing is pushed. The image is named exactly $BENTO_TAG, which is what you load into the cluster.

bentoml containerize "$BENTO_TAG"
IMAGE="$BENTO_TAG"

Cross-architecture: when the build machine's arch differs from the cluster, pass the target platform (--platform=linux/amd64 is an accepted legacy spelling); see references/cross-platform.md.

bentoml containerize "$BENTO_TAG" -t "$IMAGE" --opt platform=linux/amd64

Containerize bakes every referenced model into the image (BentoModel, HuggingFaceModel): large image, long first build for LLM-sized models; gated HF models again need HF_TOKEN exported here.

Step 4 — Smoke test locally with docker run

The entrypoint serves automatically, no command needed. Skip this step only when the image platform doesn't match the local machine (cross-arch build), and say so. Container port is always 3000; map it to an uncommon host port (3007 below), since dev servers often hold 3000.

docker rm -f bento-smoke 2>/dev/null || true
# Add -e NAME=value per runtime env var (e.g. models downloaded at startup).
docker run -d --name bento-smoke -p 3007:3000 "$IMAGE" || { echo "docker run failed"; exit 1; }

# Confirm it is up BEFORE curling — another process on the host port yields a false READY.
docker ps --filter name=bento-smoke --format '{{.Status}}'   # must show "Up ..."

# Wait for readiness (model loading can take minutes for big models)
for i in $(seq 1 60); do
  if curl -sf http://localhost:3007/readyz > /dev/null; then echo READY; break; fi
  sleep 5
done
curl -s -o /dev/null -w '%{http_code}\n' http://localhost:3007/readyz   # expect 200

docker run fails or /readyz never returns 200: read docker logs bento-smoke. Missing dependency → Step 1, missing env var → add -e, port conflict → another host port (container port stays 3000). Clean up afterwards even on failure.

Then exercise one real endpoint. Mandatory: a 200 on /readyz proves neither that this is your container nor that the model loaded. Take route and payload from the @bentoml.api methods in service.py (POST /<method_name>, JSON body of the parameters) and judge the response body, not the status code.

curl -s -X POST http://localhost:3007/<method_name> \
  -H 'Content-Type: application/json' \
  -d '{"<param>": <value>}'
docker rm -f bento-smoke      # clean up

Step 5 — Push to the user's registry

Logins, naming rules, visibility gotchas (GHCR packages default to private!) and no-registry paths (kind load / minikube image load, ttl.sh): references/registries.md. Generic flow once -t named the image:

docker login <registry-host>       # per-registry specifics in the reference
docker push "$IMAGE"

Verify: push output ends with a digest (or docker manifest inspect "$IMAGE").

Step 6 — Hand off to a deploy skill

• bentoml-k8s-deploy — one deploy/config.yml rendering one Deployment + Service per BentoML service the bento declares.
• bentoml-ec2-deploy — the pushed image (typically ECR) on EC2 with docker.

Report and pass to the deploy skill:

Service topology is informational; the deploy skill reads bento.yaml itself (local bento store, or from the image under --skip-build). Report it anyway: number of @bentoml.service classes, the entry service, the dependency edges, from bento.yaml inside the image (entry_service, services[].name, services[].dependencies[].service) rather than guessed from service.py. Take BENTO_PATH from the image, never hard-coded: a custom base image can move it.

BENTO_PATH=$(docker inspect -f '{{range .Config.Env}}{{println .}}{{end}}' "$IMAGE" \
    | sed -n 's/^BENTO_PATH=//p' | head -n1)
docker run --rm --entrypoint cat "$IMAGE" "${BENTO_PATH:-/home/bentoml/bento}/bento.yaml"

Do not suggest Kubernetes object names: derived per BentoML service from the bento's own service names, an invented name is ignored or wrongly written into the config. Never derive one from the image repository, where a ttl.sh repo is a random UUID that may start with a digit, invalid as a DNS-1035 Service name.

Cleaning up (optional)

When done iterating:

docker rmi "$IMAGE"              # plus docker rmi "$BENTO_TAG" if built without -t
bentoml delete "$BENTO_TAG" -y
# ECR repositories bill for storage:
aws ecr delete-repository --repository-name <repo-used-in-Step-5> --region "$AWS_REGION" --force

ttl.sh images expire on their own; Docker Hub/GHCR images go via their web UIs.